Peephole LSTM Error

[richagadgil]

The peephole is causing this error in line 1095

auto spph = m.insert_instruction(ins, make_op("squeeze", {{"axes", {0}}}), pph);

Pass: rewrite_rnn
terminate called after throwing an instance of 'migraphx::version_2_12_0::exception'
  what():  /code/AMDMIGraphX/src/normalize_attributes.cpp:120: tune_attribute: squeeze: axes: value out of range!
Aborted (core dumped)

Printing out the pph instruction.get_shape() value gives me: @literal half_type, {}, {}

Quantization is causing this to be an empty parameter instead of undefined.

[codecov]

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 92.29%. Comparing base (810b3f2) to head (b2c7d09).
Report is 6 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/propagate_constant.cpp	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3768      +/-   ##
===========================================
- Coverage    92.29%   92.29%   -0.01%     
===========================================
  Files          519      519              
  Lines        22217    22219       +2     
===========================================
+ Hits         20505    20506       +1     
- Misses        1712     1713       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[richagadgil]

This fix works to compile the model and run perf, but there is still an issue with the .mxr files once compiled:

FP32:

[ MIGraphX Version: 2.12.0.a9919ec08 ] Complete: /code/AMDMIGraphX/build/bin/migraphx-driver compile model.onnx --input-dim @symbol 1 1 @hidden_in_1 2 1 320 @hidden_in_2 2 1 320 --fill1 symbol -o model_fp32.mxr

[ MIGraphX Version: 2.12.0.a9919ec08 ] Complete: /code/AMDMIGraphX/build/bin/migraphx-driver perf model_fp32.mxr --fill1 symbol

FP16:

[ MIGraphX Version: 2.12.0.a9919ec08 ] Complete: /code/AMDMIGraphX/build/bin/migraphx-driver compile model.onnx --input-dim @symbol 1 1 @hidden_in_1 2 1 320 @hidden_in_2 2 1 320 --fill1 symbol --fp16 -o model_fp16.mxr

Running [ MIGraphX Version: 2.12.0.a9919ec08 ]: /code/AMDMIGraphX/build/bin/migraphx-driver perf model_fp16.mxr --fill1 symbol
Compiling ... 
Reading: model_fp16.mxr
terminate called after throwing an instance of 'msgpack::v1::insufficient_bytes'
  what():  insufficient bytes
Aborted (core dumped)

[richagadgil]

log.txt

MIGRAPHX_TRACE_COMPILE=1 /code/AMDMIGraphX/build/bin/migraphx-driver perf model.onnx --input-dim @symbol 1 1 @hidden_in_1 2 1 320 @hidden_in_2 2 1 320 --fill1 symbol --fp16 -o model_fp16.mxr

[richagadgil]

Changed the PR to skip const prop over undefined instructions instead.

[migraphx-bot]

Test	Batch	Rate new b2c7d0	Rate old 810b3f	Diff	Compare
torchvision-resnet50	64	3,235.00	3,227.48	0.23%	✅
torchvision-resnet50_fp16	64	6,871.61	6,935.80	-0.93%	✅
torchvision-densenet121	32	2,437.35	2,443.57	-0.25%	✅
torchvision-densenet121_fp16	32	4,195.05	4,190.37	0.11%	✅
torchvision-inceptionv3	32	1,613.71	1,618.09	-0.27%	✅
torchvision-inceptionv3_fp16	32	2,688.06	2,703.46	-0.57%	✅
cadene-inceptionv4	16	750.24	754.54	-0.57%	✅
cadene-resnext64x4	16	809.07	799.94	1.14%	✅
slim-mobilenet	64	6,663.18	7,038.75	-5.34%	🔴
slim-nasnetalarge	64	199.01	203.74	-2.32%	✅
slim-resnet50v2	64	3,427.87	3,418.15	0.28%	✅
bert-mrpc-onnx	8	1,144.73	1,130.84	1.23%	✅
bert-mrpc-tf	1	482.86	483.04	-0.04%	✅
pytorch-examples-wlang-gru	1	484.98	471.20	2.92%	✅
pytorch-examples-wlang-lstm	1	464.03	446.88	3.84%	🔆
torchvision-resnet50_1	1	802.85	802.10	0.09%	✅
cadene-dpn92_1	1	434.06	426.83	1.69%	✅
cadene-resnext101_1	1	389.88	383.89	1.56%	✅
onnx-taau-downsample	1	373.57	372.62	0.26%	✅
dlrm-criteoterabyte	1	31.81	32.54	-2.24%	✅
dlrm-criteoterabyte_fp16	1	51.06	51.07	-0.03%	✅
agentmodel	1	8,660.86	8,676.04	-0.17%	✅
unet_fp16	2	58.09	57.90	0.33%	✅
resnet50v1_fp16	1	1,027.11	1,041.98	-1.43%	✅
resnet50v1_int8	1	786.40	779.61	0.87%	✅
bert_base_cased_fp16	64	1,172.26	1,172.53	-0.02%	✅
bert_large_uncased_fp16	32	362.53	362.45	0.02%	✅
bert_large_fp16	1	199.73	200.21	-0.24%	✅
distilgpt2_fp16	16	2,219.27	2,216.14	0.14%	✅
yolov5s	1	526.82	518.19	1.67%	✅
tinyllama	1	43.62	43.59	0.05%	✅
vicuna-fastchat	1	175.74	174.63	0.63%	✅
whisper-tiny-encoder	1	412.08	406.05	1.49%	✅
whisper-tiny-decoder	1	412.19	398.24	3.50%	🔆

This build is not recommended to merge 🔴

[migraphx-bot]

     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     ✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

     ✅ unet: PASSED: MIGraphX meets tolerance

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance