本教程详细记录了使用周易 AIPU 进行模型推理部署。
以 resnet_v1_152 为例
1.下载 resnet_v1_152 预训练模型:
wget http://download.tensorflow.or...
2.export graph
vim export_graph.sh
加入以下内容
python3 export_inference_graph.py \
--alsologtostderr \
--model_name=resnet_v1_152 \
--image_size=224 \
--labels_offset=1 \
--output_file=tmp/resnet_v1_152_inf.pb执行 ./export_graph.sh 后将在 tmp文件夹下生成 resnet_v1_152_inf.pb
3.freeze graph
vim freeze_graph.sh
加入以下内容：
python3 freeze_graph.py \
--input_graph=tmp/resnet_v1_152_inf.pb \
--input_checkpoint=tmp/resnet_v1_152.ckpt \
--input_binary=true --output_graph=tmp/resnet_v1_152_frozen.pb \
--output_node_names= resnet_v1_152/predictions/Reshape_1
执行 ./freeze_graph.sh 后将在 tmp 文件夹下生成 resnet_v1_152_frozen.pb
root@10c4628e25a5:~/demos/resnet152pb/tmp# ll
total 1651900
drwxrwxrwx 3 root root 4096 Jul 16 05:50 ./
drwxrwxrwx 11 root root 4096 Jul 16 05:50 ../
drwxrwxrwx 2 root root 4096 Jul 16 04:25 04:16:37/
-rw-r----- 1 388142 5762 6759425 Apr 14 2017 eval.graph
-rwxrwxrwx 1 root root 76 Jul 16 04:28 get_resnet_v1_152_ckpt.sh*
-rw-r----- 1 388142 5762 724550366 Apr 14 2017 resnet_v1_152.ckpt
-rw-r--r-- 1 root root 675629399 Apr 19 2017 resnet_v1_152_2017_04_14.tar.gz
-rw-r--r-- 1 root root 241994757 Jul 16 05:46 resnet_v1_152_frozen.pb
-rw-r--r-- 1 root root 1415702 Jul 16 05:45 resnet_v1_152_inf.pb
-rw-r----- 1 388142 5762 41160257 Apr 14 2017 train.graph4.准备量化校准数据集
这里使用了镜像提供的图片和标签。
root@10c4628e25a5:~/demos/resnet152pb/dataset/img# ll
total 588
drwxrwxrwx 2 root root 4096 Jul 16 04:25 ./
drwxrwxrwx 3 root root 4096 Jul 16 04:25 ../
-rwxrwxrwx 1 root root 109527 Jul 16 04:25 ILSVRC2012_val_00000001.JPEG*
-rwxrwxrwx 1 root root 140296 Jul 16 04:25 ILSVRC2012_val_00000002.JPEG*
-rwxrwxrwx 1 root root 122660 Jul 16 04:25 ILSVRC2012_val_00000003.JPEG*
-rwxrwxrwx 1 root root 84885 Jul 16 04:25 ILSVRC2012_val_00000004.JPEG*
-rwxrwxrwx 1 root root 130340 Jul 16 04:25 ILSVRC2012_val_00000005.JPEG*制作量化校准数据集：
python3 preprocess_dataset.py
执行如上操作会生成 dataset.npy 和 label.npy
root@10c4628e25a5:~/demos/resnet152pb/dataset# ll
total 2972
drwxrwxrwx 3 root root 4096 Jul 16 04:25 ./
drwxrwxrwx 11 root root 4096 Jul 16 05:50 ../
-rwxrwxrwx 1 root root 3010688 Jul 16 05:46 dataset.npy*
drwxrwxrwx 2 root root 4096 Jul 16 04:25 img/
-rwxrwxrwx 1 root root 168 Jul 16 05:46 label.npy*
-rwxrwxrwx 1 root root 163 Jul 16 04:25 label.txt*
-rwxrwxrwx 1 root root 5374 Jul 16 04:25 preprocess_dataset.py*5.NN Compiler 配置
在 config 目录下创建 resnet_v1_152_build.cfg 和 resnet_v1_152_run.cfg 配置文件。
resnet_v1_152_build.cfg 内容如下：
[Common]
mode=build
[Parser]
model_name = resnet_152
detection_postprocess =
model_domain = image_classification
output = resnet_v1_152/predictions/Reshape_1
input_model = ./tmp/resnet_v1_152_frozen.pb
input = input
input_shape = [1,224,224,3]
[AutoQuantizationTool]
model_name = resnet_152
quantize_method = SYMMETRIC
ops_per_channel = DepthwiseConv
calibration_data = ./dataset/dataset.npy
calibration_label = ./dataset/label.npy
preprocess_mode = normalize
quant_precision=int8
reverse_rgb = False
label_id_offset = 0
[GBuilder]
outputs=./aipu.bin
profile= True
target=Z1_0701resnet_101_run.cfg 内容如下：
[Common]
mode=run
[Parser]
model_name = resnet_152
detection_postprocess =
model_domain = image_classification
output = resnet_v1_152/predictions/Reshape_1
input_model = ./tmp/resnet_v1_152_frozen.pb
input = input
input_shape = [1,224,224,3]
output_dir = ./
[AutoQuantizationTool]
model_name = resnet_152
quantize_method = SYMMETRIC
ops_per_channel = DepthwiseConv
calibration_data = ./dataset/dataset.npy
calibration_label = ./dataset/label.npy
preprocess_mode = normalize
quant_precision=int8
reverse_rgb = False
label_id_offset = 0
[GBuilder]
inputs=./model/input.bin
simulator=aipu_simulator_z1
outputs=output_resnet_v1_152.bin
profile= True
target=Z1_07016.AIPU 仿真
aipubuild confg/resnet_v1_152_build.cfg
root@10c4628e25a5:~/demos/resnet152pb# aipubuild config/resnet_v1_152_build.cfg
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
* https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
* https://github.com/tensorflow/addons
* https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.
[I] Parsing model....
[I] [Parser]: Begin to parse tensorflow model resnet_152...
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/AIPUBuilder/Parser/passes/tf/__init__.py:73: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/AIPUBuilder/Parser/passes/tf/__init__.py:73: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2021-07-16 06:15:14.457980: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-07-16 06:15:14.478773: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3399905000 Hz
2021-07-16 06:15:14.478955: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x6e5a060 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-07-16 06:15:14.478969: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
[I] [Parser]: Bias Op (resnet_v1_152/conv1/BiasAdd[?,112,112,64]) is fused into Linear Op (resnet_v1_152/conv1/Conv2D[?,112,112,64])!
[I] [Parser]: Parser done!
[I] Parse model complete
[I] Quantizing model....
[I] AQT start: model_name:resnet_152, calibration_method:MEAN, batch_size:1
[I] ==== read ir ================
[I] float32 ir txt: /tmp/AIPUBuilder_1626416111.1700537/resnet_152.txt
[I] float32 ir bin2: /tmp/AIPUBuilder_1626416111.1700537/resnet_152.bin
[I] ==== read ir DONE.===========
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.
[I] ==== auto-quantization ======
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and:
`tf.data.TFRecordDataset(path)`
WARNING:tensorflow:Entity <bound method ImageNet.data_transform_fn of <AIPUBuilder.AutoQuantizationTool.auto_quantization.data_set.ImageNet object at 0x7f102ccc1940>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: <cyfunction ImageNet.data_transform_fn at 0x7f106d09ad38> is not a module, class, method, function, traceback, frame, or code object
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py:330: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py:330: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py:915: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead.
[I] step1: get max/min statistic value DONE
[I] step2: quantization each op DONE
[I] step3: build quantization forward DONE
[I] step4: show output scale of end node:
[I] layer_id:314, layer_top:resnet_v1_152/predictions/Reshape_1_0, output_scale:[0.07918436]
[I] ==== auto-quantization DONE =
[I] Quantize model complete
[I] Building ...
[I] [common_options.h: 276] BuildTool version: 4.0.175. Build for target Z1_0701 at frequency 800MHz
[I] [common_options.h: 297] using default profile events to profile AIFF
[I] [IRChecker] Start to check IR: /tmp/AIPUBuilder_1626416111.1700537/resnet_152_int8.txt
[I] [IRChecker] model_name: resnet_152
[I] [IRChecker] IRChecker: All IR pass
[I] [graph.cpp : 846] loading graph weight: /tmp/AIPUBuilder_1626416111.1700537/resnet_152_int8.bin size: 0x39d4128
[I] [builder.cpp:1059] Total memory for this graph: 0x4128770 Bytes
[I] [builder.cpp:1060] Text section: 0x00086170 Bytes
[I] [builder.cpp:1061] RO section: 0x00010900 Bytes
[I] [builder.cpp:1062] Desc section: 0x00020400 Bytes
[I] [builder.cpp:1063] Data section: 0x039a0d00 Bytes
[I] [builder.cpp:1064] BSS section: 0x00690800 Bytes
[I] [builder.cpp:1065] Stack : 0x00040400 Bytes
[I] [builder.cpp:1066] Workspace(BSS): 0x000c4000 Bytes
Total errors: 0, warnings: 0aipubuild config/resnet_v1_152_run.cfg
root@10c4628e25a5:~/demos/resnet152pb# aipubuild config/resnet_v1_152_run.cfg
[I] [Parser]: Parser done!
[I] Parse model complete
[I] Quantizing model....
[I] AQT start: model_name:resnet_152, calibration_method:MEAN, batch_size:1
[I] ==== read ir ================
[I] float32 ir txt: /tmp/AIPUBuilder_1626416278.7189496/resnet_152.txt
[I] float32 ir bin2: /tmp/AIPUBuilder_1626416278.7189496/resnet_152.bin
[I] ==== read ir DONE.===========
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.
[I] ==== auto-quantization ======
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and:
`tf.data.TFRecordDataset(path)`
WARNING:tensorflow:Entity <bound method ImageNet.data_transform_fn of <AIPUBuilder.AutoQuantizationTool.auto_quantization.data_set.ImageNet object at 0x7f3b32167978>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: <cyfunction ImageNet.data_transform_fn at 0x7f3b7258ed38> is not a module, class, method, function, traceback, frame, or code object
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py:330: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py:330: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py:915: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.
WARNING:tensorflow:From /usr/local/bin/aipubuild:8: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead.
[I] step1: get max/min statistic value DONE
[I] step2: quantization each op DONE
[I] step3: build quantization forward DONE
[I] step4: show output scale of end node:
[I] layer_id:314, layer_top:resnet_v1_152/predictions/Reshape_0, output_scale:[0.07918436]
[I] ==== auto-quantization DONE =
[I] Quantize model complete
[I] Building ...
[I] [common_options.h: 276] BuildTool version: 4.0.175. Build for target Z1_0701 at frequency 800MHz
[I] [common_options.h: 297] using default profile events to profile AIFF
[I] [IRChecker] Start to check IR: /tmp/AIPUBuilder_1626416278.7189496/resnet_152_int8.txt
[I] [IRChecker] model_name: resnet_152
[I] [IRChecker] IRChecker: All IR pass
[I] [graph.cpp : 846] loading graph weight: /tmp/AIPUBuilder_1626416278.7189496/resnet_152_int8.bin size: 0x39d4128
[I] [builder.cpp:1059] Total memory for this graph: 0x4128770 Bytes
[I] [builder.cpp:1060] Text section: 0x00086170 Bytes
[I] [builder.cpp:1061] RO section: 0x00010900 Bytes
[I] [builder.cpp:1062] Desc section: 0x00020400 Bytes
[I] [builder.cpp:1063] Data section: 0x039a0d00 Bytes
[I] [builder.cpp:1064] BSS section: 0x00690800 Bytes
[I] [builder.cpp:1065] Stack : 0x00040400 Bytes
[I] [builder.cpp:1066] Workspace(BSS): 0x000c4000 Bytes
[I] [main.cpp : 467] # autogenrated by aipurun, do NOT modify!
LOG_FILE=log_default
FAST_FWD_INST=0
INPUT_INST_CNT=1
INPUT_DATA_CNT=2
CONFIG=Z1-0701
LOG_LEVEL=0
INPUT_INST_FILE0=/tmp/temp_6115b9954f4848725c7378f0124b8.text
INPUT_INST_BASE0=0x0
INPUT_INST_STARTPC0=0x0
INPUT_DATA_FILE0=/tmp/temp_6115b9954f4848725c7378f0124b8.ro
INPUT_DATA_BASE0=0x10000000
INPUT_DATA_FILE1=/tmp/temp_6115b9954f4848725c7378f0124b8.data
INPUT_DATA_BASE1=0x20000000
OUTPUT_DATA_CNT=2
OUTPUT_DATA_FILE0=output_resnet_v1_152.bin
OUTPUT_DATA_BASE0=0x240ca200
OUTPUT_DATA_SIZE0=0x3e9
OUTPUT_DATA_FILE1=profile_data.bin
OUTPUT_DATA_BASE1=0x23ae1100
OUTPUT_DATA_SIZE1=0x2100
RUN_DESCRIPTOR=BIN[0]
[I] [main.cpp : 118] run simulator:
aipu_simulator_z1 /tmp/temp_6115b9954f4848725c7378f0124b8.cfg
[INFO]:SIMULATOR START!
[INFO]:========================================================================
[INFO]: STATIC CHECK
[INFO]:========================================================================
[INFO]: INST START ADDR : 0x0(0)
[INFO]: INST END ADDR : 0x8616f(549231)
[INFO]: INST SIZE : 0x86170(549232)
[INFO]: PACKET CNT : 0x8617(34327)
[INFO]: INST CNT : 0x2185c(137308)
[INFO]:------------------------------------------------------------------------
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x238: 0x472021b(POP R27,Rc7) vs 0x5f00000(MVI R0,0x0,Rc7), PACKET:0x238(568) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x245: 0x472021b(POP R27,Rc7) vs 0x5f00000(MVI R0,0x0,Rc7), PACKET:0x245(581) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x3aa: 0x472021b(POP R27,Rc7) vs 0x9f80020(ADD.S R0,R0,0x1,Rc7), PACKET:0x3aa(938) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x5e7: 0x4520180(BRL R0) vs 0x47a03e4(ADD R4,R0,R31,Rc7), PACKET:0x5e7(1511) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x780: 0x4720204(POP R4,Rc7) vs 0x9f80020(ADD.S R0,R0,0x1,Rc7), PACKET:0x780(1920) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0xc2a: 0x4720204(POP R4,Rc7) vs 0x47a1be0(ADD R0,R6,R31,Rc7), PACKET:0xc2a(3114) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x1050: 0x472021b(POP R27,Rc7) vs 0x5f00000(MVI R0,0x0,Rc7), PACKET:0x1050(4176) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x105d: 0x472021b(POP R27,Rc7) vs 0x5f00000(MVI R0,0x0,Rc7), PACKET:0x105d(4189) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x11c2: 0x472021b(POP R27,Rc7) vs 0x9f80020(ADD.S R0,R0,0x1,Rc7), PACKET:0x11c2(4546) SLOT:0 vs 3
[WARN]:[0803] INST WR/RD REG CONFLICT! PACKET 0x136b: 0x4520180(BRL R0) vs 0x47a03e5(ADD R5,R0,R31,Rc7), PACKET:0x136b(4971) SLOT:0 vs 3
[INFO]:========================================================================
[INFO]: STATIC CHECK END
[INFO]:========================================================================
[INFO]:AIPU START RUNNING: BIN[0]
[INFO]:TOTAL TIME: 21.323492s.
[INFO]:SIMULATOR EXIT!
[I] [main.cpp : 135] Simulator finished.
Total errors: 0, warnings: 07.仿真结果比对
python3 quant_predict.py
root@259519f3937d:~/demos/resnet152pb_v1# python3 quant_predict.py
predict first 5 label:
index 230, prob 216, name: Old English sheepdog, bobtail
index 231, prob 39, name: Shetland sheepdog, Shetland sheep dog, Shetland
index 999, prob 0, name: ear, spike, capitulum
index 342, prob 0, name: hog, pig, grunter, squealer, Sus scrofa
index 340, prob 0, name: sorrel
true first 5 label:
index 230, prob 109, name: Old English sheepdog, bobtail
index 231, prob 96, name: Shetland sheepdog, Shetland sheep dog, Shetland
index 232, prob 57, name: collie
index 226, prob 54, name: malinois
index 263, prob 53, name: Brabancon griffon
Detect picture save to result.jpeg
1.原始模型文件（可选）
2.矫正集的data.npy和label.npy
3.NN compiler的cfg文件
4.simulator执行的输入输出结果，比较运算量化误差
相关文件在网盘中
链接：https://pan.baidu.com/s/1rJs7...
提取码：afbp
