Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

MKL-DNN Quantization Examples and README #12808

Merged
merged 28 commits into from
Oct 19, 2018
Merged

MKL-DNN Quantization Examples and README #12808

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
1d7c88e
add gluoncv support
xinyu-intel Oct 8, 2018
855a4dd
add ssd readme
xinyu-intel Oct 9, 2018
68df9bc
improve ssd readme
xinyu-intel Oct 9, 2018
f9e30fe
add custom readme
xinyu-intel Oct 9, 2018
8d349f5
add ssd model link
xinyu-intel Oct 9, 2018
73dc2bf
add squeezenet
xinyu-intel Oct 9, 2018
82aac56
add ssd quantization script
xinyu-intel Oct 9, 2018
9942129
fix topo of args
xinyu-intel Oct 9, 2018
a93bbff
improve custom readme
xinyu-intel Oct 10, 2018
f7f6bcb
fix topo bug
xinyu-intel Oct 11, 2018
9530732
fix squeezenet
xinyu-intel Oct 12, 2018
b93cb29
add squeezenet accuracy
xinyu-intel Oct 12, 2018
15545fd
Add initializer for min max to support quantization
ZhennanQin Oct 12, 2018
1baaeaf
add dummy data inference
xinyu-intel Oct 12, 2018
a049351
rebase code
xinyu-intel Oct 12, 2018
19283ad
add test case for init_param
xinyu-intel Oct 12, 2018
40195bc
add subgraph docs
xinyu-intel Oct 12, 2018
9c0c2bc
improve docs
xinyu-intel Oct 14, 2018
2b20043
add two models and fix default rgb_std to 1
xinyu-intel Oct 14, 2018
1df46d4
fix doc link
xinyu-intel Oct 14, 2018
a5b309d
improve MKLDNN_README
xinyu-intel Oct 14, 2018
4d1338d
add quantization for mobilenetv1
xinyu-intel Oct 15, 2018
c7a35dc
Merge remote-tracking branch 'upstream/master' into mkldnn_quantizati…
xinyu-intel Oct 15, 2018
f8fbc3f
fix ssd benchmark_score label shapes
xinyu-intel Oct 15, 2018
05b88cd
add resnet101_v1 and inceptionv3 support
xinyu-intel Oct 16, 2018
32b44d3
Refine some descriptions in the MKLDNN_README
juliusshufan Oct 16, 2018
a80a628
improve docs
xinyu-intel Oct 16, 2018
bf35236
improve link in perf.md
xinyu-intel Oct 16, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add resnet101_v1 and inceptionv3 support
  • Loading branch information
@xinyu-intel
xinyu-intel committed Oct 16, 2018
commit 05b88cd10b104165b94369b3be95e840377e7f92
73 changes: 62 additions & 11 deletions example/quantization/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,13 @@ The following models have been tested on Linux systems.
| Model | Source | Dataset | FP32 Accuracy | INT8 Accuracy |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this accuracy top-1 / top-5? Can you be specific?

|:---|:---|---|:---:|:---:|
| [ResNet50-V1](#3) | [Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) | 75.87%/92.72% | 75.71%/92.65% |
|[Squeezenet 1.0](#4)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|57.01%/79.71%|56.62%/79.55%|
|[MobileNet 1.0](#5)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|69.76%/89.32%|69.61%/89.09%|
|[ResNet152-V2](#6)|[MXNet ModelZoo](http://data.mxnet.io/models/imagenet/resnet/152-layers/)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|76.76%/93.03%|76.48%/92.96%|
|[Inception-BN](#7)|[MXNet ModelZoo](http://data.mxnet.io/models/imagenet/inception-bn/)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|72.09%/90.60%|72.00%/90.53%|
| [SSD-VGG](#8) | [example/ssd](https://github.com/apache/incubator-mxnet/tree/master/example/ssd) | VOC2007/2012 | 0.83 mAP | 0.82 mAP |
| [ResNet101-V1](#4) | [Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) | 77.3%/93.58% | 77.09%/93.41% |
|[Squeezenet 1.0](#5)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|57.01%/79.71%|56.62%/79.55%|
|[MobileNet 1.0](#6)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|69.76%/89.32%|69.61%/89.09%|
|[Inception V3](#7)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|76.49%/93.10% |76.38%/93% |
|[ResNet152-V2](#8)|[MXNet ModelZoo](http://data.mxnet.io/models/imagenet/resnet/152-layers/)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|76.76%/93.03%|76.48%/92.96%|
|[Inception-BN](#9)|[MXNet ModelZoo](http://data.mxnet.io/models/imagenet/inception-bn/)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|72.09%/90.60%|72.00%/90.53%|
| [SSD-VGG](#10) | [example/ssd](https://github.com/apache/incubator-mxnet/tree/master/example/ssd) | VOC2007/2012 | 0.83 mAP | 0.82 mAP |

<h3 id='3'>ResNet50-V1</h3>

Expand All @@ -53,7 +55,32 @@ python imagenet_inference.py --symbol-file=./model/resnet50_v1-symbol.json --bat
python imagenet_inference.py --symbol-file=./model/resnet50_v1-quantized-5batches-naive-symbol.json --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
```

<h3 id='4'>SqueezeNet1.0</h3>
<h3 id='4'>ResNet101-V1</h3>

The following command is to download the pre-trained model from Gluon-CV and transfer it into the symbolic model which would be finally quantized. The validation dataset is available [here](http://data.mxnet.io/data/val_256_q90.rec) for testing the pre-trained models:

```
python imagenet_gen_qsym_mkldnn.py --model=resnet101_v1 --num-calib-batches=5 --calib-mode=naive
```

The model would be automatically replaced in fusion and quantization format and saved as the quantized symbol and parameter fils in `./model` dictionary. The following command is to launch inference.

```
# USE MKLDNN AS SUBGRAPH BACKEND
export MXNET_SUBGRAPH_BACKEND=MKLDNN

# Launch FP32 Inference
python imagenet_inference.py --symbol-file=./model/resnet101_v1-symbol.json --param-file=./model/resnet101_v1-0000.params --rgb-mean=123.68,116.779,103.939 --rgb-std=58.393,57.12,57.375 --num-skipped-batches=50 --batch-size=64 --num-inference-batches=500 --dataset=./data/val_256_q90.rec --ctx=cpu --data-nthreads=1

# Launch INT8 Inference
python imagenet_inference.py --symbol-file=./model/resnet101_v1-quantized-5batches-naive-symbol.json --param-file=./model/resnet101_v1-quantized-0000.params --rgb-mean=123.68,116.779,103.939 --rgb-std=58.393,57.12,57.375 --num-skipped-batches=50 --batch-size=64 --num-inference-batches=500 --dataset=./data/val_256_q90.rec --ctx=cpu --data-nthreads=1

# Launch dummy data Inference
python imagenet_inference.py --symbol-file=./model/resnet101_v1-symbol.json --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
python imagenet_inference.py --symbol-file=./model/resnet101_v1-quantized-5batches-naive-symbol.json --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
```

<h3 id='5'>SqueezeNet1.0</h3>

The following command is to download the pre-trained model from Gluon-CV and transfer it into the symbolic model which would be finally quantized. The validation dataset is available [here](http://data.mxnet.io/data/val_256_q90.rec) for testing the pre-trained models:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update link...


Expand All @@ -77,7 +104,7 @@ python imagenet_inference.py --symbol-file=./model/squeezenet1.0-symbol.json --b
python imagenet_inference.py --symbol-file=./model/squeezenet1.0-quantized-5batches-naive-symbol.json --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
```

<h3 id='5'>MobileNet1.0</h3>
<h3 id='6'>MobileNet1.0</h3>

The following command is to download the pre-trained model from Gluon-CV and transfer it into the symbolic model which would be finally quantized. The validation dataset is available [here](http://data.mxnet.io/data/val_256_q90.rec) for testing the pre-trained models:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update link...


Expand All @@ -101,7 +128,31 @@ python imagenet_inference.py --symbol-file=./model/mobilenet1.0-symbol.json --ba
python imagenet_inference.py --symbol-file=./model/mobilenet1.0-quantized-5batches-naive-symbol.json --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
```

<h3 id='6'>ResNet152-V2</h3>
<h3 id='7'>Inception-V3</h3>

The following command is to download the pre-trained model from Gluon-CV and transfer it into the symbolic model which would be finally quantized. The validation dataset is available [here](http://data.mxnet.io/data/val_256_q90.rec) for testing the pre-trained models:

```
python imagenet_gen_qsym_mkldnn.py --model=inceptionv3 --image-shape=3,299,299 --num-calib-batches=5 --calib-mode=naive
```
The model would be automatically replaced in fusion and quantization format and saved as the quantized symbol and parameter fils in `./model` dictionary. The following command is to launch inference.

```
# USE MKLDNN AS SUBGRAPH BACKEND
export MXNET_SUBGRAPH_BACKEND=MKLDNN

# Launch FP32 Inference
python imagenet_inference.py --symbol-file=./model/inceptionv3-symbol.json --param-file=./model/inceptionv3-0000.params --image-shape=3,299,299 --rgb-mean=123.68,116.779,103.939 --rgb-std=58.393,57.12,57.375 --num-skipped-batches=50 --batch-size=64 --num-inference-batches=500 --dataset=./data/val_256_q90.rec --ctx=cpu --data-nthreads=1

# Launch INT8 Inference
python imagenet_inference.py --symbol-file=./model/inceptionv3-quantized-5batches-naive-symbol.json --param-file=./model/inceptionv3-quantized-0000.params --image-shape=3,299,299 --rgb-mean=123.68,116.779,103.939 --rgb-std=58.393,57.12,57.375 --num-skipped-batches=50 --batch-size=64 --num-inference-batches=500 --dataset=./data/val_256_q90.rec --ctx=cpu --data-nthreads=1

# Launch dummy data Inference
python imagenet_inference.py --symbol-file=./model/inceptionv3-symbol.json --image-shape=3,299,299 --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
python imagenet_inference.py --symbol-file=./model/inceptionv3-quantized-5batches-naive-symbol.json --image-shape=3,299,299 --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
```

<h3 id='8'>ResNet152-V2</h3>

The following command is to download the pre-trained model from [MXNet ModelZoo](http://data.mxnet.io/models/imagenet/resnet/152-layers/) which would be finally quantized. The validation dataset is available [here](http://data.mxnet.io/data/val_256_q90.rec) for testing the pre-trained models:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from the...
update the here link


Expand All @@ -126,7 +177,7 @@ python imagenet_inference.py --symbol-file=./model/imagenet1k-resnet-152-symbol.
python imagenet_inference.py --symbol-file=./model/imagenet1k-resnet-152-quantized-5batches-naive-symbol.json --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
```

<h3 id='7'>Inception-BN</h3>
<h3 id='9'>Inception-BN</h3>

The following command is to download the pre-trained model from [MXNet ModelZoo](http://data.mxnet.io/models/imagenet/inception-bn/) which would be finally quantized. The validation dataset is available [here](http://data.mxnet.io/data/val_256_q90.rec) for testing the pre-trained models:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from the...
update the here link


Expand All @@ -151,7 +202,7 @@ python imagenet_inference.py --symbol-file=./model/imagenet1k-inception-bn-symbo
python imagenet_inference.py --symbol-file=./model/imagenet1k-inception-bn-quantized-5batches-naive-symbol.json --batch-size=64 --num-inference-batches=500 --ctx=cpu --benchmark=True
```

<h3 id='8'>SSD-VGG</h3>
<h3 id='10'>SSD-VGG</h3>

Go to [example/ssd](https://github.com/apache/incubator-mxnet/tree/master/example/ssd) dictionary. Following the [instruction](https://github.com/apache/incubator-mxnet/tree/master/example/ssd#train-the-model) in [example/ssd](https://github.com/apache/incubator-mxnet/tree/master/example/ssd) to train a FP32 `SSD-VGG16_reduced_300x300` model based on Pascal VOC dataset. You can also download our [pre-trained model](http://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/models/ssd_vgg16_reduced_300-dd479559.zip) and [packed binary data](http://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/ssd-val-fc19a535.zip) then rename them and extract to `model/` and `data/` dictionary as below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow the [SSD example's instructions] to train... (you can delete that first link to the example).
[SSD VGG16 pre-trained model]
end sentence after the second link...
"Extract the zip files, then rename the directories to model and data respectively."


Expand Down Expand Up @@ -188,7 +239,7 @@ python benchmark_score.py --deploy --prefix=./model/ssd_
python benchmark_score.py --deploy --prefix=./model/cqssd_
```

<h3 id='9'>Custom Model</h3>
<h3 id='11'>Custom Model</h3>

This script also supports custom symbolic models. You can easily add some quantization layer configs in `imagenet_gen_qsym_mkldnn.py` like below:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like below --> as follows:


Expand Down
21 changes: 16 additions & 5 deletions example/quantization/imagenet_gen_qsym_mkldnn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def download_model(model_name, logger=None):
logger.info('Downloading model %s... into path %s' % (model_name, model_path))
return modelzoo.download_model(args.model, os.path.join(dir_path, 'model'))

def convert_from_gluon(model_name, classes=1000, logger=None):
def convert_from_gluon(model_name, image_shape, classes=1000, logger=None):
dir_path = os.path.dirname(os.path.realpath(__file__))
model_path = os.path.join(dir_path, 'model')
if logger is not None:
Expand All @@ -66,7 +66,8 @@ def convert_from_gluon(model_name, classes=1000, logger=None):
mod = mx.mod.Module(symbol=symnet, context=mx.cpu(),
label_names = ['softmax_label'])
mod.bind(for_training=False,
data_shapes=[('data', (1, 3, 224, 224))])
data_shapes=[('data', (1,) +
tuple([int(i) for i in image_shape.split(',')]))])
mod.set_params(arg_params=args, aux_params=auxs)
dst_dir = os.path.join(dir_path, 'model')
prefix = os.path.join(dir_path, 'model', model_name)
Expand All @@ -92,6 +93,8 @@ def save_params(fname, arg_params, aux_params, logger=None):
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Generate a calibrated quantized model from a FP32 model with Intel MKL-DNN support')
parser.add_argument('--model', type=str, choices=['resnet50_v1',
'resnet101_v1',
'inceptionv3',
'squeezenet1.0',
'mobilenet1.0',
'imagenet1k-resnet-152',
Expand Down Expand Up @@ -160,11 +163,11 @@ def save_params(fname, arg_params, aux_params, logger=None):
download_calib_dataset('http://data.mxnet.io/data/val_256_q90.rec', args.calib_dataset)

# download model
if args.model in ['resnet50_v1', 'squeezenet1.0', 'mobilenet1.0']:
if args.model in ['resnet50_v1', 'resnet101_v1', 'squeezenet1.0', 'mobilenet1.0', 'inceptionv3']:
logger.info('model %s is converted from GluonCV' % args.model)
args.use_gluon_model = True
if args.use_gluon_model == True:
prefix = convert_from_gluon(model_name=args.model, classes=1000, logger=logger)
prefix = convert_from_gluon(model_name=args.model, image_shape=args.image_shape, classes=1000, logger=logger)
epoch = 0
sym, arg_params, aux_params = mx.model.load_checkpoint(prefix, epoch)
elif args.model == 'custom':
Expand Down Expand Up @@ -211,7 +214,7 @@ def save_params(fname, arg_params, aux_params, logger=None):
excluded_sym_names += ['flatten', 'fc1']
if exclude_first_conv:
excluded_sym_names += ['conv_1']
elif args.model == 'resnet50_v1':
elif args.model in ['resnet50_v1', 'resnet101_v1']:
rgb_mean = '123.68,116.779,103.939'
rgb_std = '58.393, 57.12, 57.375'
calib_layer = lambda name: name.endswith('_output')
Expand All @@ -238,6 +241,14 @@ def save_params(fname, arg_params, aux_params, logger=None):
'mobilenet0_pool0_fwd']
if exclude_first_conv:
excluded_sym_names += ['mobilenet0_conv0_fwd']
elif args.model == 'inceptionv3':
rgb_mean = '123.68,116.779,103.939'
rgb_std = '58.393, 57.12, 57.375'
calib_layer = lambda name: name.endswith('_output')
excluded_sym_names += ['inception30_dense0_fwd',
'inception30_pool0_fwd']
if exclude_first_conv:
excluded_sym_names += ['inception30_conv0_fwd']
elif args.model == 'custom':
# add rgb mean/std of your model.
rgb_mean = '0,0,0'
Expand Down