| 
 | 1 | +# TensorFlow-TensorRT Examples  | 
 | 2 | + | 
 | 3 | +This script will run inference using a few popular image classification models  | 
 | 4 | +on the ImageNet validation set.  | 
 | 5 | + | 
 | 6 | +You can turn on TensorFlow-TensorRT integration with the flag `--use_trt`. This  | 
 | 7 | +will apply TensorRT inference optimization to speed up execution for portions of  | 
 | 8 | +the model's graph where supported, and will fall back to native TensorFlow for  | 
 | 9 | +layers and operations which are not supported. See  | 
 | 10 | +https://devblogs.nvidia.com/tensorrt-integration-speeds-tensorflow-inference/  | 
 | 11 | +for more information.  | 
 | 12 | + | 
 | 13 | +When using TF-TRT, you can also control the precision with `--precision`.  | 
 | 14 | +float32 is the default (`--precision fp32`) with float16 (`--precision fp16`) or  | 
 | 15 | +int8 (`--precision int8`) allowing further performance improvements, at the cost  | 
 | 16 | +of some accuracy. int8 mode requires a calibration step which is done  | 
 | 17 | +automatically.  | 
 | 18 | + | 
 | 19 | +## Models  | 
 | 20 | + | 
 | 21 | +This test supports the following models for image classification:  | 
 | 22 | +* MobileNet v1  | 
 | 23 | +* MobileNet v2  | 
 | 24 | +* NASNet - Large  | 
 | 25 | +* NASNet - Mobile  | 
 | 26 | +* ResNet50 v1  | 
 | 27 | +* ResNet50 v2  | 
 | 28 | +* VGG16  | 
 | 29 | +* VGG19  | 
 | 30 | +* Inception v3  | 
 | 31 | +* Inception v4  | 
 | 32 | + | 
 | 33 | +## Setup  | 
 | 34 | +```  | 
 | 35 | +# Clone [tensorflow/models](https://github.com/tensorflow/models)  | 
 | 36 | +git clone https://github.com/tensorflow/models.git  | 
 | 37 | +
  | 
 | 38 | +# Add the models directory to PYTHONPATH to install tensorflow/models.  | 
 | 39 | +cd models  | 
 | 40 | +export PYTHONPATH="$PYTHONPATH:$PWD"  | 
 | 41 | +
  | 
 | 42 | +# Run the TF Slim setup.  | 
 | 43 | +cd research/slim  | 
 | 44 | +python setup.py install  | 
 | 45 | +
  | 
 | 46 | +# You may also need to install the requests package  | 
 | 47 | +pip install requests  | 
 | 48 | +```  | 
 | 49 | +Note: the PYTHONPATH environment variable will be not be saved between different  | 
 | 50 | +shells. You can either repeat that step each time you work in a new shell, or  | 
 | 51 | +add `export PYTHONPATH="$PYTHONPATH:/path/to/tensorflow_models"` to your .bashrc  | 
 | 52 | +file (replacing /path/to/tensorflow_models with the path to your  | 
 | 53 | +tensorflow/models repository).  | 
 | 54 | + | 
 | 55 | +### Data  | 
 | 56 | + | 
 | 57 | +The script supports only TFRecord format for data. The script  | 
 | 58 | +assumes that validation TFRecords are named according to the pattern:  | 
 | 59 | +`validation-*-of-00128`.  | 
 | 60 | + | 
 | 61 | +You can download and process Imagenet using [this script provided by TF  | 
 | 62 | +Slim](https://github.com/tensorflow/models/blob/master/research/slim/datasets/download_imagenet.sh).  | 
 | 63 | +Please note that this script downloads both the training and validation sets,  | 
 | 64 | +and this example only requires the validation set.  | 
 | 65 | + | 
 | 66 | +## Usage  | 
 | 67 | + | 
 | 68 | +`python inference.py --data_dir /imagenet_validation_data --model vgg_16 [--use_trt]`  | 
 | 69 | + | 
 | 70 | +Run with `--help` to see all available options.  | 
0 commit comments