[bug] torch2trt test failed on new tensorrt and cuda

[andrey-avdeev]

1. Spec:

• OS: Ubuntu 18.04

• GPU: Nvidia 2080Ti

• CUDA: 10.1.243

• Docker: 19.03.02, build 6a30dfc

• Nvidia-docker was installed too

• Based on Dockerfile provided in this pull request Dockerfile was added #68

python -m torch2trt.test
| torchvision.models.alexnet.alexnet | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 6.87E-05 | 1.03e+03 | 1.56e+03 | 0.922 | 0.65 |
/usr/local/lib/python3.6/dist-packages/torchvision/models/squeezenet.py:94: UserWarning: nn.init.kaiming_uniform is now deprecated in favor of nn.init.kaiming_uniform_.
init.kaiming_uniform(m.weight.data)
/usr/local/lib/python3.6/dist-packages/torchvision/models/squeezenet.py:92: UserWarning: nn.init.normal is now deprecated in favor of nn.init.normal_.
init.normal(m.weight.data, mean=0.0, std=0.01)
| torchvision.models.squeezenet.squeezenet1_0 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.95E-03 | 409 | 1.96e+03 | 2.73 | 0.377 |
| torchvision.models.squeezenet.squeezenet1_1 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.95E-03 | 418 | 2.74e+03 | 2.6 | 0.339 |
| torchvision.models.resnet.resnet18 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 3.91E-03 | 408 | 1.16e+03 | 2.55 | 0.761 |
| torchvision.models.resnet.resnet34 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 9.38E-02 | 230 | 677 | 4.71 | 1.51 |
| torchvision.models.resnet.resnet50 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.41E-01 | 144 | 765 | 6.57 | 1.36 |
| torchvision.models.resnet.resnet101 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 0.00E+00 | 80.1 | 371 | 12.3 | 2.78 |
| torchvision.models.resnet.resnet152 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 0.00E+00 | 55 | 252 | 18 | 3.97 |
/usr/local/lib/python3.6/dist-packages/torchvision/models/densenet.py:212: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_.
nn.init.kaiming_normal(m.weight.data)
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/test.py", line 100, in
max_error, fps, fps_trt, ms, ms_trt = run(test)
File "/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/test.py", line 25, in run
module_trt = torch2trt(module, inputs, **self.torch2trt_kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py", line 252, in torch2trt
outputs = module(*inputs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 545, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torchvision/models/densenet.py", line 222, in forward
out = F.avg_pool2d(out, kernel_size=7, stride=1).view(features.size(0), -1)
File "/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py", line 97, in wrapper
converter(ctx)
File "/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/converters/view.py", line 11, in convert_view
layer = ctx.network.add_shuffle(input._trt)
AttributeError: 'Tensor' object has no attribute '_trt'

[andrey-avdeev]

Can you tell me on which spec your tests successfully passed?

[nhiem2000]

Ran okay for me with NGC TensorRT container 19.09
OS: Ubuntu 18.04
GPU: Nvidia V100
CUDA: 10.1.243
cuDNN: 7.6.3
TRT: 6.0.1
Installed Torch 1.2.0
torch2trt master (as of 9/25/19)

| torchvision.models.alexnet.alexnet | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 9.16E-05 | 1.01e+03 | 2.48e+03 | 1.17 | 0.444 |
| torchvision.models.squeezenet.squeezenet1_0 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 6.71E-04 | 275 | 2.79e+03 | 3.68 | 0.43 |
| torchvision.models.squeezenet.squeezenet1_1 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 9.77E-04 | 279 | 3.44e+03 | 3.62 | 0.355 |
| torchvision.models.resnet.resnet18 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 4.88E-03 | 279 | 1.37e+03 | 3.62 | 0.729 |
| torchvision.models.resnet.resnet34 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.25E-01 | 156 | 825 | 6.4 | 1.18 |
| torchvision.models.resnet.resnet50 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 2.19E-01 | 111 | 675 | 8.69 | 1.54 |
| torchvision.models.resnet.resnet101 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 0.00E+00 | 57 | 348 | 17.3 | 2.74 |
| torchvision.models.resnet.resnet152 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 0.00E+00 | 38.9 | 251 | 25.2 | 3.77 |
| torchvision.models.densenet.densenet121 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.95E-03 | 44 | 278 | 22.6 | 3.61 |
| torchvision.models.densenet.densenet169 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 3.91E-03 | 31.1 | 140 | 32.3 | 6.93 |
| torchvision.models.densenet.densenet201 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 3.91E-03 | 25.4 | 96.5 | 39.3 | 10.3 |
| torchvision.models.densenet.densenet161 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 5.86E-03 | 32.2 | 142 | 31.4 | 6.96 |
| torchvision.models.vgg.vgg11 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 2.20E-03 | 625 | 1.08e+03 | 1.92 | 0.937 |
| torchvision.models.vgg.vgg13 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.95E-03 | 517 | 954 | 2.14 | 1.09 |
| torchvision.models.vgg.vgg16 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 2.11E-03 | 439 | 765 | 2.45 | 1.32 |
| torchvision.models.vgg.vgg19 | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.77E-03 | 406 | 676 | 2.82 | 1.54 |
| torchvision.models.vgg.vgg11_bn | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 2.11E-03 | 554 | 1.13e+03 | 2.3 | 0.948 |
| torchvision.models.vgg.vgg13_bn | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 2.44E-03 | 473 | 980 | 2.61 | 1.08 |
| torchvision.models.vgg.vgg16_bn | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 2.11E-03 | 372 | 810 | 3.18 | 1.3 |
| torchvision.models.vgg.vgg19_bn | float16 | [(1, 3, 224, 224)] | {'fp16_mode': True} | 1.96E-03 | 322 | 678 | 3.57 | 1.54 |

[reborm]

This error happened because the avg_pool2d(torch.nn.functional.avg_pool2d) hasn't been implemented. You can replace the "F.avg_pool2d" by "torch.nn.AvgPool2d"

[jaybdub]

Ah, perhaps the underlying implementation changed in torchvision package. It should be relatively straightforward to add converter for functional version F.avg_pool2d. I will update this thread if this support changes.

@andrey-avdeev / @reborm. Which version of the torchvision package are you using?

Best,
John

[reborm]

The version of my torchvision is 0.4.0

[jaybdub]

Okay thanks. Support for torch.nn.functional.avg_pool2d and torch.nn.functional.max_pool2d has been added in this pull request.

#73

It's now merged into master, so those failing due to this should pass.

Best,
John