[Question] How to debug custom handlers?

[veronikayurchuk]

Hi! I am very excited about torch serve, so thanks to all contributors to this awesome tool!
I have already run my custom model with custom postprocessing and here is a question that I am struggling to find an answer on. Any help would be very appreciated!

Question:
How to debug my custom handlers? In other words, how can I see what is happening with the data on each step (i.e initialize, preprocess, inference, postprocess, and my custom one) in my IDE while sending requests to running torchserve server? I was able to fix simple issues using the ts_log.log file and it was helpful. BUT it becomes not very comfortable for me once I want to do something more complicated.

Thanks for any help!

[HamidShojanazeri]

@veronikayurchuk for debugging purposes in a custom handler, you can add prints / logs inside any of the functions in the handler to obtain necessary info and you can see the info/logs in the terminal/shell that torchserve is running if you serving locally. I wonder if that helps you instead of checking the ts_log.log.

[saurabheights-ecr]

Not a repository owner.

This is not exactly what you are looking for but you can pass inputs to your handler directly same way as torchserve does.
For this, you can check :-

model_service_worker.TorchModelServiceWorker
:L108 - which shows prediction call
:L84 - creates ts.model_loader.TsModelLoader object for loading model

Makes debugging quite easy.

Furthermore, it might be possible to do torchserve integration with IDE, but I think it would involve understanding how Java frontend connects to model_service_worker.py. It already has the code to start backend server, so just need to know how to connect this server run inside pycharm with torchserve(modified torchserve with no backend).

Code owners might be able to help more.

[veronikayurchuk]

Thanks, @HamidShojanazeri and @saurabheights-ecr!
I found my alternative way of solving the issue.
It is not exactly what I was looking for, so I used a simple workaround, that temporarily helped me. Leaving it here in case someone is having the same issue.

I had to slightly modify my custom handler class. More specifically, in the preprocess method I changed
data.get("image") to request["image"] and while calling the handle method, I am sending simply a dict, where values are bytes. Also, I removed ctx argument in the initialize method.

[harshbafna]

Closing as the query has been resolved.