Skip to content

Conversation

@finlay-hudson
Copy link

@finlay-hudson finlay-hudson commented Feb 12, 2024

This PR is to allow for batch mode to work with the TRT models in both img2img and txt2img pipelines. It mimics batching in default pytorch method, whereby all images have to be of consistent sizes throughout the batch.

Speeds

Tested on 50 iterations, base model, txt2img pipeline

Batch Size Total Time (s) Time Per Image (s)
1 5.26 5.26
4 19.78 4.94
8 38.39 4.79

Tested on 37 iterations, base model, img2img pipeline

Batch Size Total Time (s) Time Per Image (s)
1 4.09 4.09
4 15.45 3.85
8 30.25 3.78

Tested on 37 iterations, refiner model, img2img pipeline

Batch Size Total Time (s) Time Per Image (s)
1 4.48 4.48
4 17.24 4.31
8 33.53 4.19

@height
Copy link

height bot commented Feb 12, 2024

Link Height tasks by mentioning a task ID in the pull request title or commit messages, or description and comments with the keyword link (e.g. "Link T-123").

💡Tip: You can also use "Close T-X" to automatically close a task when the pull request is merged.

@finlay-hudson finlay-hudson marked this pull request as ready for review February 12, 2024 14:21
@nhnt11 nhnt11 changed the base branch from tyler/spectral_2 to nihanth/batching February 29, 2024 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant