[New Model]: Blip2 Support required

[anisingh1]

The model to consider.

https://huggingface.co/Salesforce/blip2-opt-2.7b

The closest model vllm already supports.

https://huggingface.co/liuhaotian/llava-v1.5-7b

What's your difficulty of supporting the model you want?

Support for Llava 1.5 was recently added. Blip2 has a different architecture which is currently not supported.

[dhl1402]

How can I use it for image captioning task (not VQA task)? With transformer, I just have to omit the prompt.

[DarkLight1337]

Try inputting prompt_token_ids=[] to LLM.generate which should correspond to an empty prompt (without special tokens).