Skip to content

Support fine-tuning MLLama. #2132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Sep 26, 2024
Merged

Support fine-tuning MLLama. #2132

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
daa39c6
support llama3.2-vision
Jintao-Huang Sep 26, 2024
cdac706
support llama3.2-vision
Jintao-Huang Sep 26, 2024
fda9778
update
Jintao-Huang Sep 26, 2024
a22d15a
update
Jintao-Huang Sep 26, 2024
d292d08
update
Jintao-Huang Sep 26, 2024
74b8d83
fix batch
Jintao-Huang Sep 26, 2024
ca9e1cb
update readme
Jintao-Huang Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ You can contact us and communicate with us by adding our group:
<img src="asset/discord_qr.jpg" width="200" height="200"> | <img src="asset/wechat.png" width="200" height="200">

## 🎉 News
- 2024.09.24: Support for training and deploying llama3.2 series models. Experience it using `swift infer --model_type llama3_2-1b-instruct`.
- 2024.09.26: Support for training and deploying llama3.2-vision series models. Experience it using `swift infer --model_type llama3_2-11b-vision-instruct`.
- 2024.09.26: Support for training and deploying llama3.2 series models. Experience it using `swift infer --model_type llama3_2-1b-instruct`.
- 2024.09.25: Support for training to deployment with got-ocr2. Best practices can be found [here](https://github.com/modelscope/ms-swift/issues/2122).
- 2024.09.24: Support for training and deploying llama3_1-8b-omni. Experience it using `swift infer --model_type llama3_1-8b-omni`.
- 2024.09.23: Support for training and deploying pixtral-12b. Experience it using `swift infer --model_type pixtral-12b --dtype fp16`.
Expand Down Expand Up @@ -623,6 +624,7 @@ The complete list of supported models and datasets can be found at [Supported Mo
|------------------------------------------------------------|----------------------------------------------------------------------------------------|--------------------|---------------------------------------|--------------------------|
| Qwen-VL<br>Qwen2-VL | [Tongyi Qwen vision model](https://github.com/QwenLM) | Chinese<br>English | 2B-72B<br>including quantized versions | base model<br>chat model |
| Qwen-Audio<br>Qwen2-Audio | [Tongyi Qwen speech model](https://github.com/QwenLM) | Chinese<br>English | 7B | base model<br>chat model |
| Llama3.2-Vision | [Llama3.2](https://huggingface.co/collections/meta-llama/llama-32-66f448ffc8c32f949b04c8cf) | English | 11B-90B | base model<br>chat model |
| YI-VL | [01AI's YI series vision models](https://github.com/01-ai) | Chinese<br>English | 6B-34B | chat model |
| XComposer2<br>XComposer2.5 | [Pujiang AI Lab InternLM vision model](https://github.com/InternLM/InternLM-XComposer) | Chinese<br>English | 7B | chat model |
| DeepSeek-VL | [DeepSeek series vision models](https://github.com/deepseek-ai) | Chinese<br>English | 1.3B-7B | chat model |
Expand Down
2 changes: 2 additions & 0 deletions README_CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ SWIFT具有丰富全面的文档，请查看我们的文档网站:


## 🎉 新闻
- 2024.09.26: 支持llama3.2-vision系列模型的训练到部署. 使用`swift infer --model_type llama3_2-11b-vision-instruct`进行体验.
- 2024.09.26: 支持llama3.2系列模型的训练到部署. 使用`swift infer --model_type llama3_2-1b-instruct`进行体验.
- 2024.09.25: 支持got-ocr2的训练到部署. 最佳实践可以查看[这里](https://github.com/modelscope/ms-swift/issues/2122).
- 2024.09.24: 支持llama3_1-8b-omni的训练与部署. 使用`swift infer --model_type llama3_1-8b-omni`进行体验.
Expand Down Expand Up @@ -616,6 +617,7 @@ CUDA_VISIBLE_DEVICES=0 swift deploy \
|---------------------------------------------------------|----------------------------------------------------------------------------|----------|------------------|------------------|
| Qwen-VL<br>Qwen2-VL | [通义千问视觉模型](https://github.com/QwenLM) | 中文<br>英文 | 2B-72B<br>包含量化版本 | base模型<br>chat模型 |
| Qwen-Audio<br>Qwen2-Audio | [通义千问语音模型](https://github.com/QwenLM) | 中文<br>英文 | 7B | base模型<br>chat模型 |
| Llama3.2-Vision | [Llama3.2](https://huggingface.co/collections/meta-llama/llama-32-66f448ffc8c32f949b04c8cf) | 英文 | 11B-90B | base模型<br>chat模型 |
| YI-VL | [01AI的YI系列视觉模型](https://github.com/01-ai) | 中文<br>英文 | 6B-34B | chat模型 |
| XComposer2<br>XComposer2.5 | [浦江实验室书生浦语视觉模型](https://github.com/InternLM/InternLM-XComposer) | 中文<br>英文 | 7B | chat模型 |
| DeepSeek-VL | [幻方系列视觉模型](https://github.com/deepseek-ai) | 中文<br>英文 | 1.3B-7B | chat模型 |
Expand Down
Loading
Loading