-
Notifications
You must be signed in to change notification settings - Fork 1.8k
ReDrafter support for Qwen #4875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: darraghdog <[email protected]>
Signed-off-by: darraghdog <[email protected]>
Signed-off-by: darraghdog <[email protected]>
/bot run |
/bot run |
1 similar comment
/bot run |
PR_Github #8552 [ run ] triggered by Bot |
PR_Github #8552 [ run ] completed with state |
Signed-off-by: darraghdog <[email protected]>
Head branch was pushed to by a user without write access
/bot run |
/bot run |
PR_Github #9897 [ run ] triggered by Bot |
PR_Github #9897 [ run ] completed with state |
Signed-off-by: Darragh Hanley <[email protected]>
Head branch was pushed to by a user without write access
@rakib-hasan the |
/bot run |
PR_Github #10170 [ run ] triggered by Bot |
PR_Github #10170 [ run ] completed with state |
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Signed-off-by: darraghdog <[email protected]> Signed-off-by: Darragh Hanley <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
Support the ReDrafter functionalities used in AIMO2,
fp8
base model.convert_checkpoint.py
into two steps. The idea is to give more flexibility around the base model used.1/ convert or quantise base model with normal
convert_model.py
orquantise.py
specific to the base model.2/ convert redrafter and attach to converted base model
bfloat16
in redrafter build - if we get a base model with abfloat16
layer, covert to float16, this is switched tofloat16
.ReDrafterForLLaMALM
,ReDrafterForQWenLM
... more can be added as needed, just need to test each one - I may add some more later, eg. Deepseek.Tests done on README examples, attaching logs from tests here :
redrafter_pr_test_logs.txt
.
Update on tests draft_len 6, beam_width 6:
redrafter_pr_beam6_draftlen_6.txt
[FIXED] Update2 on tests draft_len 6, beam_width 6:
redrafter_pr_beam6_base_beam1.txt