Skip to content

Conversation

@gante
Copy link
Contributor

@gante gante commented Sep 10, 2024

What does this PR do?

This PR propagates the updates to the RoPE structure to cohere -- the logic for RoPE was abstracted into a separate module for llama3.1 (#32135). Using the new structure, a model has access to all RoPE scaling strategies.

While touching the modeling code, I've taken the liberty to:

  1. update copied from statements, which were disabled in previous PRs;
  2. update (postpone) deprecation messages to ensure deprecated features are removed from all models in the same version.

✅ all slow tests passing


Note: #31999 was originally open to migrate all modern RoPE models into the upgraded structure. However, working on cohere, I noticed that there may be important implementation differences in RoPE. As such, I'll be opening multiple PRs, batching similar RoPE implementations together.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is copy/paste from llama

Comment on lines +139 to +140
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from the line highlighted with a comment, this is copy/paste from llama

@gante gante requested a review from LysandreJik September 10, 2024 15:14
@gante
Copy link
Contributor Author

gante commented Sep 10, 2024

@LysandreJik a PR like this one will be open for a few more modern models. Since part of the changes consists of having a global view of the model to update the copied from statements, would you like me to update the import structures as well? 🤗

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Member

@LysandreJik LysandreJik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks clean, nice to reuse the llama code

@LysandreJik
Copy link
Member

@LysandreJik a PR like this one will be open for a few more modern models. Since part of the changes consists of having a global view of the model to update the copied from statements, would you like me to update the import structures as well? 🤗

No need to updat the import structure for now!

@gante gante merged commit 95e816f into huggingface:main Sep 16, 2024
@gante gante deleted the unify_rope branch September 16, 2024 08:45
BernardZach pushed a commit to BernardZach/transformers that referenced this pull request Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants