Rotary Positional Embedding: [paper]
-
Let
-
We need the transformation functions above (
$f_q$ and$f_k$ ), such that they satisfy the below equation, the resultant function should encode relative position information
-
This repository was archived by the owner on Jun 21, 2025. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 0
Simple Implementation of Rotary Positional Embeddings (RoPE) and Sinusoidal Positional Embeddings in JAX
License
VachanVY/Rotary-Embeddings
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Simple Implementation of Rotary Positional Embeddings (RoPE) and Sinusoidal Positional Embeddings in JAX
Topics
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published