Inquiry About Training Script

[weijinbao1998]

Congratulations on your outstanding work being accepted to ICML!

I was wondering if you could share any plans regarding the release of the training script? I’m very interested in reproducing and building upon your work, and having access to the full implementation details would be immensely valuable.

Thank you again for your contribution, and I look forward to your updates!

Best regards,

[Lin-Tianwei]

Thank you for your interest! We've been handling several tasks recently, but releasing the training scripts is on our roadmap and we’ll work to make them available as soon as possible.

[cpenguf]

@Lin-Tianwei Thank you for your awesome work. I am wondering if there has been any update on the pretraining script.