20% of Atlas Score
The ORBIT repository provides a Python implementation of the ORBIT optimizer for transformer models with rotary positional embeddings. It includes the optimizer, adapters for external models, baselines, training scripts, and detailed documentation for reproduction and use.