Training Experiment · Python

bf16-update-realization-in-transformers

Rick@JuiceB0xC0de · Atlas 73.3 · #2= builder

What it is

The repository investigates the behavior of AdamW optimizer in bfloat16 precision when fine-tuning a small language model (SmolLM2-135M) on GSM8K, by measuring the discrepancy between intended and realized weight updates through custom observer and analysis tools.

Scored from the code at its last scan, 22 Oct 2026. How merit is scored

Where it counts

Machine Learning85%
NLP & LLMs85%
MLOps15%

How much of the project is about each field, as scoring counts it. It ranks in the field it is most about.

Around it on the Machine Learning board

Whole board

RankProjectBuilderFieldMerit

22, shared rankmines-livive-netTraining Experiment · JavaScriptThis repository provides a training and evaluation pipeline for reinforcement learning agents applied to the 8x8 Minesweeper game. It includes custom game environment, model architecture, training scripts (behavior cloning and REINFORCE), evaluation tools, and generates…57.4/100ImplementationJarad PeckenpaughMachine Learning
25pcap-ml-traffic-classifierML Application · PythonThis repository provides a machine learning-based network intrusion detection system that extracts L3/L4 features from PCAP files using Scapy and trains a CatBoost classifier to distinguish benign from malicious traffic. It includes a threat intelligence reporting feature that…56.2/100ImplementationPop AlexandruMachine Learning
26bf16-update-realization-in-transformersTraining Experiment · Python55.8/100ImplementationRickMachine Learning
27bf16-4-3-overshoot-ratio-investigationTraining Experiment · PythonThe repository investigates the effect of different parameter writing rules (e.g., bfloat16 stochastic vs. round-to-nearest-even) on the overshoot ratio in optimizer steps by replaying recorded Camera-3 optimizer captures from Mamba-3 training. It reconstructs the intended…55.5/100ImplementationRickMachine Learning
28, shared rankASTROTraining Experiment · PythonASTRO is a research project developing and evaluating a new optimizer (ASTRO) for language model training, featuring ablation studies, scaling experiments, and a reproducible benchmark harness.55.3/100ImplementationPop AlexandruMachine Learning

What it is

Where it counts

Around it on the Machine Learning board

More from Rick