25% of Atlas Score
This repository provides a self-contained C++ implementation of the Byte Pair Encoding (BPE) tokenization algorithm, including a driver program that trains BPE on a text corpus and outputs the merged pairs. A companion Python script automates repeated compilation and execution for demonstration.