magi-tokenizer v0.0.1 — Exact, fast o200k_base BPE tokenizer with build-time tables and a hand-written pretokenizer

Sign in to join the discussion — your thebay.events account works here.