proxima-tokenizer v0.1.0 — Sans-IO byte-level BPE tokenizer: encode/decode over a caller-supplied vocab (built directly, or parsed from proxima-gguf metadata), no IO in the core. crates.io· crate · ▲ 0 points · Oct 13, 2026
0 comments
No comments yet.