Someone built a visualization of what 1 million tokens actually looks like
lemmy.world
Someone built a visualization of what 1 million tokens actually looks like
I found One Million Tokens [https://www.1millioncontext.com/]. Its rough scale: text 1M tokens ~ 750K words ~ 3,000 pages ~ 83 hours of conversation ~ 75K lines of code 🧠 The more interesting part is the timeline. It starts with GPT-3 at 2,048 tokens in 2020, then walks through ChatGPT 4K, GPT-4 32K, Claude 100K, Gemini 1M, and the later multi-million-token era. The visual change is kind of absurd when you see all the pages stacked together. One caveat: maximum context is not the same as perfect memory/retrieval. A model accepting 1M tokens can still fail to use information buried inside that window. The line-of-code/page conversions are approximations too. I am curious how people here design around 1M+ context in practice. Do you actually feed giant source sets directly, or still prefer retrieval + smaller focused contexts for cost/attention reasons?
0 comments
No comments yet.