≈ the.bay.news

I crawled 150 pages from 6 popular docs sites into RAG chunks: one page often eats half the token budget

DEV Community
I crawled 150 pages from 6 popular docs sites into RAG chunks: one page often eats half the token budget
n8n, FastAPI, Stripe, Supabase, Astro and React docs, 25 pages each, converted to clean Markdown and 500-token chunks. Where the tokens actually go, and three filters to set before you index a docs site.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.