The RAG chunk size calculator above converts a corpus and a chunking configuration into the three numbers that follow from it: how many chunks you end up storing, how many tokens each query injects into the model prompt, and what that retrieval load costs across a month of traffic. Chunk size is usually chosen from a tutorial default and never revisited, even though it sets both the size of your index and the recurring cost of every answer your system produces.
Arb Digital works through this trade-off when scoping retrieval systems for clients, because the same corpus can produce wildly different bills depending on two settings that take ten seconds to change. The calculator makes those consequences visible before you commit to an index you would have to rebuild to undo.
What This RAG Chunk Size Calculator Does
Enter your corpus size, chunk size, overlap, and how many chunks you retrieve per query. The headline figure is the total number of chunks the configuration produces. The grid then shows chunks per document, the retrieval token load added to each query, the monthly cost of that load at your generation model's input rate, and the duplication factor — how much larger your index is than the underlying text because of overlap.
The bars split every stored token into unique content and repeated content, which is the clearest way to see what an overlap setting is actually buying and costing. Change the overlap field and watch both the chunk count and the duplication factor move together.
How to Use It
- Enter documents and average word count. The tool converts words to tokens at roughly 1.3 tokens per word for English prose; if your content is code, tables, or another language, size it directly with the AI token calculator and work in tokens.
- Set chunk size. This is the maximum token length of a single retrievable unit, not the length of a source document.
- Set overlap. Overlap is measured in tokens and is repeated from the end of one chunk at the start of the next. Zero is valid and cheapest.
- Set top-k. The number of chunks your retriever passes to the model per query. This is the field with the most direct effect on cost.
- Add query volume and your model's input rate, then click Calculate to see index size and running cost together.
The Formula / How It's Calculated
The chunker advances through a document by a fixed stride, which is chunk size minus overlap. With a 512-token chunk and 64 tokens of overlap, the stride is 448 — each new chunk starts 448 tokens further along, repeating the previous 64. The number of chunks in a document is therefore ceiling((document tokens − overlap) ÷ stride), with a floor of one chunk for anything shorter than a full window.
Total stored tokens are chunks × chunk size, which exceeds the original text by the duplication factor = chunk size ÷ stride. At 512 and 64 that is 1.14, so the index holds about 14% more tokens than the corpus contains. Retrieval load is simpler and more expensive: top-k × chunk size tokens are added to every prompt, and the monthly cost is that figure times query volume times your input rate.
Overlap Costs Less Than People Fear — And More Than They Notice
The duplication factor is non-linear, and that is the part worth internalising. Going from zero to 64 tokens of overlap on a 512-token chunk adds 14%. Going from 64 to 128 adds another 19 percentage points. At 256 tokens of overlap on a 512-token chunk the stride halves and you are storing and embedding the corpus twice over. The curve is gentle at the start and brutal past a quarter of the chunk size.
What you get in exchange is that no sentence spanning a boundary is orphaned. A definition split across the seam of two chunks may be unretrievable in both, because neither chunk on its own contains the full statement. Modest overlap — somewhere around 10 to 20% of chunk size — buys most of that protection cheaply. Beyond that you are paying to store the same sentences repeatedly for diminishing benefit. Feed your chosen average chunk length into the embedding cost calculator to price the indexing side of that decision.
Retrieval Load Is the Recurring Cost, Not Storage
Teams optimise index size because it is the number they can see in a dashboard. The number that actually accumulates is retrieval load, because it is billed as input tokens on every generation call for the life of the product.
At the defaults — five chunks of 512 tokens — every query carries 2,560 tokens of retrieved context before the system prompt, conversation history, or the user's own question are counted. Across 50,000 queries a month that is 128 million input tokens. Raise top-k to 10 and chunk size to 1,024 because "more context helps", and the same traffic now moves 512 million tokens for four times the cost. Nothing about the corpus changed. Two configuration fields did.
Bigger Chunks Are Not Automatically Better Chunks
Larger chunks preserve more surrounding context, which helps when an answer depends on a whole argument rather than a single fact. They also dilute the embedding: a vector representing 2,000 tokens covering four topics matches many queries weakly and none of them strongly, so genuinely relevant material can rank below a tightly focused chunk that happens to share vocabulary with the query. The retrieval-augmented generation approach described in the original RAG paper by Lewis and colleagues depends on the retriever surfacing genuinely relevant passages; chunks that cover too much ground make that job harder, not easier.
Small chunks have the opposite failure. They match precisely but arrive stripped of the context needed to interpret them — a paragraph that says "this limit does not apply to enterprise accounts" is useless without knowing which limit. A common middle path is to retrieve small and expand at read time: index tightly scoped chunks for matching, then pass the surrounding section to the model once a chunk is selected. That decouples what you match on from what you send, at the cost of a slightly more involved pipeline.
Boundaries Should Follow Structure, Not Token Counts
A fixed token window cuts wherever it lands, which on structured content means the middle of a table row, a code block, or a numbered list. Splitting on document structure first — headings, sections, list items — and only falling back to a token limit inside oversized sections produces chunks that make sense on their own and usually needs less overlap, because the boundaries are already in natural gaps.
The same applies to metadata. Prefixing each chunk with its document title and section heading costs a handful of tokens and makes an otherwise ambiguous fragment interpretable both to the embedding model and to the chat model reading it. Add that prefix length to your chunk size when estimating retrieval load here, since it is sent on every retrieved chunk. OpenAI's embeddings guide covers the input limits that constrain how large any single chunk can be.
Matching Chunk Size to Query Type
The right configuration depends less on the corpus than on what people ask. Lookup questions — a price, a policy clause, an error code — are best served by small, precise chunks and a low top-k, because the answer lives in one place and extra context only adds noise and cost. Synthesis questions that require comparing several sources need higher top-k, and often larger chunks, since the model has to reason across passages.
If your traffic mixes both, do not average them into one compromise setting that serves neither well. Classify the query and vary top-k at runtime: retrieve three chunks for a lookup and ten for a comparison. Run this calculator twice, once per query type, and weight the monthly costs by traffic share for a realistic total. Then check the whole assembled prompt fits comfortably in your model's window using the context window planner.
Arb Digital designs retrieval pipelines around how people actually query your content, with the token economics settled before the index is built.
Browse Free Tools Talk to Arb DigitalCommon Mistakes to Avoid
- Copying a tutorial's chunk size — defaults were chosen for a demo corpus and a demo query pattern, neither of which is yours.
- Setting overlap as a large share of chunk size — past roughly a quarter of the chunk, duplication climbs steeply for very little retrieval benefit.
- Raising top-k to fix poor retrieval — if the right chunk is not being found, more chunks mostly add cost and noise rather than the missing answer.
- Ignoring the prompt budget — retrieved chunks compete with system prompt, history, and output allowance inside a fixed context window.
- Chunking structured content by token count alone — tables, code, and lists split mid-record become unusable to both the retriever and the model.
Related Free Tools From Arb Digital
Size the text itself with the words to tokens converter, price indexing with the embedding cost calculator, cost storage using the vector database cost calculator, check the assembled prompt with the context window planner, and compare model rates in the LLM cost comparison tool. The free online tools hub has the rest.
Frequently Asked Questions
There is no universal answer, because it depends on your content structure and the kind of questions asked. Precise lookup questions favour smaller chunks; questions requiring synthesis across a passage favour larger ones. Test both against real queries rather than adopting a default.
Enough that a sentence or definition is not lost at a boundary, which for most prose is roughly 10 to 20 percent of chunk size. Above about a quarter of the chunk, storage and embedding duplication rise sharply for little added retrieval quality.
Not directly, since retrieval load depends on top-k and chunk size rather than overlap. Overlap increases the number of chunks stored and embedded, and it makes it more likely that two retrieved chunks contain some of the same text.
Because retrieved chunks are billed as input tokens on every generation call. Top-k multiplied by chunk size is the token load added to each prompt, so raising either one raises the cost of every query for as long as the system runs.
Structure-aware splitting generally produces more coherent chunks and needs less overlap, because boundaries fall in natural gaps. A token limit is still needed as a fallback for sections that exceed the maximum chunk length.
Yes, but it requires re-embedding the entire corpus, since the chunks themselves change. That is why it is worth modelling the cost of a configuration before committing to it rather than after.
Prefixing each chunk with its document title and section heading gives the embedding model and the reading model context that a bare fragment lacks. It adds a small number of tokens to every stored and retrieved chunk, so include it in your chunk size estimate.
Figures produced by this tool are planning estimates only — actual chunk counts depend on your splitter's implementation, your tokenizer, and how your documents are structured.
