Skip to content

Text Chunker for RAG

Split long documents into chunks for embeddings and retrieval-augmented generation: by size in tokens or characters, with overlap, and along paragraph, sentence or Markdown heading boundaries. The result is a JSON array of chunks with their sizes.

Reset

Frequently asked questions

Which splitting modes are there?

Recursive splits at paragraphs and, for long paragraphs, at sentences. Sentences keeps whole sentences together. Headings starts new sections at Markdown headings. Fixed cuts the text into equal pieces regardless of structure.

How is the size measured?

In tokens with the o200k_base tokenizer used by current OpenAI models, or in characters. Pieces longer than the size are cut at the limit.

What does overlap do?

Each chunk repeats the end of the previous one, up to the overlap size, so context that spans a boundary is not lost. 10% to 20% of the chunk size is a common choice.

Which chunk size should I use?

It depends on the embedding model and the questions. 200 to 800 tokens is a common range: smaller chunks retrieve precise passages, larger ones keep more context.

What does the output look like?

A JSON array where each chunk has its index, its size in the chosen unit and its text, ready to send to an embedding API.

How long can the text be?

Up to 2 million characters, and at most 5,000 chunks per run.

About OneClick Pro

OneClick Pro is a free collection of more than 150 web tools for everyday work: converting images and units, formatting and validating code, checking domains and SEO, and working with text, dates and numbers. Every tool runs without registration or subscription, and what you enter is only used to calculate your result and is never stored. Use it as a quick toolbox for one-off tasks, or bookmark the tools you rely on every day.