RAG Cleaner

RAG Cleaner

Paste text or HTML — or drop a .txt, .md, or .html file — and this page converts it to clean Markdown, then splits it into deterministic chunks. Download the result as Markdown, JSONL, or a ZIP bundle with a chunk manifest. Identical input always produces identical chunks, ids, and files.

Private by default: your text is processed entirely in this tab. It is never sent to a server, never logged, and never stored. This page makes no network requests with your content.

Clean & chunk

Auto-detect treats input that starts with < as HTML. HTML keeps headings, lists, tables, links, and emphasis as Markdown; script and style blocks are dropped.

Paste text or HTML, or use the file picker below. Files are read locally and never leave your device.

Choose a .txt, .md, or .html file, or drop one here

No file selected yet.

Chunks never exceed this size (50–8000).

Repeats the tail of each chunk at the start of the next (0–500).

Keeps sections together and splits on # headings.

Ready. Paste text or HTML, then press Clean & chunk.

How it works

The pipeline has three deterministic stages, all running in this tab:

  1. Normalize. Plain text is normalized (Unicode NFC, CRLF/CR to LF, trailing-whitespace removal, blank-line collapse). HTML is parsed with a lenient tokenizer and rendered to Markdown — headings, lists, tables, links, and emphasis are preserved, while scripts, styles, and other non-content blocks are dropped.
  2. Chunk. The Markdown is split into blocks and combined up to your maximum size. With heading-aware mode, a new chunk starts at every # heading. Oversized single blocks are split at word boundaries when possible.
  3. Stabilize. Every chunk gets a content-derived id (FNV-1a over the document id plus the chunk index) and a sequential index. The same input and options always produce the same ids, texts, and files — nothing random, nothing timestamped.

Downloads are generated with the browser's Blob API and never touch the network. The ZIP bundle contains one Markdown file per chunk, a full output.md, output.jsonl, and a manifest.json listing each chunk id, index, and size.

Supported inputs and limits

Frequently asked questions

Does my content leave my computer?

No. The page is a static bundle of JavaScript; normalization, chunking, and file generation all run in your browser tab. There is no server call that receives your text.

Why are the chunk ids always the same?

Chunk ids are derived from the content itself (a stable hash of the document id plus the chunk index), not from the clock or random numbers. That makes re-running, diffing, and deduplicating chunks reliable.

What is the manifest for?

manifest.json records the options you used, the document id, and every chunk's id, index, and character count — no content. It lets you prove which chunks came from which settings without re-processing.

Will tables and links survive conversion?

Yes. Tables become GFM pipe tables, links become [text](url), and images become ![alt](src). Cell content is escaped so a pipe inside a cell cannot break the table.

Part of Local Toolworks. Last reviewed: 2026-08-09.