Turn documentation sites into Markdown you can feed to an LLM, a wiki, or a static site
Documentation lives in HTML, but most of the places you want to reuse it, from AI assistants and RAG pipelines to Obsidian vaults and static site generators, prefer Markdown. This tool crawls a website section, strips the navigation, headers, footers, and scripts, and keeps the actual content: headings, paragraphs, lists, tables, images, links, and fenced code blocks with their language.
Every page becomes one .md file whose path mirrors the URL, so /dev/api/auction-house ends up as dev/api/auction-house.md. The ZIP also contains an index and, optionally, a single combined file that is handy as one big context document.
How the crawl works
- You enter a start URL. The path of that URL becomes the default scope, for example /dev.
- The page is loaded through a small proxy because browsers block cross-origin HTML reads. The proxy only returns the raw HTML; all parsing and conversion happens in your browser.
- Every link on the page that points to the same host and starts with the scope prefix is queued, up to the page limit you set. Assets such as images, PDFs, and scripts are skipped.
- Each page is converted to Markdown with the main content area detected automatically. You can stop at any time and still download what has finished.
Options explained
- Path scope: restricts the crawl. Set it to / to follow the entire site, or narrow it to a sub-section such as /dev/api.
- Max pages: a hard cap so a large site cannot run away. Start small to check the output quality, then raise it.
- Frontmatter: adds a YAML block with title, source URL, and meta description to every file, useful for Obsidian, Hugo, Astro, or Docusaurus imports.
- Combined file: writes _combined.md with all pages in crawl order, separated by horizontal rules. Ideal for pasting into an AI chat or building a single embedding source.
- Rewrite internal links: links between crawled pages are turned into relative .md links so the ZIP works offline and in Markdown editors. External links stay absolute.
Getting the best output
Sites built with Docusaurus, MkDocs, VitePress, Mintlify, GitBook, Nextra, and similar generators convert very well because they render semantic HTML with a clear <main> or <article> element. Single-page applications that render content only with JavaScript will return an almost empty page, since the tool reads the HTML the server sends rather than executing scripts.
Code blocks keep their language when the site annotates them with language-* classes, and tables are emitted as GitHub-flavored Markdown tables. Images are referenced by absolute URL and are not downloaded.
Privacy and fair use
The HTML passes through toolit's fetch proxy and is not stored. Conversion, ZIP packaging, and the download itself happen entirely in your browser. Please respect the terms of the sites you crawl, keep page limits reasonable, and prefer official exports when a project offers them.