mensfeld/llm-docs-builder
Transform and optimize your markdown documentation for Large Language Models (LLMs) and RAG systems. Generate llms.txt automatically.
This tool helps technical writers, content managers, and documentation teams optimize their existing markdown documentation for use with AI systems like Large Language Models (LLMs) and RAG (Retrieval Augmented Generation). It takes your standard documentation files or URLs as input, cleans them by removing unnecessary content, and outputs leaner, more efficient versions specifically tailored for AI, often reducing the text an AI needs to process by over 80%. This ensures AI-powered assistants can answer questions more accurately and cost-effectively from your docs.
Use this if you manage technical documentation and want to ensure that AI bots and LLM-powered systems can efficiently and accurately retrieve information from your content without wasting resources on human-centric design elements.
Not ideal if your documentation is not in markdown or HTML, or if you only need documentation for human readers and have no plans to integrate with AI question-answering systems.
Stars
80
Forks
3
Language
Ruby
License
MIT
Category
Last pushed
Mar 12, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/mensfeld/llm-docs-builder"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
any4ai/AnyCrawl
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts...
kreuzberg-dev/html-to-markdown
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the...
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
adbar/trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping,...
paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file...