m92vyas/llm-reader
Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.
This tool helps developers transform raw webpage content into a clean, structured text format optimized for Large Language Models (LLMs). It takes a URL or HTML source code as input and outputs a simplified, easy-to-parse text that includes extracted website and image links, and table data. This is ideal for developers building AI-powered web scraping, data extraction, or RAG (Retrieval-Augmented Generation) applications.
280 stars.
Use this if you are a developer integrating web content into an LLM workflow and need to preprocess web pages for better data extraction and reduced LLM token usage.
Not ideal if you are an end-user without programming skills looking for a ready-to-use web scraping application.
Stars
280
Forks
19
Language
Python
License
MIT
Category
Last pushed
Feb 14, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/m92vyas/llm-reader"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
any4ai/AnyCrawl
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts...
kreuzberg-dev/html-to-markdown
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the...
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
adbar/trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping,...
paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file...