intergalacticalvariable/reader
📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simple prefix http://127.0.0.1:3000/https://website-to-scrape.com/
This tool helps developers and AI engineers prepare web content for Large Language Models (LLMs) and agent systems. It takes any public URL as input and converts its content into a clean, LLM-friendly format like Markdown or plain text. You can also generate screenshots. This is ideal for those building RAG systems, agents, or other AI applications that need to process information directly from websites.
295 stars. No commits in the last 6 months.
Use this if you need to feed web page content to your AI models or agents in a structured, clean format and prefer to run the processing tool on your own local infrastructure without external API keys.
Not ideal if you need to parse content directly from PDF documents, as this tool is currently focused solely on web URLs.
Stars
295
Forks
55
Language
TypeScript
License
Apache-2.0
Category
Last pushed
Jul 18, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/intergalacticalvariable/reader"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
any4ai/AnyCrawl
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts...
kreuzberg-dev/html-to-markdown
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the...
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
adbar/trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping,...
paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file...