adbar/trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
Trafilatura helps researchers, analysts, and marketers gather specific text content and its associated details from websites. It takes raw HTML from online sources or local files and transforms it into clean, structured data like articles, comments, and metadata. This is ideal for anyone needing to collect large amounts of web-based text for analysis without the 'noise' of headers or footers.
5,481 stars. Used by 28 other packages. No commits in the last 6 months. Available on PyPI.
Use this if you need to reliably extract main article text, metadata (like author or date), and even comments from web pages for research or content analysis.
Not ideal if you primarily need to extract numerical data from complex tables or interact with dynamic web elements requiring browser automation.
Stars
5,481
Forks
348
Language
Python
License
Apache-2.0
Category
Last pushed
Sep 12, 2025
Commits (30d)
0
Dependencies
7
Reverse dependents
28
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/adbar/trafilatura"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
any4ai/AnyCrawl
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts...
kreuzberg-dev/html-to-markdown
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the...
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file...
lightfeed/extractor
Using LLMs and AI browser automation to robustly extract web data