amerkurev/scrapper
Web scraper with a simple REST API living in Docker and using a Headless browser and Readability.js for parsing.
This tool helps you gather information from websites by automatically downloading pages and extracting article content into a clean, structured format, much like a browser's "Reader View." You provide a web address, and it returns the main article text and any related news links. This is ideal for content researchers, marketers, or data analysts who need to collect and process web-based text.
312 stars. No commits in the last 6 months.
Use this if you need to reliably extract clean article content from many different websites, including those with interactive elements or pop-ups, and want to automate this process.
Not ideal if you need to perform complex data extraction from specific, highly customized website layouts that go beyond standard article content.
Stars
312
Forks
52
Language
Python
License
MIT
Category
Last pushed
Apr 11, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/amerkurev/scrapper"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers....
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.