hominee/dyer
Dyer is designed for reliable, flexible and fast web crawling, providing some high-level, comprehensive features without compromising speed.
Dyer helps developers quickly gather specific information from websites by automating the process of visiting web pages and extracting structured data. It takes website URLs or rules for navigating sites as input and outputs the desired text, images, or other content in a structured format. This tool is ideal for software developers building applications that require large-scale data collection from the web.
127 stars. No commits in the last 6 months.
Use this if you need to build a custom application for fast, reliable, and flexible web scraping or data processing from online sources.
Not ideal if you're a non-developer looking for a no-code solution to extract data, or if you only need to scrape a few pages manually.
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/hominee/dyer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.