jgontrum/aiohttp-scraper
A robust asynchronous web scraping client using aiohttp.
This helps developers reliably gather public information from websites. It takes a list of URLs and returns the HTML or JSON content, making web data collection more robust. Developers building applications that need to collect data from many websites will find this useful for managing the complexities of web scraping.
No commits in the last 6 months.
Use this if you are a developer building a web scraper and need to handle common issues like connection errors, website throttling, and rotating IP addresses.
Not ideal if you are a non-technical user looking for a ready-to-use application to extract data from websites without writing code.
Stars
19
Forks
6
Language
Python
License
MIT
Category
Last pushed
Dec 08, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/jgontrum/aiohttp-scraper"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.