daijro/hrequests
🚀 Web scraping for humans
This tool helps you gather information from websites reliably and efficiently, even from complex sites that try to block automated access. It takes a web address (URL) and returns the page's content, just as a human browsing would see it. Anyone who needs to extract data from websites for research, market analysis, or competitive intelligence would find this useful.
997 stars. Used by 1 other package. No commits in the last 6 months. Available on PyPI.
Use this if you need to extract data from websites that are difficult to access with standard tools, or if you require the data to be rendered by a full browser.
Not ideal if your data extraction needs are simple and only involve static website content, or if you prefer a no-code solution.
Stars
997
Forks
69
Language
Python
License
MIT
Category
Last pushed
Dec 01, 2024
Commits (30d)
0
Dependencies
17
Reverse dependents
1
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/daijro/hrequests"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers....
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.