psf/requests-html
Pythonic HTML Parsing for Humans™
This tool helps you quickly gather specific information from websites, even those with dynamic content loaded by JavaScript. You provide web page addresses, and it delivers the content of those pages, allowing you to easily extract elements like text, links, or images. It's designed for anyone who needs to systematically collect data from multiple web pages for analysis or reporting.
13,863 stars. No commits in the last 6 months.
Use this if you need to extract structured data from websites for tasks like market research, content aggregation, or competitor analysis, especially when the websites rely heavily on JavaScript.
Not ideal if you only need to download entire web pages without needing to parse or extract specific elements from their HTML or if you prefer a graphical user interface for data extraction.
Stars
13,863
Forks
1,003
Language
Python
License
MIT
Category
Last pushed
Apr 16, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/psf/requests-html"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.