yujiosaka/headless-chrome-crawler
Distributed crawler powered by Headless Chrome
This tool helps developers gather data from dynamic websites that render content using modern JavaScript frameworks like React or Angular. It takes a list of URLs as input and provides the processed content from those web pages, including the ability to extract specific elements like titles or even screenshots. It's designed for developers building web scraping solutions or data collection services that need to interact with websites like a real browser.
5,699 stars. No commits in the last 6 months. Available on npm.
Use this if you need to reliably extract data from modern websites where content loads dynamically after the initial page request.
Not ideal if your primary goal is to scrape static HTML content or if you need a non-code solution for web data extraction.
Stars
5,699
Forks
405
Language
JavaScript
License
MIT
Category
Last pushed
Apr 29, 2023
Commits (30d)
0
Dependencies
7
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/yujiosaka/headless-chrome-crawler"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers....
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.