sjdirect/abotx
Cross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.
This is a C# web crawler designed for developers to systematically gather information from websites. It takes in one or more website URLs and can extract the content of pages, including those that render with JavaScript. Developers would use this to build applications that need to collect web data at scale.
139 stars. No commits in the last 6 months.
Use this if you are a C# developer building an application that needs to extract data from multiple websites, handle JavaScript rendering, and manage crawl speed and concurrency.
Not ideal if you need a no-code solution for basic data extraction, or if you are not a C# developer.
Stars
139
Forks
24
Language
C#
License
—
Category
Last pushed
Oct 02, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/sjdirect/abotx"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers....
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.