MarHai/ScrapeBot
A Selenium-driven tool for automated website interaction and scraping.
This tool helps computational social scientists and researchers automate regular visits to websites, interact with web pages (like clicking buttons or filling forms), and collect specific data. You provide a set of web-based tasks, and it outputs structured data and logs of its interactions. It's designed for academics and researchers who need to monitor or gather information from the web systematically over time, often for 'agent-based testing' or large-scale data collection.
No commits in the last 6 months.
Use this if you need to regularly perform the same web-based tasks, like checking for updates on a site, collecting data from specific pages, or simulating user interactions across many variations for academic research.
Not ideal if you're looking for a simple, one-off data extraction tool without the need for agent-based testing or complex, recurring web interactions.
Stars
20
Forks
7
Language
Python
License
GPL-2.0
Category
Last pushed
Sep 01, 2021
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/MarHai/ScrapeBot"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers....
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.