hermit-crab/ScrapeMate
Scraping assistant tool. Editing and maintaining CSS/XPath selectors across webpages.
This tool helps web developers or data analysts efficiently identify and manage the specific parts of a webpage (like product names or prices) you want to extract for tasks such as website testing or data scraping. You interactively select elements on a live webpage, and it provides the necessary technical codes (CSS/XPath selectors) you can then use with other scraping or testing programs. It's for anyone who needs to pinpoint and maintain data elements from multiple webpages.
125 stars. No commits in the last 6 months.
Use this if you regularly need to extract specific pieces of information from websites and need a quick, interactive way to generate and manage the technical selectors required for your data scraping or automated testing tools.
Not ideal if you need a complete, end-to-end data scraping solution that also handles the actual extraction and storage of data, as this tool focuses solely on selector generation and management.
Stars
125
Forks
15
Language
JavaScript
License
GPL-3.0
Category
Last pushed
May 19, 2018
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/hermit-crab/ScrapeMate"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.