speed/newcrawler
Free Web Scraping Tool with Java
This tool helps you automatically gather information from websites. You tell it which websites to look at, and it systematically collects the data you need, such as product prices, contact details, or news articles. The output is structured data that you can then analyze or use in other applications. It's designed for data analysts, marketers, or business intelligence professionals who need to collect large amounts of public web data.
586 stars. No commits in the last 6 months.
Use this if you need to regularly collect publicly available data from various websites for competitive analysis, market research, lead generation, or content aggregation.
Not ideal if you need to scrape data from websites that require complex authentication, CAPTCHA solving, or have very aggressive anti-scraping measures, or if you only need to extract data from a single page occasionally.
Stars
586
Forks
111
Language
JavaScript
License
—
Category
Last pushed
Nov 25, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/speed/newcrawler"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.