All Perception Tools
10,540 tools ranked by quality score
| # | Tool | Score | Tier |
|---|---|---|---|
| 1 |
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python. |
|
Verified |
| 2 |
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection. |
|
Verified |
| 3 |
browser-use/browser-use
🌐 Make websites accessible for AI agents. Automate tasks online with ease. |
|
Verified |
| 4 |
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets,... |
|
Verified |
| 5 |
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build... |
|
Verified |
| 6 |
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can... |
|
Verified |
| 7 |
plabayo/rama
modular service framework to move and transform network packets |
|
Verified |
| 8 |
intoli/user-agents
A JavaScript library for generating random user agents with data that's... |
|
Verified |
| 9 |
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build... |
|
Verified |
| 10 |
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution. |
|
Verified |
| 11 |
D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single... |
|
Verified |
| 12 |
soxoj/maigret
🕵️♂️ Collect a dossier on a person by username from 3000+ sites |
|
Verified |
| 13 |
firecrawl/firecrawl
🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or... |
|
Verified |
| 14 |
freelawproject/juriscraper
An API to scrape American court websites for metadata. |
|
Verified |
| 15 |
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library. |
|
Verified |
| 16 |
0x676e67/wreq-python
An ergonomic Python HTTP Client with TLS fingerprint |
|
Verified |
| 17 |
fake-useragent/fake-useragent
Up-to-date simple useragent faker with real world database |
|
Verified |
| 18 |
php-curl-class/php-curl-class
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs |
|
Verified |
| 19 |
orangecoding/fredy
❤️ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for... |
|
Verified |
| 20 |
bellingcat/auto-archiver
Automatically archive links to videos, images, and social media content from... |
|
Verified |
| 21 |
apache/stormcrawler
A scalable, mature and versatile web crawler based on Apache Storm |
|
Verified |
| 22 |
sardanioss/httpcloak
Go HTTP client with browser-identical TLS/HTTP2 fingerprinting. Bypass bot... |
|
Verified |
| 23 |
jaebradley/basketball_reference_web_scraper
NBA Stats API via Basketball Reference |
|
Verified |
| 24 |
rushter/selectolax
Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS... |
|
Verified |
| 25 |
JustinBeckwith/linkinator
Broken link checker that crawls websites and validates links. Find broken... |
|
Verified |
| 26 |
Kaliiiiiiiiii-Vinyzu/patchright-python
Undetected Python version of the Playwright testing and automation library. |
|
Verified |
| 27 |
saifyxpro/HeadlessX
The undetected self-hosted browser automation platform. Powered by Camoufox... |
|
Verified |
| 28 |
serpapi/google-search-results-python
Google Search Results via SERP API pip Python Package |
|
Verified |
| 29 |
scrapfly/scrapfly-scrapers
Scalable Python web scraping scripts for +40 popular domains |
|
Verified |
| 30 |
scrapy/parsel
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors |
|
Verified |
| 31 |
umihico/docker-selenium-lambda
The simplest demo of chrome automation by python and selenium in AWS Lambda |
|
Verified |
| 32 |
omkarcloud/botasaurus
The All in One Framework to Build Undefeatable Scrapers |
|
Verified |
| 33 |
drudge/n8n-nodes-puppeteer
n8n node for browser automation using Puppeteer |
|
Verified |
| 34 |
iiab/iiab
Internet-in-a-Box - Build your own LIBRARY OF ALEXANDRIA with a Raspberry Pi ! |
|
Verified |
| 35 |
getmaxun/maxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI... |
|
Verified |
| 36 |
microlinkhq/browserless
The headless Chrome/Chromium driver on top of Puppeteer. Take screenshots,... |
|
Verified |
| 37 |
datawhores/OF-Scraper
A completely revamped and redesigned fork, reimagined from scratch based on... |
|
Verified |
| 38 |
jordantete/OddsHarvester
A python app designed to scrape and process sports betting data directly... |
|
Established |
| 39 |
autoscrape-labs/pydoll
Pydoll is a library for automating chromium-based browsers without a... |
|
Established |
| 40 |
neon-mmd/websurfx
:rocket: An open source alternative to searx which provides a modern-looking... |
|
Established |
| 41 |
ndgigliotti/shopify-spy
Extract structured data from Shopify websites. |
|
Established |
| 42 |
Lookyloo/lookyloo
Lookyloo is a web interface that allows users to capture a website page and... |
|
Established |
| 43 |
lightpanda-io/browser
Lightpanda: the headless browser designed for AI and automation |
|
Established |
| 44 |
AIPexStudio/AIPex
AIPex: AI browser automation assistant, no migration and privacy first.... |
|
Established |
| 45 |
apify/apify-client-python
Apify API client for Python |
|
Established |
| 46 |
deedy5/primp
HTTP client that can impersonate web browsers |
|
Established |
| 47 |
mxschmitt/awesome-playwright
A curated list of awesome tools, utils and projects using Playwright |
|
Established |
| 48 |
gosom/google-maps-scraper
scrape data from Google Maps. Extracts data such as the name, address,... |
|
Established |
| 49 |
webrecorder/browsertrix-crawler
Run a high-fidelity browser-based web archiving crawler in a single Docker container |
|
Established |
| 50 |
0x676e67/wreq-util
Common utilities for wreq |
|
Established |
| 51 |
lorien/awesome-web-scraping
List of libraries, tools and APIs for web scraping and data processing. |
|
Established |
| 52 |
tinyfish-io/agentql
AgentQL is a suite of tools for connecting your AI to the web. Featuring a... |
|
Established |
| 53 |
dgtlmoon/changedetection.io
Best and simplest tool for website change detection, web page monitoring,... |
|
Established |
| 54 |
A9T9/RPA
Ui.Vision Open-Source RPA Software with Computer Vision, OCR, Anthropic... |
|
Established |
| 55 |
Skyvern-AI/skyvern
Automate browser based workflows with AI |
|
Established |
| 56 |
lewisdonovan/google-news-scraper
Lightweight scraper for Google News |
|
Established |
| 57 |
botswin/BotBrowser
Advanced Privacy Browser Core with Unified Fingerprint Defense: Cloudflare,... |
|
Established |
| 58 |
d60/twikit
Twitter API Scraper | Without an API key | Twitter Internal API | Free |... |
|
Established |
| 59 |
MontFerret/ferret
Declarative web scraping |
|
Established |
| 60 |
daijro/browserforge
🎭 Intelligent browser header & fingerprint generator |
|
Established |
| 61 |
okfn-brasil/querido-diario
📰 Diários oficiais brasileiros acessíveis a todos | 📰 Brazilian... |
|
Established |
| 62 |
ZacharyHampton/HomeHarvest
Python package for scraping real estate property data |
|
Established |
| 63 |
karust/openserp
Get [Google, Yandex, Baidu, Bing, DuckDuckGo] search results via API for free 🎉 |
|
Established |
| 64 |
programminghistorian/jekyll
Jekyll-based static site for The Programming Historian |
|
Established |
| 65 |
mherrmann/helium
Lighter web automation with Python |
|
Established |
| 66 |
scrapinghub/web-poet
Web scraping Page Objects core library |
|
Established |
| 67 |
EchterAlsFake/PHUB
A lightweight API for Pornhub |
|
Established |
| 68 |
EchterAlsFake/xnxx_api
Unofficial Python client for xnxx.com: search, metadata extraction, and... |
|
Established |
| 69 |
jpjacobpadilla/Stealth-Requests
Undetected web-scraping & seamless HTML parsing in Python! |
|
Established |
| 70 |
AndyTheFactory/newspaper4k
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles,... |
|
Established |
| 71 |
themm1/procyclingstats
procyclingstats scraper |
|
Established |
| 72 |
franckferman/MetaDetective
Unleash Metadata Intelligence with MetaDetective. Your Assistant Beyond Metagoofil. |
|
Established |
| 73 |
pinchtab/pinchtab
High-performance browser automation bridge and multi-instance orchestrator... |
|
Established |
| 74 |
iamatulsingh/pinscrape
A simple library to scrape Pinterest images. |
|
Established |
| 75 |
claffin/cloudproxy
Hide your scrapers IP behind the cloud. Provision proxy servers across... |
|
Established |
| 76 |
nirholas/crypto-vision
The most comprehensive cryptocurrency API. Real-time prices, OHLCV, order... |
|
Established |
| 77 |
MarginaliaSearch/MarginaliaSearch
Internet search engine for text-oriented websites. Indexing the small, old... |
|
Established |
| 78 |
City-Bureau/city-scrapers
Scrape, standardize and share public meetings from local government websites |
|
Established |
| 79 |
tomasbedrich/pycaching
A Python 3 interface for working with Geocaching.com website. |
|
Established |
| 80 |
lit26/finvizfinance
Finviz analysis python library. |
|
Established |
| 81 |
jensjeflensje/marktplaats-py
Small Python package to request listings & users from marktplaats.nl |
|
Established |
| 82 |
scrapy-plugins/scrapy-zyte-api
Zyte API integration for Scrapy |
|
Established |
| 83 |
ecoron/SerpScrap
SEO python scraper to extract data from major searchengine result pages.... |
|
Established |
| 84 |
PhialsBasement/LibreCrawl
Free desktop SEO crawler - open source alternative to Screaming Frog and... |
|
Established |
| 85 |
outscraper/outscraper-python
The library provides convenient access to the Outscraper API from... |
|
Established |
| 86 |
snooppr/snoop
Snoop — инструмент разведки на основе открытых данных (OSINT world) |
|
Established |
| 87 |
nuhmanpk/WebScrapper
Powerful Telegram bot for web scraping and crawling. Fast, easy, and loved... |
|
Established |
| 88 |
lumpinif/deepcrawl
100% free and full open-source edge Firecrawl alternative with better links... |
|
Established |
| 89 |
anilabhadatta/educative.io_scraper
Educative.io Course Downloader developed using Python and Selenium. Refer... |
|
Established |
| 90 |
apify/fingerprint-suite
Browser fingerprinting tools for anonymizing your scrapers. Developed by Apify. |
|
Established |
| 91 |
vibesurf-ai/VibeSurf
A powerful browser assistant for vibe surfing 一个开源的AI浏览器智能助手 |
|
Established |
| 92 |
AnonCatalyst/Ominis-OSINT
This Python application is an OSINT (Open Source Intelligence) tool called... |
|
Established |
| 93 |
wikimedia/html-metadata
MetaData html scraper and parser for Node.js (supports Promises only) |
|
Established |
| 94 |
viu-media/viu
Your browser anime experience from the terminal |
|
Established |
| 95 |
ispras/web-scraper-chrome-extension
Web data extraction tool implemented as chrome extension |
|
Established |
| 96 |
Norconex/crawlers
Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for... |
|
Established |
| 97 |
ultrafunkamsterdam/undetected-chromedriver
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation... |
|
Established |
| 98 |
WayneChang65/ptt-crawler
ptt-crawler is a web crawler module designed to scarpe data from Ptt. |
|
Established |
| 99 |
dcstats/CBBpy
A Python-based web scraper for NCAA basketball. |
|
Established |
| 100 |
ing-bank/INGenious
INGenious Playwright Studio |
|
Established |