All Perception Tools

A framework for creating semi-automatic web content extractors

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

A free & open source IMDb front-end.

Attack Surface Discovery tool built on a microservice approach, utilizing...

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Netflix like full-stack application with SPA client and backend implemented...

Real-time detection of anti-bot systems, CAPTCHAs & fingerprinting...

A Chrome extension for writing custom web scraping programs and web...

COVID-19 Coronavirus data scraped from government and curated data sources.

A Python module to scrape several search engines (like Google, Yandex, Bing,...

Learn Python for the next 30 (or so) Days.

Example end to end data engineering project.

CLI tool for saving a faithful copy of a complete web page in a single HTML...

:rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web...

Ingest, parse, and optimize any data format ➡️ from documents to multimedia...

Simple but useful Python web scraping tutorial code.

The unix-way web crawler

An Unofficial API for 1337x, Piratebay, Nyaasi, Torlock, Torrent Galaxy,...

`scrape_linkedin` is a python package that allows you to scrape personal...

:newspaper: Read RSS feed from LeMonde.fr and display news inside the App

This whitepaper describes a new concept for building serverless microapps...

A script that lets you download mangas from mangafox.com

Extract social media links and account names from websites.

看理想 Python 客户端 / 下载器，下载看理想的音频和文稿

Unofficial Python client for youporn.com: search, metadata extraction, and...

HTTP client for Node.js with browser TLS fingerprint impersonation

Official Python library for interacting with Kameleo Client

Ferret CLI

Unofficial Python client for xfreehd.com: search, metadata extraction, and...

Multiple chrome extensions that speed up the tasks within MMU CliC.

A simple Telegram channel scraper

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用...

:movie_camera: R data package to explore Pixar films, the people, and reception data

Scraping the TikTok discovery web API every 15 minutes using Github Actions...

Turn any website into a typed API

[Unmaintained] A simple and clean video/music/image downloader 👾

This is a client module to automate the Mozilla Firefox browser via the...

Parse and traverse the SSR payload NUXT_DATA - Useful for scraping Nuxt.js websites

Image scraping library for creating deep learning datasets

Containerized Ferret worker

:soccer: Free API with results from national soccer competitions

📰 Build RSS 2.0 feeds from websites (and JSON APIs) automatically or with a...

Learn everything web scraping with David Teather Codes on YouTube

Home Assistant custom component for scraping (html, xml or json) multiple...

Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.

Generate Uber Eats accounts using IMAP or Hotmail emails using mobile packets

An infinite Pinterest crawler/scraper. Crawl image with inifnite-scroll!

The web scraper that's nearly impossible to block - now called @ulixee/hero

HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

Локальный API-прокси для Qwen AI с поддержкой сохранения контекста диалогов...

Generate 100% human-realistic Mouse Trajectories with Timings.

Discover and verify professional emails using names + domains

一个灵活、友好的爬虫框架

Python scripts for building 'Short Jokes' dataset, featured on Kaggle

Set up free and scalable Scrapyd cluster for distributed web-crawling with...

Python framework to scrape Pastebin pastes and analyze them

Automatically fetch and update proxy lists from multiple sources every 6...

Official Python SDK for the Supadata API.

A simple distributed crawler for zhihu && data analysis

SDK to access ZenRows API directly from Python. We handle proxies rotation,...

🌱 goClone - clone websites in seconds

Web crawling & scraping framework for Node.js on top of headless Chrome browser

Run Selenium with Python via Github Actions using Headless or Non-Headless browsers!

This is a Twitter Scraper which uses Selenium for scraping tweets. It is...

Learn how to leverage Python's amazing tools to scrape data from other...

Web-based browser profile manager for CloakBrowser — create, launch, and...

Python-based web crawling script with randomized intervals, user-agent...

Ignareo the Carillon, a web crawler/spider template of ultimate high...

A micro-framework for asynchronous deep crawls and web scraping with Python

GPC Web Crawler for detecting websites' compliance with GPC privacy...

TikTok User Info Scraper allows you to fetch detailed information about...

Command Line Tool to download torrents

Anti-detect browser for web scraping and automation. Engine-level...

PyMultiDictionary is a dictionary module that gets meanings, translations,...

DPULSE - Tool for complex approach to domain OSINT

Apify integration for Zapier

A no-code Google Maps lead scraper built with Google AI Studio (Gemini) and...

The simple, easy to use command line web crawler.

Stealth Chromium browser for large-scale web scraping.

Updated python-beginners docs and examples

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

Unofficial Python client for porntrex.com: search, metadata extraction, and...

Twitter Intelligence OSINT project performs tracking and analysis of the Twitter

A web scraping tool to systematically extract the text of scientific papers...

Chromium via noVNC (Browser in Browser)

reborn of https://bitbucket.org/rflechner/scrapysharp

🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)

Generate JSON representations of HTML tables

Scrappey.js: A versatile JavaScript wrapper for Scrappey API for solving...

A robust and flexible web scraping framework designed to run on Android...

Distributed task redisqueue(最简单python分布式函数调度框架)

Web Scaper for RSI

An unofficial API for Quora.

Terminal version of Cambridge Dictionary by default. Also supports the...

A Python tool for crawling and processing TradingView's PineScript V6...

A simple and easy to use web crawler for Python

Web Data Scraper - no-code internet scraping. Extract and export to CSV,...

Android app for saving webpages for offline reading.

Automated Aviator Betting Bot for Betika, Spribe & Other Aviator-style sites...

Bulk Downloader for Reddit