FayAndXan/spectrawl
The unified web layer for AI agents. Search (8 engines), stealth browse, auth, and act on 24 platforms. One npm install, self-hosted.
This project helps AI agents and automation workflows interact with websites by providing a unified way to search across multiple engines, browse pages stealthily, and extract content. It takes a web query or URL and returns rich, structured content from web pages, including full page scrapes, screenshots, and block detection details. Marketers, researchers, or anyone building AI agents that need to gather information from the web would find this useful.
Available on npm.
Use this if your AI agent or automated system needs to reliably search, browse, and extract content from websites, including those with anti-bot measures or login requirements.
Not ideal if you only need simple search snippets and don't require deep page scraping, stealth browsing, or interaction with specific platforms.
Stars
10
Forks
2
Language
JavaScript
License
MIT
Category
Last pushed
Mar 11, 2026
Commits (30d)
0
Dependencies
4
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/mcp/FayAndXan/spectrawl"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
apify/apify-mcp-server
The Apify MCP server enables your AI agents to extract data from social media, search engines,...
bartholomej/node-csfd-api
ČSFD API in JavaScript. Amazing NPM library for scrapping csfd.cz. Now with MCP server
brightdata/brightdata-mcp
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public...
ScrapeGraphAI/scrapegraph-mcp
ScapeGraph MCP Server
firecrawl/firecrawl-mcp-server
🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and...