rowyio/LLM-Web-Crawler
Web Scraper and Crawler for LLM Apps and AI Workflows with NoCode / LowCode. Plug and play with your own logic and customize it flexibly and scalably on BuildShip.
This tool helps you gather information from websites to create custom AI assistants or knowledge bases. You provide website URLs or even an entire site, and it extracts the text content or specific structured data you define. This is for anyone building AI-powered tools who needs to feed them fresh, relevant information from the web.
No commits in the last 6 months.
Use this if you need to pull data from websites to build a custom 'Chat with Data' application, power an AI assistant, or create a knowledge base for your business.
Not ideal if you're looking for a simple, one-off data extraction tool without needing to integrate it into an AI workflow or complex data pipeline.
Stars
31
Forks
8
Language
TypeScript
License
—
Category
Last pushed
Jul 11, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/rowyio/LLM-Web-Crawler"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
carlosplanchon/spidercreator
Automated web scraping spider generation using Browser Use and LLMs. Streamline the creation of...
raznem/parsera
Lightweight library for scraping web-sites with LLMs
rednafi/html-to-text
Extract pure text from any webpage
supadata-ai/js
Official TypeScript/JavaScript SDK for the Supadata API.
yeahhe365/JustSearch
基于 Playwright 的自主 AI 搜索智能体。支持迭代式任务规划、深度网页爬取，以及带引用来源的多源知识整合。