rodricios/wxpath
wxpath - declarative web crawling with XPath; a Web Query Language (WQL)
This tool helps you gather information from websites by describing what links to follow and what content to extract using a single expression. You provide a starting URL and an XPath expression, and it gives you structured data like titles, descriptions, and other page elements. It's ideal for market researchers, data analysts, or content strategists who need to collect specific information from multiple web pages.
108 stars. Available on PyPI.
Use this if you need to extract structured data by recursively navigating a website's links, like building a knowledge graph from Wikipedia or collecting product information from an e-commerce site.
Not ideal if you need to interact with dynamic web elements, fill out forms, or automate complex browser actions beyond simply following links and extracting content.
Stars
108
Forks
5
Language
Python
License
MIT
Category
Last pushed
Feb 28, 2026
Commits (30d)
0
Dependencies
4
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/rodricios/wxpath"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
any4ai/AnyCrawl
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts...
kreuzberg-dev/html-to-markdown
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the...
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
adbar/trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping,...
paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file...