gamemann/web-extract-with-chatgpt
A Python project that extracts data from websites with the option to process the data through @openai's ChatGPT API. The results are either printed to stdout or sent via a POST request.
This project helps you automatically pull content from websites, then optionally process it using ChatGPT. You provide a web page URL, and it gives you either the full text from the page body or specific forum topic content, which can then be summarized, analyzed, or transformed by ChatGPT. This is useful for researchers, content strategists, or anyone needing to gather and understand information from specific web pages without manual copy-pasting.
No commits in the last 6 months.
Use this if you need to programmatically extract text from a website, especially from a Discourse forum topic, and want to leverage ChatGPT to process or summarize that content.
Not ideal if you need to interact extensively with JavaScript-heavy sites that require waiting for dynamic content to load, or if you primarily work on Windows without making specific adjustments.
Stars
22
Forks
1
Language
Python
License
—
Category
Last pushed
Jan 20, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/gamemann/web-extract-with-chatgpt"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jamesturk/scrapeghost
👻 Experimental library for scraping websites using OpenAI's GPT API.
Priyanshu-hawk/ChatGPT-unofficial-api-selenium
This is unofficial ChatGPT API using selenium for prompt testing and flow testing purposes
3281448091/easyChatGPT
An unofficial yet elegant interface of the ChatGPT API using browser automation that bypasses...
ryuseisan/auto-chatgpt
Automate interaction with the browser version of ChatGPT.
Ryaang/gpt-web-crawler
A web crawler for GPTs to build knowledge bases 用于GPT构建知识库的网站爬虫