codingforentrepreneurs/Web-Scraping-with-Django-Celery
Learn how to schedule regular web scraping, save the data, and more with Django & Celery.
This project helps developers build systems that automatically gather data from websites at regular intervals. It takes instructions on what data to extract and from where, then stores the collected information in a database. This is for Python and Django developers who need to implement robust, scheduled web scraping functionalities within their applications, typically for business intelligence, data analysis, or content aggregation tasks.
No commits in the last 6 months.
Use this if you are a Django developer needing to integrate reliable, scheduled web scraping into a web application, handling data storage and task management.
Not ideal if you are looking for a no-code solution or a simple, one-off script for basic data extraction without needing a full application framework or scheduling.
Stars
33
Forks
10
Language
Jupyter Notebook
License
MIT
Category
Last pushed
Mar 18, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/codingforentrepreneurs/Web-Scraping-with-Django-Celery"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers....
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.