weiyu666/Graduation_Design-Distributed_Web_Spider
基于微博用户信息数据的分布式爬虫所做的毕业设计，有一小部分简单的数据分析。这个也是为了纪念大学四年！里面包括了源代码，论文的一稿二稿等等还有查重终稿，UML图 、PPT等等
This project helps you gather and understand large amounts of public user data from Sina Weibo. It takes the mobile web pages of Weibo users and extracts their profile information and social relationships, giving you structured data that can be used for basic analysis. Researchers, marketers, or social scientists interested in public social network data would find this useful.
No commits in the last 6 months.
Use this if you need to collect extensive public user profiles and their connections from Weibo for social research or market analysis.
Not ideal if you require real-time data, have ethical concerns about scraping public data, or need to analyze private user information.
Stars
78
Forks
6
Language
Python
License
MIT
Category
Last pushed
May 19, 2018
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/weiyu666/Graduation_Design-Distributed_Web_Spider"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.