dobbersc/fundus-evaluation
[ACL 2024] Evaluation of the Fundus News Scraper
This project helps researchers and data scientists compare the quality of different news article scraping tools. It takes raw HTML content from news articles and their corresponding manually extracted 'ground truth' text. The output is a detailed evaluation, including tables and plots, that shows how accurately each scraper extracts clean, complete news article text compared to the human-verified version. Researchers and data scientists who need to reliably gather large volumes of news content for analysis would use this.
No commits in the last 6 months.
Use this if you need to objectively assess and compare the performance of various news scraping tools to ensure you're extracting high-quality, artifact-free text for your research or data pipeline.
Not ideal if you're looking for a simple, out-of-the-box news scraper for direct use without needing to benchmark its extraction quality.
Stars
10
Forks
1
Language
Python
License
MIT
Category
Last pushed
Aug 14, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/dobbersc/fundus-evaluation"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
flairNLP/fundus
A very simple news crawler with a funny name
fhamborg/news-please
news-please - an integrated web crawler and information extractor for news that just works
affjljoo3581/canrevan
대량의 네이버 뉴스 기사를 수집하는 라이브러리입니다.
FreeDiscovery/FreeDiscovery
Web Service for E-Discovery Analytics
tirthajyoti/Web-Database-Analytics
Web scrapping and related analytics using Python tools