NLPatVCU/PaperScraper
A web scraping tool to systematically extract the text of scientific papers and corresponding metadata from university accessible journals.
This tool helps researchers, academics, and data scientists systematically gather scientific journal articles. You provide a journal article's URL or an identifier like a DOI or PubMed ID, and it returns a structured JSON object containing the article's full text, abstract, authors, keywords, and other metadata. It's designed for anyone needing to build datasets of scientific literature for analysis.
221 stars. No commits in the last 6 months.
Use this if you need to programmatically extract the full text and metadata from scientific journal articles to create a corpus for text analysis, literature reviews, or other research purposes.
Not ideal if you only need to read or download a few articles manually, or if your required journals are not supported by the current scraping capabilities.
Stars
221
Forks
69
Language
Python
License
GPL-3.0
Category
Last pushed
Oct 14, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/NLPatVCU/PaperScraper"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers....
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.