cerbero90/lazy-json-pages
📜 Framework-agnostic API scraper to load items from any paginated JSON API into a Laravel lazy collection via async HTTP requests.
This is a developer tool designed to efficiently retrieve data from online services that provide their information through paginated JSON APIs. It helps developers collect items from multiple pages of data, like user lists or product catalogs, and process them in a memory-efficient way. The typical user is a PHP developer who needs to integrate data from external APIs into their Laravel applications.
196 stars. No commits in the last 6 months.
Use this if you are a PHP developer working with Laravel and need to pull large datasets from paginated JSON APIs without running out of memory.
Not ideal if you are not a PHP developer, or if you need to read large JSON files that are not paginated from an API.
Stars
196
Forks
2
Language
PHP
License
MIT
Category
Last pushed
Sep 13, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/cerbero90/lazy-json-pages"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Altimis/Scweet
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers,...
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser...
plabayo/rama
modular service framework to move and transform network packets
scrapinghub/spidermon
Scrapy Extension for monitoring spiders execution.