danijar/granular
Fast dataset format and loader
Granular is a tool for developers who work with large, complex datasets. It helps you store and load diverse data types like images, text, and numerical arrays efficiently. You put in raw data in various formats and get out a structured, performant dataset that can be easily accessed and processed, especially for machine learning workflows. It's designed for data engineers and machine learning engineers managing big data pipelines.
Available on PyPI.
Use this if you need a flexible and high-performance way to store and load custom datasets with diverse data types, especially when random access and resumable processing are important.
Not ideal if you're looking for a simple CSV or JSON file loader, or if your dataset is small and easily fits into memory.
Stars
24
Forks
2
Language
Python
License
MIT
Category
Last pushed
Mar 06, 2026
Commits (30d)
0
Dependencies
3
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/danijar/granular"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
pykt-team/pykt-toolkit
pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models
microsoft/archai
Accelerate your Neural Architecture Search (NAS) through fast, reproducible and modular research.
google-research/morph-net
Fast & Simple Resource-Constrained Learning of Deep Network Structure
IDEALLab/EngiBench
Benchmarks for automated engineering design
AI-team-UoA/pyJedAI
An open-source library that leverages Python’s data science ecosystem to build powerful...