skylight-org/sparse-attention-hub
Advancing the frontier of efficient AI
This framework helps AI researchers and developers working with large language models efficiently manage and evaluate the 'attention' mechanism. It takes in various sparse attention configurations and a chosen large language model, then outputs performance benchmarks across multiple long-context datasets. The primary users are AI research engineers and machine learning scientists focused on optimizing transformer models for longer text inputs.
Use this if you need to experiment with, implement, and rigorously benchmark different sparse attention algorithms for large language models to improve efficiency and performance on long-context tasks.
Not ideal if you are an end-user of an AI application and not directly involved in the research and development of transformer model architectures.
Stars
54
Forks
5
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 10, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/skylight-org/sparse-attention-hub"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
fla-org/flash-linear-attention
🚀 Efficient implementations of state-of-the-art linear attention models
thu-ml/SageAttention
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x...
thu-ml/SpargeAttn
[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
fla-org/flame
🔥 A minimal training framework for scaling FLA models
foundation-model-stack/fms-fsdp
🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for...