zetta-app/llama.cpp_turboquant
LLM inference with 7x KV cache compression. Combines llama.cpp (production inference engine) with TurboQuant (KV quantization). Run 131K token context on 16GB VRAM. OpenAI-compatible API server. Supports 100+ model architectures.
Stars
—
Forks
—
Language
Shell
License
—
Last pushed
Apr 09, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/zetta-app/llama.cpp_turboquant"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
fastapi/fastapi
FastAPI framework, high performance, easy to learn, fast to code, ready for production
scikit-learn/scikit-learn
scikit-learn: machine learning in Python
probabl-ai/skore
Track your Data Science. Skore's open-source Python library accelerates ML model development...
Farama-Foundation/Gymnasium
An API standard for single-agent reinforcement learning environments, with popular reference...
WMD-group/SMACT
Python package to aid materials design and informatics