e1n00r/tinyserve

30 tok/s for 20B MoE on 8 GB VRAM. Flat throughput to 32K context. Native MXFP4 + GGUF Q4_K/Q5_K/Q6_K via ggml CUDA kernels — zero dequant. Expert offloading for models that don't fit in GPU memory.

39
/ 100
Emerging
No Package No Dependents
Maintenance 13 / 25
Adoption 4 / 25
Maturity 9 / 25
Community 13 / 25

How are scores calculated?

Stars

5

Forks

2

Language

Python

License

MIT

Last pushed

Apr 05, 2026

Commits (30d)

0

GitHub

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/e1n00r/tinyserve"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

campfirein/byterover-cli

ByteRover CLI (brv) - The portable memory layer for autonomous coding agents (formerly Cipher)

88

mistralai/client-python

Python client library for Mistral AI platform

88

openai/openai-python

The official Python library for the OpenAI API

86

pydantic/pydantic

Data validation using Python type hints

83

milla-jovovich/mempalace

The highest-scoring AI memory system ever benchmarked. And it's free.

82

Explore LLM Tools

All categories Trending LLM Tool directory Insights