dataflowr/llm_efficiency

KV Cache & LoRA for minGPT

/ 100

Emerging

This project helps developers working with Large Language Models (LLMs) to make their models run faster and fine-tune more efficiently. It provides implementations of KV Caching to speed up how LLMs generate text, and LoRA (Low-Rank Adaptation) to reduce the cost of adapting pre-trained models to new tasks. If you're building or customizing LLMs, you can use these techniques to optimize performance and resource use.

Use this if you are a machine learning engineer or researcher looking to improve the inference speed or reduce the fine-tuning costs of transformer-based language models.

Not ideal if you are an end-user of an LLM or are looking for a high-level API to interact with pre-trained models without needing to modify their internal workings.

Large Language Models LLM fine-tuning AI model optimization Machine Learning Engineering Deep Learning performance

No Package No Dependents

Maintenance 10 / 25

Adoption 8 / 25

Maturity 11 / 25

Community 12 / 25

How are scores calculated?

Stars

Forks

Language

Python

License

Apache-2.0

Compare

llm_efficiency and LMCache

Higher-rated alternatives

LMCache/LMCache

Supercharge Your LLM with the Fastest KV Cache Layer

Zefan-Cai/KVCache-Factory

Unified KV Cache Compression Methods for Auto-Regressive Models

OnlyTerp/kvtc

First open-source KVTC implementation (NVIDIA, ICLR 2026) -- 8-32x KV cache compression via PCA...

itsnamgyu/block-transformer

Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)

OnlyTerp/turboquant

First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache...

Explore Transformer Models

All categories Trending Transformer directory Insights