liangyuwang/zo2
ZO2 (Zeroth-Order Offloading): Full Parameter Fine-Tuning 175B LLMs with 18GB GPU Memory [COLM2025]
Fine-tuning large language models (LLMs) often requires significant GPU memory, which can be a barrier for many users. This framework allows you to fine-tune very large LLMs, such as those with 175 billion parameters, using only 18GB of GPU memory by intelligently offloading computational tasks to the CPU. Researchers and machine learning practitioners who need to adapt powerful LLMs to specific tasks without access to high-end, multi-GPU systems would benefit from this.
202 stars. No commits in the last 6 months.
Use this if you need to fine-tune massive language models (like OPT-175B or Qwen3-32B) but are limited to a single GPU with relatively modest memory (e.g., 18GB or less).
Not ideal if you have ample GPU resources (multiple high-memory GPUs) and prefer traditional full-precision fine-tuning methods for maximum speed and simplicity.
Stars
202
Forks
18
Language
Python
License
Apache-2.0
Category
Last pushed
Jul 16, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/liangyuwang/zo2"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
hkproj/pytorch-llama
LLaMA 2 implemented from scratch in PyTorch
4AI/LS-LLaMA
A Simple but Powerful SOTA NER Model | Official Code For Label Supervised LLaMA Finetuning
luchangli03/export_llama_to_onnx
export llama to onnx
ayaka14732/llama-2-jax
JAX implementation of the Llama 2 model
harleyszhang/lite_llama
A light llama-like llm inference framework based on the triton kernel.