hpcaitech/Elixir
Elixir: Train a Large Language Model on a Small GPU Cluster
This project helps machine learning engineers train very large language models efficiently, even with a smaller cluster of GPUs. It takes your model and optimizer configurations and automatically determines the most memory-efficient way to distribute parameters and manage memory across CPUs and GPUs. This is for ML engineers and researchers working on large-scale model training.
No commits in the last 6 months.
Use this if you need to train extremely large language models but are constrained by the memory capacity of your existing GPU cluster.
Not ideal if you are working with smaller models or already have access to a very large, high-memory GPU cluster.
Stars
15
Forks
5
Language
Python
License
—
Category
Last pushed
Jun 08, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/hpcaitech/Elixir"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
NX-AI/xlstm
Official repository of the xLSTM.
sinanuozdemir/oreilly-hands-on-gpt-llm
Mastering the Art of Scalable and Efficient AI Model Deployment
DashyDashOrg/pandas-llm
Pandas-LLM
wxhcore/bumblecore
An LLM training framework built from the ground up, featuring a custom BumbleBee architecture...
MiniMax-AI/MiniMax-01
The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model &...