knagrecha/hydra
Execution framework for multi-task model parallelism. Enables the training of arbitrarily large models with a single GPU, with linear speedups for multi-gpu multi-task execution.
This tool helps machine learning engineers and researchers train extremely large deep learning models, even with limited GPU memory. It takes your PyTorch models and data loaders, then efficiently trains them on a single GPU or across multiple GPUs, allowing you to work with models far larger than typically possible. The output is a trained deep learning model, optimized for multi-task execution.
No commits in the last 6 months.
Use this if you need to train deep learning models with billions of parameters or run multiple model training tasks simultaneously on a single server with limited GPU resources.
Not ideal if your models are recurrent neural networks, if you require optimizers other than SGD, or if you are planning a multi-node, distributed training setup.
Stars
21
Forks
3
Language
Python
License
Apache-2.0
Category
Last pushed
Aug 13, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/knagrecha/hydra"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
qualcomm/ai-hub-models
Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized...
petuum/adaptdl
Resource-adaptive cluster scheduler for deep learning training.
zszazi/Deep-learning-in-cloud
List of Deep Learning Cloud Providers
lincc-frameworks/hyrax
Hyrax - A low-code framework for rapid experimentation with ML & unsupervised discovery in astronomy
intel/ai-reference-models
Intel® AI Reference Models: contains Intel optimizations for running deep learning workloads on...