hkproj/pytorch-transformer-distributed
Distributed training (multi-node) of a Transformer model
This project helps machine learning engineers train large transformer models more efficiently by distributing the computational load across multiple GPU-enabled machines. You provide your transformer model code and training data, and it outputs a trained model faster than single-machine setups. This is for machine learning engineers and researchers working with substantial AI models.
No commits in the last 6 months.
Use this if you need to accelerate the training of a large transformer model that is too computationally intensive for a single GPU machine.
Not ideal if you are training smaller models or do not have access to a multi-node, multi-GPU cloud computing environment.
Stars
94
Forks
40
Language
Python
License
—
Category
Last pushed
Apr 10, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/hkproj/pytorch-transformer-distributed"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference...
helmholtz-analytics/heat
Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python
hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
horovod/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
bsc-wdc/dislib
The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.