alibaba/TePDist
TePDist (TEnsor Program DISTributed) is an HLO-level automatic distributed system for DL models.
TePDist is an infrastructure that automatically distributes the training of large deep learning models across multiple GPUs or machines. It takes a model's computational graph (in XLA HLO format) and figures out the best way to split the work, then manages the distributed training. Data scientists and AI researchers working with massive neural networks will find this useful for speeding up their training processes.
No commits in the last 6 months.
Use this if you need to train very large deep learning models efficiently across multiple GPUs or servers without manually configuring complex parallelization strategies.
Not ideal if you are working with small models that train quickly on a single GPU or if you prefer to manually control every aspect of your distributed training setup.
Stars
98
Forks
10
Language
C++
License
Apache-2.0
Category
Last pushed
Apr 22, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/alibaba/TePDist"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference...
helmholtz-analytics/heat
Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python
hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
horovod/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
bsc-wdc/dislib
The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.