qhliu26/Dive-into-Big-Model-Training

📑 Dive into Big Model Training

/ 100

Emerging

This report explains how to train extremely large AI models, like those used for generating text or images, that require significant computational resources. It details various strategies to efficiently use hardware like GPUs, covering how to split data and model components across many machines. Researchers and machine learning engineers developing or working with very large neural networks will find this valuable.

115 stars. No commits in the last 6 months.

Use this if you need to understand the underlying principles and practical methodologies for scaling up the training of massive AI models to achieve state-of-the-art performance.

Not ideal if you are looking for an introduction to basic machine learning concepts or how to train smaller, more conventional neural networks on a single machine.

AI model training large language models distributed deep learning neural network scaling high-performance computing

No License Stale 6m No Package No Dependents

Maintenance 0 / 25

Adoption 10 / 25

Maturity 8 / 25

Community 12 / 25

How are scores calculated?

Stars

115

Forks

Language

—

License

—

Higher-rated alternatives

deepspeedai/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference...

helmholtz-analytics/heat

Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python

hpcaitech/ColossalAI

Making large AI models cheaper, faster and more accessible

horovod/horovod

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

bsc-wdc/dislib

The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.

Explore ML Frameworks

All categories Trending ML Framework directory Insights