datawhalechina/diy-llm
🎓 系统性大语言模型构建课程｜🛠️ 覆盖预训练数据工程、Tokenizer、Transformer、MoE、GPU 编程 (CUDA/Triton)、分布式训练、Scaling Laws、推理优化及对齐 (SFT/RLHF/GRPO)｜🚀 6 个渐进式作业 + 代码驱动，建立 LLM 全栈认知体系
This project is a comprehensive educational program that guides you through building a large language model (LLM) from scratch. It takes raw data and programming knowledge as input, and enables you to develop a fully functional LLM while understanding its core components and optimization techniques. This is ideal for deep learning practitioners and researchers looking to gain hands-on experience in LLM development.
Use this if you are a deep learning practitioner with Python and PyTorch experience who wants to build and understand large language models from the ground up.
Not ideal if you are looking for a pre-built LLM solution or lack foundational knowledge in Python, deep learning, and basic linear algebra.
Stars
62
Forks
10
Language
Jupyter Notebook
License
—
Category
Last pushed
Mar 09, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/datawhalechina/diy-llm"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
PacktPublishing/Mastering-NLP-from-Foundations-to-LLMs
Mastering NLP from Foundations to LLMs, Published by Packt
HandsOnLLM/Hands-On-Large-Language-Models
Official code repo for the O'Reilly Book - "Hands-On Large Language Models"
mlabonne/llm-course
Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
louisfb01/start-llms
A complete guide to start and improve your LLM skills in 2026 with little background in the...
Denis2054/Transformers-for-NLP-and-Computer-Vision-3rd-Edition
Transformers 3rd Edition