rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
This project provides the practical code and guidance to build your own custom GPT-like large language model (LLM) from the ground up. You'll learn how to take raw text data, process it, and train a functional LLM that can generate text or follow instructions. This is designed for AI practitioners, machine learning engineers, and researchers who want to deeply understand and implement LLMs.
87,892 stars. Actively maintained with 8 commits in the last 30 days.
Use this if you are a machine learning engineer or researcher who wants to learn the inner workings of large language models by implementing one yourself, rather than just using existing frameworks.
Not ideal if you are looking for a pre-built, production-ready LLM solution or a high-level API to integrate into an existing application without diving into the underlying code.
Stars
87,892
Forks
13,408
Language
Jupyter Notebook
License
—
Category
Last pushed
Mar 07, 2026
Commits (30d)
8
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/rasbt/LLMs-from-scratch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Related models
facebookresearch/LayerSkip
Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
FareedKhan-dev/train-llm-from-scratch
A straightforward method for training your LLM, from downloading data to generating text.
kmeng01/rome
Locating and editing factual associations in GPT (NeurIPS 2022)
datawhalechina/llms-from-scratch-cn
仅需Python基础，从0构建大语言模型；从0逐步构建GLM4\Llama3\RWKV6， 深入理解大模型原理
geeks-of-data/knowledge-gpt
Extract knowledge from all information sources using gpt and other language models. Index and...