JohnMachado11/Build-a-Large-Language-Model-from-Scratch
Building a GPT-like LLM from scratch with PyTorch.
This project helps machine learning engineers and researchers understand the core mechanics of large language models (LLMs) by guiding them through building one from the ground up. You'll input raw text data and configuration parameters, and the output will be a functional, albeit small, language model. This is for professionals who want to deeply grasp how models like ChatGPT are constructed.
337 stars. No commits in the last 6 months.
Use this if you are a machine learning practitioner who wants to learn the internal workings of large language models by actively building one.
Not ideal if you are looking for a pre-built, production-ready LLM to use out-of-the-box for applications, or if you don't have a background in machine learning and programming.
Stars
337
Forks
83
Language
Python
License
—
Category
Last pushed
Dec 20, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/JohnMachado11/Build-a-Large-Language-Model-from-Scratch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AI-Hypercomputer/maxtext
A simple, performant and scalable Jax LLM!
rasbt/reasoning-from-scratch
Implement a reasoning LLM in PyTorch from scratch, step by step
mindspore-lab/mindnlp
MindSpore + 🤗Huggingface: Run any Transformers/Diffusers model on MindSpore with seamless...
mosaicml/llm-foundry
LLM training code for Databricks foundation models
rickiepark/llm-from-scratch
<밑바닥부터 만들면서 공부하는 LLM>(길벗, 2025)의 코드 저장소