sondotpin/llm-scratch
ChatGPT-like LLM in PyTorch from scratch, step by step
This project helps machine learning engineers and researchers understand and implement large language models (LLMs) from the ground up. You'll put in foundational knowledge of neural networks and Python, and get out a fully functional GPT-like LLM, including pretraining, finetuning for classification, and instruction-following capabilities. It's designed for those who want to build and customize their own LLMs.
No commits in the last 6 months.
Use this if you are a machine learning engineer or researcher who wants to learn the inner workings of large language models by building one from scratch.
Not ideal if you are looking for a pre-built solution or a high-level library to simply use an LLM without understanding its underlying architecture.
Stars
17
Forks
4
Language
Jupyter Notebook
License
—
Category
Last pushed
Apr 30, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/sondotpin/llm-scratch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
liangyuwang/Tiny-DeepSpeed
Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library
catherinesyeh/attention-viz
Visualizing query-key interactions in language + vision transformers (VIS 2023)
microsoft/Text2Grad
🚀 Text2Grad: Converting natural language feedback into gradient signals for precise model...
huangjia2019/llm-gpt
From classic NLP to modern LLMs: building language models step by step. 异æ¥å›¾ä¹¦ï¼šã€Š GPT图解 å¤§æ¨¡åž‹æ˜¯æ€Žæ ·æž„å»ºçš„ã€‹-...