LLM Implementation Tutorials LLM Tools

Educational repositories and step-by-step guides for building LLMs from scratch using frameworks like PyTorch and Transformers. Includes basic implementations, minimal examples, and learning-focused projects. Does NOT include production LLM frameworks, pre-trained model fine-tuning, or inference optimization tools.

There are 49 llm implementation tutorials tools tracked. 1 score above 70 (verified tier). The highest-rated is Lightning-AI/litgpt at 72/100 with 13,225 stars. 1 of the top 10 are actively maintained.

Get all 49 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=llm-tools&subcategory=llm-implementation-tutorials&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	Lightning-AI/litgpt 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.	72	Verified	13,225	Python
2	liangyuwang/Tiny-DeepSpeed Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library	43	Emerging	50	Python
3	catherinesyeh/attention-viz Visualizing query-key interactions in language + vision transformers (VIS 2023)	42	Emerging	162	HTML
4	microsoft/Text2Grad 🚀 Text2Grad: Converting natural language feedback into gradient signals for...	42	Emerging	31	Python
5	FareedKhan-dev/Building-llama3-from-scratch LLaMA 3 is one of the most promising open-source model after Mistral, we...	40	Emerging	203	Jupyter Notebook
6	huangjia2019/llm-gpt From classic NLP to modern LLMs: building language models step by step....	40	Emerging	194	Jupyter Notebook
7	sondotpin/llm-scratch ChatGPT-like LLM in PyTorch from scratch, step by step	39	Emerging	17	Jupyter Notebook
8	FareedKhan-dev/train-tiny-llm Train a 29M parameter GPT from Scratch	39	Emerging	34	Python
9	CoinCheung/gdGPT Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline...	37	Emerging	97	Python
10	better-py/learn-py Test code was written for research and verification of some Python libraries.	36	Emerging	14	Jupyter Notebook
11	loke-x/jam-gpt An Experimental Reimplementation of LLM models for research and development process	33	Emerging	21	Jupyter Notebook
12	RWKV-Wiki/MultilingualShareGPT MultilingualShareGPT, the free multi-language corpus for LLM training	33	Emerging	73	—
13	MuzzammilShah/Road-to-AI A structured documentation hub for AI and ML concepts, based on Andrej...	32	Emerging	19	—
14	wozeparrot/tinyrwkv tinygrad port of the RWKV large language model.	31	Emerging	45	Python
15	logic-OT/Decoder-Only-LLM This repository features a custom-built decoder-only language model (LLM)...	29	Experimental	16	Jupyter Notebook
16	liangyuwang/train-large-model-from-scratch A minimal, hackable pre-training stack for GPT-style language models	29	Experimental	7	Python
17	MistyozAI/CosmicFish CosmicFish is a fully original, efficient transformer-based language model...	29	Experimental	3	Python
18	ag-sc/CRAM-Generation-LLM Using ChatGPT & GPT-4 to generate CRAM & PyCRAM designators in a one-shot fashion	29	Experimental	4	Python
19	kreasof-ai/Homunculus-Project Long term project about a custom AI architecture. Consist of cutting-edge...	28	Experimental	8	Python
20	IAAR-Shanghai/Awesome-Attention-Heads An awesome repository & A comprehensive survey on interpretability of LLM...	28	Experimental	400	TeX
21	max1ab/smol-training-playbook-zh Hugging Face 文章《The Smol Training Playbook》中文翻译	27	Experimental	1	HTML
22	wln20/Attention-Viewer A plug-and-play tool for visualizing attention-score heatmap in generative...	26	Experimental	51	Python
23	sheikhcoders/SheikhLM Blazingly fast and remarkably powerful.	26	Experimental	1	Python
24	benthecoder/AI learning AI from scratch	24	Experimental	15	Jupyter Notebook
25	njmarko/llm-gpt-sort A new type of sorting algorithm. Use large language model (llm like gpt,...	24	Experimental	12	Jupyter Notebook
26	hulkiciray/llm_from_scratch Comprehensive fresh start for how LLMs work	23	Experimental	4	Jupyter Notebook
27	snnclsr/chatgpt-from-scratch A full-stack ChatGPT-like application built (almost) from scratch	23	Experimental	7	Python
28	Kitsunp/Small-lenguaje-Model-Hybrid-Norm-Furier-Formers A compact language model implementing HybridNorm and Fourier-based...	21	Experimental	4	Python
29	moorebrett0/microformer A minimal, dual-memory transformer language model built from scratch in...	20	Experimental	4	Python
30	FranxYao/Language-Model-Pretraining-for-Text-Generation LM pretraining for generation, reading list, resources, conference mappings.	19	Experimental	20	—
31	bobo0810/MiniGPT-4-DeepSpeed MiniGPT-4基于DeepSpeed加速➕ 扩充模型规模 ➕ 实验分析	19	Experimental	4	—
32	UEFI-code/PyTorch_For_PoorGuys This is a special PyTorch For Poor Guys Who can't afford big GPU	19	Experimental	4	C++
33	anafisa/Text2Text-Transformer Large Multi-Language Models for News Translation	17	Experimental	1	Jupyter Notebook
34	pprp/smol_training_zh 《Smol 训练手册》：打造世界级大模型的秘诀	16	Experimental	9	Shell
35	ikun-llm/ikun-2.5B 练习时长两年半的 AI 大模型 (实际 26M params，2.5B = 两年半) \| ikun meme-culture chatbot 🐔🏀	16	Experimental	2	Jinja
36	llami-team/gpt-torch Compress the HTML as much as possible for LLM to inference.	16	Experimental	44	—
37	ariannamethod/RRPRAM RRPRAM: Recurcsive Resonant Pattern Recognition Attention Mechanism by Arianna Method	15	Experimental	1	C
38	RajdeepKushwaha5/LLMs-from-scratch This repository contains the code for developing, pretraining, and...	14	Experimental	—	Jupyter Notebook
39	feljost/fgpt Building a conversational LLM from scratch	14	Experimental	1	Python
40	ikun-llm/ikun-pretrain 从零预训练语言模型 \| Pretrain LLM from scratch 📚	14	Experimental	—	—
41	Antoinechss/LLM-from-scratch Building a LLM (GPT-3 like architecture) from scratch in Python and training...	14	Experimental	1	Python
42	Bhardwaj-Saurabh/Build_A_Small_Language_Model A detail guide to build, train, fine-tune and deploye SLM. This repo...	13	Experimental	—	Jupyter Notebook
43	PalaashAgrawal/allmond LLM Training made Quick and Easy	12	Experimental	7	Python
44	adityanaranje/SLM-From-Scratch NanoChat is a decoder-only causal language model built from scratch, showing...	12	Experimental	1	Jupyter Notebook
45	aayushsharma-io/tensorix Tensorix is a minimalistic and highly compatible Python library designed to...	12	Experimental	1	—
46	aaaastark/NBART-Multilingual-Translator This repository contains a Python script that uses a pre-trained NBART...	12	Experimental	1	Jupyter Notebook
47	rescuerz/LLMs-from-scratch Implement a ChatGPT-like LLM in PyTorch from scratch, step by step	11	Experimental	—	Jupyter Notebook
48	jvscursulim/build-a-llm-from-scratch Studies about LLM	11	Experimental	—	Jupyter Notebook
49	B0-B/blowtorch-transformer-api LLM bootstrap loader for local CPU/GPU inference with fully customizable chat.	11	Experimental	4	Python

Comparisons in this category

litgpt and llm-gpt (72 vs 40) litgpt and jam-gpt (72 vs 33)