Llm Implementation Tutorials Transformer Models

There are 111 llm implementation tutorials models tracked. 1 score above 70 (verified tier). The highest-rated is AI-Hypercomputer/maxtext at 82/100 with 2,169 stars. 3 of the top 10 are actively maintained.

Get all 111 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=transformers&subcategory=llm-implementation-tutorials&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Model	Score	Tier	Stars	Language
1	AI-Hypercomputer/maxtext A simple, performant and scalable Jax LLM!	82	Verified	2,169	Python
2	rasbt/reasoning-from-scratch Implement a reasoning LLM in PyTorch from scratch, step by step	65	Established	3,452	Jupyter Notebook
3	mindspore-lab/mindnlp MindSpore + 🤗Huggingface: Run any Transformers/Diffusers model on MindSpore...	64	Established	913	Python
4	mosaicml/llm-foundry LLM training code for Databricks foundation models	63	Established	4,397	Python
5	rickiepark/llm-from-scratch <밑바닥부터 만들면서 공부하는 LLM>(길벗, 2025)의 코드 저장소	55	Established	97	Jupyter Notebook
6	CASE-Lab-UMD/LLM-Drop The official implementation of the paper "Uncovering the Redundancy in...	52	Established	189	Python
7	rllm-team/rllm Pytorch Library for Relational Table Learning with LLMs.	51	Established	440	Python
8	ridgerchu/matmulfreellm Implementation for MatMul-free LM.	50	Established	3,058	Python
9	FareedKhan-dev/train-llama4 Building LLaMA 4 MoE from Scratch	45	Emerging	72	Jupyter Notebook
10	joyehuang/minimind-notes 🚀 [从零构建 LLM] 极简大模型训练原理与实践指南。包含 Transformer, Pretraining, SFT 核心代码与对照实验。 \| A...	44	Emerging	67	Python
11	donaldafeith/Pytorch_Merge Merge LLM that are split in to parts	44	Emerging	27	Python
12	xinzhanguo/hellollm pre train a new llm	44	Emerging	73	Python
13	Tongjilibo/build_MiniLLM_from_scratch 从0到1构建一个MiniLLM (pretrain+sft+dpo实践中)	43	Emerging	537	Python
14	AviSoori1x/seemore From scratch implementation of a vision language model in pure PyTorch	42	Emerging	255	Jupyter Notebook
15	hitz-zentroa/whisper-lm-transformers Add n-gram and LLM language model support to HF Transformers Whisper models.	42	Emerging	14	Python
16	shivendrra/SmallLanguageModel a LLM cookbook, for building your own from scratch, all the way from...	42	Emerging	168	Jupyter Notebook
17	JohnMachado11/Build-a-Large-Language-Model-from-Scratch Building a GPT-like LLM from scratch with PyTorch.	41	Emerging	337	Python
18	fangpin/llm-from-scratch Build LLM from scratch	41	Emerging	97	Python
19	ChaitanyaK77/Building-a-Small-Language-Model-SLM- This Repository provides a Jupyter Notebook for building a small language...	41	Emerging	32	Jupyter Notebook
20	ronniross/attention-heatmap-visualizer A set of scripts to generate full attention-head heatmaps for transformer-based LLMs	41	Emerging	13	Jupyter Notebook
21	NVIDIA/logits-processor-zoo A collection of LogitsProcessors to customize and enhance LLM behavior for...	41	Emerging	384	Python
22	zeyadusf/LLMs-from-Scratch Build a Large Language Model (From Scratch) book and Finetuned Models	40	Emerging	184	Jupyter Notebook
23	SPUTNIKAI/LeechTransformer Leech-Lila: A Geometric Attention Transformer(Language Model) with the Leech...	39	Emerging	4	Jupyter Notebook
24	rasbt/pytorch-memory-optim This code repository contains the code used for my "Optimizing Memory Usage...	38	Emerging	92	Python
25	ai-glimpse/toyllm ToyLLM: Learning LLM from Scratch	37	Emerging	25	Python
26	OpenNLPLab/TransnormerLLM Official implementation of TransNormerLLM: A Faster and Better LLM	35	Emerging	252	Python
27	Yigtwxx/PredictaLM PredictaLM is a lightweight Turkish language model designed for next-word...	35	Emerging	3	Python
28	GeeeekExplorer/transformers-patch patches for huggingface transformers to save memory	35	Emerging	35	Python
29	viralcode/superGPT Train your own LLM from scratch	35	Emerging	7	Python
30	hesamsheikh/llm-mechanics Coding an LLM and its building blocks from scratch.	34	Emerging	116	Jupyter Notebook
31	JoelDeonDsouza/Zenpool_LLM Zenpool is a compact, fine-tuned MLL (Mini Language Learner) model	33	Emerging	5	Jupyter Notebook
32	GURPREETKAURJETHRA/Multi-GPU-Fine-Training-LLMs Multi GPU Fine Training LLMs using DeepSpeed and Accelerate.	33	Emerging	2	Jupyter Notebook
33	waltonfuture/InstructionGPT-4 InstructionGPT-4	32	Emerging	42	Python
34	Arezkiiiii/mini_llm 🚀 Build and understand a Large Language Model from scratch using PyTorch...	32	Emerging	3	Jupyter Notebook
35	microsoft/encoder-decoder-slm Efficient encoder-decoder architecture for small language models (≤1B...	32	Emerging	32	Python
36	ksm26/Pretraining-LLMs Master the essential steps of pretraining large language models (LLMs)....	32	Emerging	27	Jupyter Notebook
37	myscience/x-lstm Pytorch implementation of the xLSTM model by Beck et al. (2024)	32	Emerging	183	Python
38	Utshav-paudel/LLM-Zero-to-Hero This repo contains the resources, projects and documentation of mine while...	31	Emerging	34	Jupyter Notebook
39	muna-ai/muna-predictors Interesting Python functions compiled to run anywhere with Muna.	31	Emerging	11	Python
40	Nikshaan/llm-from-scratch Implementation of build a LLM from scratch by Sebastian Raschka.	31	Emerging	15	Python
41	OpenVanguard/remma-o1 Remma-O1: An open-source Language Model with 1.17B Params, built on pytorch...	31	Emerging	34	Python
42	kmkrofficial/LiteGPT LiteGPT: A 124M Small Language Model (SLM) pre-trained on FineWeb and...	30	Emerging	34	Python
43	Ajax0564/VyomAI VyomAI: state-of-the-art NLP LLM Vision MultiModel transformers ...	30	Emerging	5	Python
44	Shinichi0713/LLM-fundamental-study this site is the fundamental page of LLM-mechanism	29	Experimental	3	Jupyter Notebook
45	ranpy13/Learning-LLM Learning to build LLM from scratch, following rasbt/LLMs-from-scratch footsteps.	28	Experimental	8	Jupyter Notebook
46	KillerShoaib/RLM-From-Scratch Implementation of Recursive Language Model paper from scratch	28	Experimental	38	Python
47	harshtiwari01/llm-heatmap-visualizer A set of scripts to generate full attention-head heatmaps for transformer-based LLMs	27	Experimental	2	Jupyter Notebook
48	AnkitaMungalpara/Building-LLM-From-Scratch This repository provides a step-by-step guide to creating your own large...	27	Experimental	1	Jupyter Notebook
49	Michael-Jackson666/Zero2Hero-AI From first principles to state-of-the-art: A hands-on journey implementing...	27	Experimental	2	Python
50	feifeibear/Odysseus-Transformer Odysseus: Playground of LLM Sequence Parallelism	27	Experimental	79	Python
51	SreeEswaran/Train-your-LLM This repository contains code and resources for training, fine-tuning, and...	25	Experimental	6	Python
52	MDalamin5/Build-and-Finetune-LLM-From-Scratch-Deploy-via-vLLM-AWS-GCP A complete end-to-end learning repo covering everything from building Large...	24	Experimental	—	Jupyter Notebook
53	mtkaya/transformer-edge-optimization Optimize Transformer models for edge devices	24	Experimental	1	Jupyter Notebook
54	simply-pouria/The-LMs-Book My study notes, code implementations, etc. while reading The Hundred-Page...	23	Experimental	1	Jupyter Notebook
55	MaharshPatelX/multi-model-conda-environment This GitHub repository provides a comprehensive guide on setting up a Conda...	22	Experimental	6	—
56	aryagxr/llm-from-scratch Implementation of LLM ✨from scratch✨	22	Experimental	2	Jupyter Notebook
57	musa-kal/smoll-language-model Educational language model implementation from scratch, featuring Bigram and...	22	Experimental	1	Python
58	adorow/LlmFromScratch Notes and code from reading "Build a Large Language Model (From Scratch)"	21	Experimental	—	Jupyter Notebook
59	Tek233/Build_a_Large_Language_Model_From_Scratch Testing and experimenting with code from Sebastian Raschka’s book Build a...	21	Experimental	—	Jupyter Notebook
60	mcbieda/llm-from-scratch 1. A simple implementation of the LLM gpt-2 "from scratch" that allows easy...	21	Experimental	—	Jupyter Notebook
61	Matonice/LLM_From_Scratch Implementing all the parts and architecture of a large language model from scratch	21	Experimental	—	Python
62	10-OASIS-01/Autoregressive-Language-Model This project is a comprehensive implementation of a Transformer-based...	21	Experimental	9	Python
63	filipsedivy/SLiM-CZ-V1 🇨🇿 Slavic Linguistic integrated Micro-model for Czechia	21	Experimental	—	Python
64	AIDajiangtang/LLM-from-scratch 从零开始学大模型Transformer、GPT2、BERT pre-training and fine-tuning from scratch	21	Experimental	37	Jupyter Notebook
65	villagecomputing/superpipe Superpipe - optimized LLM pipelines for structured data	21	Experimental	109	Python
66	CS-433/ml-project-2-mlp Advancing Homepage2Vec with LLM-Generated Datasets for Multilingual Website...	20	Experimental	5	Jupyter Notebook
67	JerryPan2718/flexgpt Tradeoff between runtime and RAM usage for large language model inference.	20	Experimental	6	Jupyter Notebook
68	fahadsid1770/Aesop-LLM custom Bengali Language Model built from scratch using PyTorch, featuring...	19	Experimental	—	Jupyter Notebook
69	prasanna00019/Small-Language-Models A collection of Small Language Models (SLMs) built from scratch in PyTorch.	19	Experimental	2	Jupyter Notebook
70	ictup/Build-a-LLM-model-from-scratch-simple LLM pipeline: data→tokenizer→attention→GPT train/eval→instruction...	19	Experimental	2	Jupyter Notebook
71	sanyalsunny111/Early_Weight_Avg [COLM 2024] Early Weight Averaging meets High Learning Rates for LLM Pre-training	19	Experimental	19	Python
72	CastorYu/train-hybrid-llm-from-scratch A simplistic script for training your own hybrid llm (using autoregressive...	19	Experimental	2	Python
73	daniel-mehta/CooperLM-354M A 354M parameter GPT-2 model trained on filtered Wikipedia, BookCorpus, and...	18	Experimental	1	Jupyter Notebook
74	Ki-Seki/Awesome-Transformer-Visualization Explore visualization tools for understanding Transformer-based large...	18	Experimental	22	—
75	ashrithssreddy/llm-from-scratch Building GPTs from the ground up. A hands-on journey through attention...	17	Experimental	—	Jupyter Notebook
76	MAlshurbaji/LLM-Playground-From-Scratch A hands-on educational repository demonstrating how to build a miniature LLM...	17	Experimental	—	Jupyter Notebook
77	T-Mohamed-Shafeek/LLM-for-Language-Translation This repository contains a simple and beginner-level noteboook which employs...	17	Experimental	1	Jupyter Notebook
78	jbergq/transformer-language-modeling Notebook showing how to implement and train a transformer for language modeling	17	Experimental	1	Jupyter Notebook
79	gusye1234/gpt-adapter Collections of algorithms for efficiently fine-tuning LLM	17	Experimental	1	Python
80	sasa5linkar/Spacy-Serbian-Transformer Using Jertehs corpuses to train spacy with tranformer and LLM	17	Experimental	1	Jupyter Notebook
81	FreezB11/PsyDuck a 60M parameter LLM from scratch	15	Experimental	2	Jupyter Notebook
82	ajheshbasnet/model-reincarnated a collection of re-implementations of renowned artificial intelligence...	14	Experimental	1	Jupyter Notebook
83	nachammai779/Large-Language-Models-from-Scratch 🧠 Building Large Language Models from the ground up — BPE tokenization,...	14	Experimental	—	Jupyter Notebook
84	Adwerse/Mini_LLM 🧠 Transformer built from scratch — RoPE, SwiGLU, KV-Cache, Flash Attention....	14	Experimental	—	Jupyter Notebook
85	rudyon/pipeline Training pipeline for LLMs in PyTorch.	14	Experimental	—	Python
86	itxmjr/LLM-From-Scratch A step-by-step Guide: Build a GPT-like LLM From Scratch using PyTorch	14	Experimental	—	Jupyter Notebook
87	Daegybyte/LLM_Training_Pipeline Modular text-generation training pipeline using PyTorch, Hugging Face, and...	13	Experimental	—	Jupyter Notebook
88	Samya-S/Building-LLMs-from-scratch A hands-on guide to implementing Large Language Models from scratch	13	Experimental	—	Jupyter Notebook
89	falseywinchnet/ConvexGPT Mostly Convex ICNN based Large Language Model	13	Experimental	—	Jupyter Notebook
90	sugam24/LLM-From-Scratch Building a Large Language Model from the ground up.	13	Experimental	—	Jupyter Notebook
91	AndrewKM210/micro-llm A micro-LLM implementation that should be able to train and run on a standard laptop	13	Experimental	—	Jupyter Notebook
92	hichamcharhi/llm-from-scratch Implémentez un LLM from scratch en Python — cours pédagogique en...	13	Experimental	—	Jupyter Notebook
93	insooeric/LLM_Small Long Language Model from SCRATCH	13	Experimental	—	Jupyter Notebook
94	patrikwolf/ttt_theory Specialization after Generalization	13	Experimental	6	Python
95	capecoder08/llm-playground Playing with tokenizers, transformers, and LLMs	13	Experimental	—	Jupyter Notebook
96	storytellingengineer/Learning_LLM_using_Pytorch Hi, this repo is my learning to build LLMs from scratch \| Foundational...	13	Experimental	—	Python
97	Jkanishkha0305/LLMs-from-Scratch A curated collection of Large Language Models(LLMs), Small Language...	13	Experimental	—	Jupyter Notebook
98	gkswamy98/sequence_model_il Contains sequence-model implementations of on and off-policy imitation...	12	Experimental	5	Jupyter Notebook
99	omkarbg100/LLM-GPT This is a LLM Model with full Architecture Coded ans is based On GPT-2 Weights	11	Experimental	—	Jupyter Notebook
100	juanprida/language-model-from-scratch We train various transformers to compose music like L.A.S.	11	Experimental	—	Jupyter Notebook
101	shankezh/LLMScratchs This repository contains a comprehensive implementation of the full...	11	Experimental	3	Python
102	Imran-ml/GEFS-language-detector GEFS Language Detection model deployed at Hugging Face	11	Experimental	3	—
103	Ahmadjamil888/CUSTOM-LLM-MODEL-DL A advanced deep learning AI model meant to be trained on your desired...	11	Experimental	2	Python
104	teatonedev/Llama3.2-From-Scratch This repository serves as an educational resource for building an...	11	Experimental	—	Python
105	0xnu/tiny_llm_trainer The experiment implements a tiny language model trainer using PyTorch.	11	Experimental	—	Python
106	jeffrey-fong/llms-from-scratch A collection of language model architectures implemented from scratch on PyTorch	11	Experimental	4	Python
107	GiuseppeBellamacina/Little_Language_Model Implementation of a Transformer and training on Dante's Divina Commedia	11	Experimental	3	Jupyter Notebook
108	KarthikArumugam3/Pretraining_LLM A demo of how to use Transformer models for pretraining LLMs	11	Experimental	—	Jupyter Notebook
109	myscience/jamba Pytorch (Lightning) Implementation of the Jamba Language Model	10	Experimental	2	—
110	sabamadadi/PersianWiki-LLM-FromScratch Implementation of a decoder-only transformer for Persian Wikipedia text...	10	Experimental	1	Jupyter Notebook
111	SyedAanif/build-llm Learn about the entire process of building a Large Language Model (LLM) from scratch	10	Experimental	1	Jupyter Notebook