Transformer Architecture Education LLM Tools

Educational implementations of transformer architectures from scratch using various frameworks (PyTorch, TensorFlow, NumPy), focusing on learning the core mechanisms. Does NOT include pre-trained model applications, fine-tuning frameworks, or production-ready inference systems.

There are 49 transformer architecture education tools tracked. 2 score above 50 (established tier). The highest-rated is LowinLi/transformers-stream-generator at 58/100 with 96 stars.

Get all 49 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=llm-tools&subcategory=transformer-architecture-education&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	LowinLi/transformers-stream-generator This is a text generation method which returns a generator, streaming out...	58	Established	96	Python
2	ystemsrx/mini-nanoGPT One-click training of your own GPT. Training a GPT has never been easier for...	55	Established	371	Python
3	jaymody/picoGPT An unnecessarily tiny implementation of GPT-2 in NumPy.	48	Emerging	3,454	Python
4	kyegomez/AttentionGrid A network of attention mechanisms at your fingertips. Unleash the potential...	44	Emerging	7	Python
5	kamalkraj/minGPT-TF A minimal TF2 re-implementation of the OpenAI GPT training	43	Emerging	58	Jupyter Notebook
6	abhaskumarsinha/MinimalGPT MinimalGPT is a concise, adaptable, and streamlined code framework that...	38	Emerging	24	Python
7	Eamon2009/Codeformer-A.I A character-level GPT transformer built from scratch in PyTorch, trained on...	38	Emerging	3	Python
8	NotShrirang/tinygpt 🎈 A series of lightweight GPT models featuring TinyGPT Base (~51M params)...	37	Emerging	16	Jupyter Notebook
9	BobMcDear/trap Autoregressive transformers in APL	36	Emerging	107	APL
10	liangyuwang/Tiny-Megatron Tiny-Megatron, a minimalistic re-implementation of the Megatron library	35	Emerging	23	Python
11	naity/protein-transformer Implement, train, tune, and evaluate a transformer model for antibody...	34	Emerging	6	Jupyter Notebook
12	Esmail-ibraheem/Transformer-pytorch Language to Language Transformer model from scartch using pure Pytorch where...	34	Emerging	9	Jupyter Notebook
13	akanyaani/minGPTF A TF re-implementation of the Karpathy's minGPT (Generative Pretrained...	33	Emerging	8	Python
14	pedrodiamel/gpt_mini_mini learning about llms	32	Emerging	3	Python
15	suvash/taxophoney GPT (Decoder only Transformer - from scratch) generated fake/phoney...	30	Emerging	2	Jupyter Notebook
16	DiogoNeves/mlx-gpt An approximate implementation of A. Karpathy's Let's build GPT, with MLX	30	Emerging	6	Jupyter Notebook
17	eonu/transformers-from-scratch Modular Python implementation of encoder-only, decoder-only and...	29	Experimental	1	Python
18	87owo/EasyGPT Easily train and inference on your personal computer, no need for large...	29	Experimental	35	Python
19	codewithdark-git/titans-transformer This repository contains an experimental implementation of the Titans...	24	Experimental	4	Jupyter Notebook
20	SvetimFM/transformer-training-interface Educational transformer neural network implementation with interactive...	23	Experimental	2	Python
21	MEHARKhaoula/transformers-text-coherence Fine-Tuning Five Types of Transformers and Incorporating Syntactic Features...	23	Experimental	4	Jupyter Notebook
22	thejvdev/gpt-from-scratch Repository for Implementing a Decoder-Only GPT Model	22	Experimental	—	Jupyter Notebook
23	Gauravchy09/mini-gpt-from-scratch Mini GPT implementation from scratch to explore transformer architecture,...	22	Experimental	—	Jupyter Notebook
24	JoshKeesee/Mini-GPT Train, run, and configure your own local AI from scratch! This includes a...	22	Experimental	—	Python
25	TolaniSilas/Mini-GPT This repository is an implementation of the GPT series.	22	Experimental	1	Jupyter Notebook
26	lengyueit/gpt-mini OpenAI GPT的简单复现	22	Experimental	20	Python
27	rbbrdckybk/MiniGPT-4 Simplified local Windows OS setup of MiniGPT-4 running in an Anaconda...	21	Experimental	9	Python
28	aitechnologies-it/gpt-mini Yet another minimalistic Tensorflow (re-)re-implementation of Karpathy's...	21	Experimental	14	Jupyter Notebook
29	cpparnell/honeycrisp miniGPT built on Apple's MLX array framework	20	Experimental	5	Python
30	brianberns/MinGptSharp A F# .NET TorchSharp port of Andrej Karpathy's minGPT.	20	Experimental	10	F#
31	TrainsFormerAI/TrainsFormerAI-1.0 TrainsFormer AI is a revolutionary desktop application that brings classic...	20	Experimental	1	Python
32	QuwsarOhi/PicoGPT A simple small GPT model for educative purpose that can be trained on CPU	18	Experimental	2	Python
33	tensor-fusion/GPT-Haskell A pure Haskell implementation of a decoder-only transformer (GPT)	18	Experimental	21	Haskell
34	HMels/BassTab_Transformer This The Model scrapes huge amounts of bass music tableture from the...	17	Experimental	1	Python
35	thaisaraujom/about-gpt-and-transformer This repository provides a comprehensive and intuitive explanation of the...	17	Experimental	1	—
36	mcrimi/snakeformer Transformer model from scratch to simulate the good old game of Snake 🐍.	17	Experimental	2	Python
37	sumony2j/SeedGPT SeedGPT is a minimal decoder-only LLM from scratch, inspired by andrej...	14	Experimental	1	Python
38	UEFI-code/miniGPT An open-source project to show how to build a mini language model using PyTorch	14	Experimental	1	Python
39	LarissaTchomgang/miniGPT 🛠 Build a GPT-style language model from scratch with MiniGPT, offering an...	13	Experimental	—	Python
40	am1tyadav/tensorflow-mingpt minGPT in TensorFlow	13	Experimental	—	Python
41	muhammadhussain-2009/Building-A-Transformer-From-Scratch Coding A Decoder Only Transformer Like ChatGPT From Scratch	13	Experimental	—	Jupyter Notebook
42	Anton3090/gpt2-mini-implementation 🧸 TinyStories Transformer — A lightweight 4-layer GPT-style model trained on...	13	Experimental	—	Jupyter Notebook
43	clam004/triton-ft-api tutorial on how to deploy a scalable autoregressive causal language model...	12	Experimental	5	Python
44	crimsonKn1ght/mini-gpt mni gpt based off andrej's course	12	Experimental	1	Python
45	vivek-rd/tinystoriesGPT Train GPT style model on tinystories dataset	11	Experimental	4	Python
46	chinonsochikelue/tiny-gpt Tiny GPT is a minimal GPT-style Transformer trained from scratch on a small...	11	Experimental	2	Python
47	Muhammad-Ibrahim-Khan/minigpt A miniGPT inspired from the original NanoGPT released by OpenAI. This is a...	11	Experimental	—	Jupyter Notebook
48	thibaud-perrin/mini-gpt The goal of this project was to implement the encoder only transformer in...	11	Experimental	—	Jupyter Notebook
49	kateonbxsh/MiniGPT AI library written in Typescript implementing a small prototype of GPT.	10	Experimental	2	TypeScript

Comparisons in this category

minGPT-TF and minGPTF (43 vs 33)