Transformer Architecture Education ML Frameworks

Educational repositories focused on implementing transformer models from scratch to understand core components and mechanisms. Includes tutorials, explanations, and hands-on implementations of attention, positional encoding, and encoder-decoder structures. Does NOT include pre-trained model usage, applications (translation, BERT fine-tuning), or production frameworks.

There are 26 transformer architecture education frameworks tracked. 1 score above 50 (established tier). The highest-rated is lvapeab/nmt-keras at 51/100 with 531 stars.

Get all 26 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=transformer-architecture-education&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	lvapeab/nmt-keras Neural Machine Translation with Keras	51	Established	531	Python
2	dair-ai/Transformers-Recipe 🧠 A study guide to learn about Transformers	46	Emerging	1,624	—
3	SirawitC/Transformer_from_scratch_pytorch Build a transformer model from scratch using pytorch to understand its inner...	44	Emerging	42	Python
4	jaketae/ensemble-transformers Ensembling Hugging Face transformers made easy	42	Emerging	61	Python
5	lof310/transformer PyTorch implementation of the current SOTA Transformer. Configurable,...	39	Emerging	3	Python
6	jiangtaoxie/SoT SoT: Delving Deeper into Classification Head for Transformer	29	Experimental	50	Python
7	L-Zhe/FasySeq A fast and easy implementation of Transformer with PyTorch.	29	Experimental	7	Python
8	leeway0507/Transformer_from_scratch Transformer 구현 및 학습 방법 설명	27	Experimental	6	Jupyter Notebook
9	anmolg1997/SLM-From-Scratch Build small language models from scratch — BPE tokenizer, composable...	22	Experimental	—	Python
10	cosimo17/transformer_notebook Transformer turorial. Transformer教程	22	Experimental	6	Jupyter Notebook
11	dianjiang75/Transformer A decoder-only Transformer built entirely from scratch in PyTorch. Trained...	21	Experimental	—	Jupyter Notebook
12	mfarisadip/Multi-X-Transformers A neural network based on the encoder-decoder architecture the modeling...	20	Experimental	5	Python
13	msi1427/Original-Transformer-for-Bengali-Translation A neural machine translation project for Bengali Translation where the...	20	Experimental	8	Jupyter Notebook
14	wj-Mcat/transformer-handbook add transformer related blogs & codes	19	Experimental	3	HTML
15	Ayush-Aditya/decoder-only-seq2seq Minimal decoder-only seq2seq pipeline with proper causal masking, teacher...	17	Experimental	6	Python
16	ankushhKapoor/transformer-from-scratch Transformer from scratch implementation in PyTorch for Neural Machine...	15	Experimental	8	Python
17	Banniesdread/decoder-only-seq2seq Implement a decoder-only Transformer in PyTorch to reverse character...	14	Experimental	—	Python
18	konodiodaaaaa1/PyTorch-Transformer-From-Scratch A numerical stable implementation of Transformer from scratch using PyTorch....	14	Experimental	3	Python
19	sosopop/deepcalc 基于Transformer解码器实现的数学表达式计算器，通过思维链生成中间推理步骤提升计算准确率。本项目可作为学习Transformer架构和推理策略的实践案例。	13	Experimental	3	Python
20	Joe-Naz01/encoder-decoder This PyTorch notebook implements a complete Transformer architecture from...	13	Experimental	—	Jupyter Notebook
21	dino65-dev/Transformers Transformers from scratch implemented GQA,RoPE,RMS-Norm and trained on that code	13	Experimental	—	Jupyter Notebook
22	Xachchchch/deberta-fine-tune-comparison Experimenting with LoRA vs head-only fine-tuning for DeBERTa on sentiment analysis	13	Experimental	—	Jupyter Notebook
23	Ekoda/LanguageModelFromScratch An autoregressive transformer language model from first principles, built...	11	Experimental	—	Python
24	DolbyUUU/Learn_Transformer_from_Easy_to_Hard 一个为初学者设计的 Transformer 分阶段学习项目。通过由简到难的四个阶段，从零开始用 PyTorch 构建完整的 Transformer...	11	Experimental	2	Jupyter Notebook
25	calvin-fei/transformers Rebuild transformer model from scratch.	11	Experimental	—	—
26	JihoonChung/CTC-Product-and-Store-Vectorization This project is a collaboration between the University of Toronto and...	10	Experimental	2	—

Comparisons in this category

Transformer_from_scratch_pytorch and Transformer_from_scratch (44 vs 27)