codiceSpaghetti/numpyGPT
A from-scratch GPT built with NumPy and Python’s standard library. No autograd, no frameworks: every layer is re-implemented with its own forward and backward pass. Gradients are computed manually, updates are transparent, and every operation is spelled out.
This project helps machine learning engineers and researchers understand how large language models work from the ground up. By providing a re-implementation of a GPT model using only NumPy, it explicitly shows every mathematical operation involved in neural networks, tokenization, and optimization. You input raw text data, and it outputs a trained language model capable of generating new text, while also visualizing the training process.
Use this if you are a machine learning student, researcher, or engineer who wants a deep, transparent understanding of transformer architecture and backpropagation without the abstractions of higher-level frameworks.
Not ideal if you need to train large-scale, production-ready language models efficiently or are looking for a high-performance deep learning library.
Stars
20
Forks
1
Language
Python
License
—
Category
Last pushed
Nov 23, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/codiceSpaghetti/numpyGPT"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
tabularis-ai/be_great
A novel approach for synthesizing tabular data using pretrained large language models
EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron...
shibing624/textgen
TextGen: Implementation of Text Generation models, include LLaMA, BLOOM, GPT2, BART, T5, SongNet...
ai-forever/ru-gpts
Russian GPT3 models.
AdityaNG/kan-gpt
The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold...