DorsaRoh/transformer-from-scratch
Complete transformer from scratch, using only numpy
This project helps anyone working with language models to understand how a Transformer neural network processes sequential information. It takes an array of real numbers, representing pieces of information like words or sounds, and transforms them through layers to output a probability distribution of what comes next. This is for machine learning researchers or practitioners interested in the foundational mechanics of large language models.
No commits in the last 6 months.
Use this if you need to deeply understand the mathematical operations behind Transformer models for natural language processing or sequence prediction, without relying on high-level libraries.
Not ideal if you're looking for a pre-built, production-ready language model or a tool to quickly apply existing Transformer architectures to your data.
Stars
41
Forks
4
Language
Jupyter Notebook
License
—
Category
Last pushed
Aug 27, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/generative-ai/DorsaRoh/transformer-from-scratch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
NVIDIA-NeMo/NeMo
A scalable generative AI framework built for researchers and developers working on Large...
alexiglad/EBT
PyTorch Code for Energy-Based Transformers paper -- generalizable reasoning and scalable learning
vlm-run/vlmrun-hub
A hub for various industry-specific schemas to be used with VLMs.
HyperGAI/HPT
HPT - Open Multimodal LLMs from HyperGAI
yash9439/Falcon-Local-AI-Model
Explore this GitHub repository housing 3 versions of Falcon code for text generation. Each...