eduardoleao052/Transformer-from-scratch
Educational Transformer from scratch (no autograd), with forward and backprop.
This is an educational project for developers who want to understand how a Transformer neural network works under the hood. It allows you to feed in any plain text file and it will learn to generate new text that mimics the style and content of your input. The primary users are machine learning engineers or researchers who are building or studying large language models.
No commits in the last 6 months.
Use this if you are a machine learning developer or researcher who wants to learn the inner workings of a Transformer model by implementing it from scratch, without relying on automatic differentiation frameworks.
Not ideal if you are looking for a high-performance, production-ready text generation tool or a library that leverages advanced deep learning frameworks for speed and ease of use.
Stars
36
Forks
—
Language
Python
License
MIT
Category
Last pushed
Apr 10, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/eduardoleao052/Transformer-from-scratch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
huggingface/transformers-bloom-inference
Fast Inference Solutions for BLOOM
Tencent/TurboTransformers
a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc)...
mit-han-lab/lite-transformer
[ICLR 2020] Lite Transformer with Long-Short Range Attention
mit-han-lab/hardware-aware-transformers
[ACL'20] HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
LibreTranslate/Locomotive
Toolkit for training/converting LibreTranslate compatible language models 🚂