LLMs-from-scratch and llm-scratch-pytorch
These are competitors offering similar educational approaches to implementing transformers from scratch in PyTorch, with the former being a far more established and comprehensive resource (87k+ stars vs 100 stars) that would likely supersede the latter for most learners.
About LLMs-from-scratch
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
This project provides the practical code and guidance to build your own custom GPT-like large language model (LLM) from the ground up. You'll learn how to take raw text data, process it, and train a functional LLM that can generate text or follow instructions. This is designed for AI practitioners, machine learning engineers, and researchers who want to deeply understand and implement LLMs.
About llm-scratch-pytorch
skyloevil/llm-scratch-pytorch
lm-scratch-pytorch - The code is designed to be beginner-friendly, with a focus on understanding the fundamentals of PyTorch and implementing LLMs from scratch,step by step.
This project helps aspiring machine learning engineers and researchers understand how large language models (LLMs) like GPT-2 are built from the ground up using PyTorch. It guides you step-by-step through implementing the core components, starting from basic PyTorch concepts, all the way to optimizing performance with techniques like Flash Attention. You'll work with actual LLM architectures and gain practical knowledge of their internal workings.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work