LLMs-from-scratch and create-million-parameter-llm-from-scratch
The first is a comprehensive educational guide covering the full LLM architecture and training pipeline, while the second is a focused implementation of a specific model variant (LLaMA 1 with 2.3M parameters), making them complements that serve different depths of learning rather than competitors.
About LLMs-from-scratch
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
This project provides the practical code and guidance to build your own custom GPT-like large language model (LLM) from the ground up. You'll learn how to take raw text data, process it, and train a functional LLM that can generate text or follow instructions. This is designed for AI practitioners, machine learning engineers, and researchers who want to deeply understand and implement LLMs.
About create-million-parameter-llm-from-scratch
FareedKhan-dev/create-million-parameter-llm-from-scratch
Building a 2.3M-parameter LLM from scratch with LLaMA 1 architecture.
This project guides machine learning engineers and researchers through the practical steps of building a small-scale Large Language Model (LLM) using the LLaMA 1 architecture. It takes raw text data as input and produces a trained LLM from scratch, detailing the code and implementation without requiring high-end GPUs. This is for individuals who want to understand the nuts and bolts of LLM creation beyond just theory.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work