shivendrra/SmallLanguageModel
a LLM cookbook, for building your own from scratch, all the way from gathering data to training a model
This helps machine learning engineers or researchers build their own large language models (LLMs) from the ground up. You provide raw text data, and it guides you through collecting, processing, and then training custom BERT, GPT, or Seq-2-Seq models. The output is a functional language model tailored to your specific data and needs.
168 stars. No commits in the last 6 months.
Use this if you are a machine learning engineer or researcher who wants to understand and build a custom large language model from scratch, rather than fine-tuning a pre-existing one.
Not ideal if you're looking for a simple tool to fine-tune an existing LLM or need a ready-to-use solution without delving into model architecture and training.
Stars
168
Forks
22
Language
Jupyter Notebook
License
MIT
Category
Last pushed
Jun 25, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/shivendrra/SmallLanguageModel"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AI-Hypercomputer/maxtext
A simple, performant and scalable Jax LLM!
rasbt/reasoning-from-scratch
Implement a reasoning LLM in PyTorch from scratch, step by step
mindspore-lab/mindnlp
MindSpore + 🤗Huggingface: Run any Transformers/Diffusers model on MindSpore with seamless...
mosaicml/llm-foundry
LLM training code for Databricks foundation models
rickiepark/llm-from-scratch
<밑바닥부터 만들면서 공부하는 LLM>(길벗, 2025)의 코드 저장소