kmkrofficial/LiteGPT
LiteGPT: A 124M Small Language Model (SLM) pre-trained on FineWeb and fine-tuned on Alpaca.
This project helps machine learning engineers and researchers train a small language model from scratch or fine-tune an existing one. It takes large text datasets, like educational web data or instruction-following examples, and outputs a custom language model that can generate text or follow instructions. This is for individuals building or experimenting with their own AI language models.
Use this if you need to create your own compact, custom language model for text generation or instruction following, without needing a massive, resource-intensive model.
Not ideal if you simply want to use an off-the-shelf language model for common tasks without any custom training or model development.
Stars
34
Forks
5
Language
Python
License
—
Category
Last pushed
Dec 16, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/kmkrofficial/LiteGPT"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AI-Hypercomputer/maxtext
A simple, performant and scalable Jax LLM!
rasbt/reasoning-from-scratch
Implement a reasoning LLM in PyTorch from scratch, step by step
mindspore-lab/mindnlp
MindSpore + 🤗Huggingface: Run any Transformers/Diffusers model on MindSpore with seamless...
mosaicml/llm-foundry
LLM training code for Databricks foundation models
rickiepark/llm-from-scratch
<밑바닥부터 만들면서 공부하는 LLM>(길벗, 2025)의 코드 저장소