zhanshijinwat/Steel-LLM
Train a 1B LLM with 1T tokens from scratch by personal
Steel-LLM is a project for those who want to build their own custom Chinese large language models (LLMs) from scratch. It provides a complete guide and all necessary code for collecting and processing Chinese text data, then training an LLM. The output is a functional Chinese LLM tailored to specific data, ready for fine-tuning.
791 stars. No commits in the last 6 months.
Use this if you are a machine learning researcher or engineer with access to 8 or more GPUs (like H800 or A100) and want to pre-train a Chinese LLM from the ground up, rather than simply using an existing model.
Not ideal if you are looking for a ready-to-use LLM for immediate application without extensive training, or if you do not have significant GPU resources and expertise in LLM training.
Stars
791
Forks
78
Language
Jupyter Notebook
License
—
Category
Last pushed
Apr 27, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/zhanshijinwat/Steel-LLM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
facebookresearch/LayerSkip
Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
FareedKhan-dev/train-llm-from-scratch
A straightforward method for training your LLM, from downloading data to generating text.
kmeng01/rome
Locating and editing factual associations in GPT (NeurIPS 2022)
datawhalechina/llms-from-scratch-cn
仅需Python基础，从0构建大语言模型；从0逐步构建GLM4\Llama3\RWKV6， 深入理解大模型原理