supersimple33/Scaling-Laws
A method for calculating scaling laws for LLMs from publicly available models
This project helps machine learning researchers and practitioners understand how to train Large Language Models (LLMs) more efficiently. By inputting details like model size and training data volume from publicly available LLMs, it calculates 'scaling laws' that predict how changes in these factors affect model performance (measured by perplexity). This allows users to determine optimal training strategies, reducing computational waste and improving outcomes.
No commits in the last 6 months.
Use this if you are developing or training large language models and want to understand the mathematical relationships between model parameters, training tokens, and performance to optimize your resource usage.
Not ideal if you are looking for a tool to directly train or fine-tune an LLM, or if you need to evaluate models based on subjective human feedback rather than perplexity.
Stars
9
Forks
—
Language
Python
License
—
Category
Last pushed
Apr 22, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/supersimple33/Scaling-Laws"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jncraton/languagemodels
Explore large language models in 512MB of RAM
microsoft/unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
haizelabs/verdict
Inference-time scaling for LLMs-as-a-judge.
albertan017/LLM4Decompile
Reverse Engineering: Decompiling Binary Code with Large Language Models
bytedance/Sa2VA
Official Repo For Pixel-LLM Codebase