rhubarbwu/linguistic-collapse
Codebase for Linguistic Collapse: Neural Collapse in (Large) Language Models [NeurIPS 2024] [arXiv:2405.17767]
This project provides the tools and scripts to train and evaluate large language models (LLMs) like GPT-Neo, and then deeply analyze their internal representations for a phenomenon called 'Neural Collapse.' You can input your LLM configurations and training data, receive trained models, and then get detailed analysis results, including various metrics of linguistic collapse and visualization notebooks. It's designed for machine learning researchers and academics studying the fundamental behaviors of LLMs.
No commits in the last 6 months.
Use this if you are a machine learning researcher who wants to rigorously study the internal mechanics of how large language models learn and represent information.
Not ideal if you are looking for an off-the-shelf solution to train a production-ready LLM or to apply LLMs for practical tasks without deep architectural analysis.
Stars
18
Forks
3
Language
Python
License
—
Category
Last pushed
Apr 14, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/rhubarbwu/linguistic-collapse"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jncraton/languagemodels
Explore large language models in 512MB of RAM
microsoft/unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
haizelabs/verdict
Inference-time scaling for LLMs-as-a-judge.
albertan017/LLM4Decompile
Reverse Engineering: Decompiling Binary Code with Large Language Models
bytedance/Sa2VA
Official Repo For Pixel-LLM Codebase