locuslab/massive-activations

Code accompanying the paper "Massive Activations in Large Language Models"

/ 100

Emerging

This project helps AI researchers and practitioners understand the internal workings of Large Language Models (LLMs) and Vision Transformers (ViTs). It takes trained model weights and input prompts or images, then outputs visualizations and analysis of 'massive activations' within the models. This allows users to pinpoint critical internal features, understand attention mechanisms, and analyze the impact of these activations on model behavior.

197 stars. No commits in the last 6 months.

Use this if you are a machine learning researcher or engineer studying the interpretability, robustness, or internal dynamics of large neural networks like LLMs and ViTs.

Not ideal if you are looking to train new models, improve model performance, or deploy models for practical applications, as this tool focuses on analysis rather than development or production.

LLM interpretability Transformer analysis neural network visualization AI research model debugging

Stale 6m No Package No Dependents

Maintenance 0 / 25

Adoption 10 / 25

Maturity 16 / 25

Community 11 / 25

How are scores calculated?

Stars

197

Forks

Language

Python

License

MIT

Higher-rated alternatives

jncraton/languagemodels

Explore large language models in 512MB of RAM

microsoft/unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

haizelabs/verdict

Inference-time scaling for LLMs-as-a-judge.

albertan017/LLM4Decompile

Reverse Engineering: Decompiling Binary Code with Large Language Models

bytedance/Sa2VA

Official Repo For Pixel-LLM Codebase

Explore Transformer Models

All categories Trending Transformer directory Insights