EleutherAI/knowledge-neurons
A library for finding knowledge neurons in pretrained transformer models.
This tool helps AI researchers understand how pre-trained language models like BERT and GPT-2 store specific factual knowledge. You input a pre-trained model and a set of sentences expressing a fact (e.g., "Paris is the capital of France"), and it identifies the specific internal components (neurons) responsible for that knowledge. It's designed for researchers studying the interpretability and internal workings of large language models.
159 stars. No commits in the last 6 months. Available on PyPI.
Use this if you need to pinpoint and manipulate the exact "neurons" within a transformer model that encode particular facts, to understand or alter its knowledge.
Not ideal if you're looking for a tool to improve the performance of a language model on a task, rather than investigating its internal knowledge representation.
Stars
159
Forks
19
Language
Python
License
MIT
Last pushed
Feb 13, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/EleutherAI/knowledge-neurons"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related models
jessevig/bertviz
BertViz: Visualize Attention in Transformer Models
inseq-team/inseq
Interpretability for sequence generation models 🐛 🔍
hila-chefer/Transformer-MM-Explainability
[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for...
cdpierse/transformers-interpret
Model explainability that works seamlessly with 🤗 transformers. Explain your transformers model...
taufeeque9/codebook-features
Sparse and discrete interpretability tool for neural networks