IAAR-Shanghai/Awesome-Attention-Heads
An awesome repository & A comprehensive survey on interpretability of LLM attention heads.
This resource provides a curated collection of the latest research and a comprehensive survey on how attention heads within Large Language Models (LLMs) function. It helps AI researchers and practitioners understand the 'black box' of LLMs by showing what goes on inside these models, specifically how different parts contribute to reasoning and knowledge processing. The output is a categorized overview of research papers and methodologies, revealing the inner workings of LLMs.
400 stars. No commits in the last 6 months.
Use this if you are an AI researcher or practitioner looking to deepen your understanding of how Large Language Models (LLMs) make decisions and process information, specifically focusing on the interpretability of their attention mechanisms.
Not ideal if you are looking for a practical tool to build or deploy LLMs, or if you are not interested in the academic research and mechanistic interpretability of these models.
Stars
400
Forks
12
Language
TeX
License
—
Category
Last pushed
Mar 02, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/IAAR-Shanghai/Awesome-Attention-Heads"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
liangyuwang/Tiny-DeepSpeed
Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library
catherinesyeh/attention-viz
Visualizing query-key interactions in language + vision transformers (VIS 2023)
microsoft/Text2Grad
🚀 Text2Grad: Converting natural language feedback into gradient signals for precise model...
huangjia2019/llm-gpt
From classic NLP to modern LLMs: building language models step by step. 异æ¥å›¾ä¹¦ï¼šã€Š GPT图解 å¤§æ¨¡åž‹æ˜¯æ€Žæ ·æž„å»ºçš„ã€‹-...