Attention Mechanism Implementations Transformer Models

There are 24 attention mechanism implementations models tracked. 2 score above 50 (established tier). The highest-rated is microsoft/LoRA at 57/100 with 13,320 stars.

Get all 24 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=transformers&subcategory=attention-mechanism-implementations&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Model	Score	Tier	Stars	Language
1	microsoft/LoRA Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large...	57	Established	13,320	Python
2	jadore801120/attention-is-all-you-need-pytorch A PyTorch implementation of the Transformer model in "Attention is All You Need".	51	Established	9,651	Python
3	bhavnicksm/vanilla-transformer-jax JAX/Flax implimentation of 'Attention Is All You Need' by Vaswani et al....	45	Emerging	15	Python
4	kyegomez/SparseAttention Pytorch Implementation of the sparse attention from the paper: "Generating...	43	Emerging	94	Python
5	AbdelStark/attnres Rust implementation of Attention Residuals from MoonshotAI/Kimi	42	Emerging	47	Rust
6	sunnynguyen-ai/llm-attention-visualizer Interactive tool for analyzing attention patterns in transformer models with...	40	Emerging	14	Python
7	kyegomez/AoA-torch Implementation of Attention on Attention in Zeta	39	Emerging	5	Python
8	takara-ai/SwarmFormer A pytorch implementation of SwarmFormer for text classification.	39	Emerging	16	Python
9	takara-ai/go-attention A full attention mechanism and transformer in pure go.	38	Emerging	451	Go
10	MurrellGroup/InvariantPointAttention.jl Julia implementation of AlphaFold 2's Invariant Point Attention	37	Emerging	6	Julia
11	SingleZombie/LLSA Official implementation of Log-linear Sparse Attention (LLSA).	36	Emerging	62	Python
12	tranquoctrinh/transformer This is a PyTorch implementation of the Transformer model in the paper...	35	Emerging	37	Python
13	HKUNLP/efficient-attention [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control...	32	Emerging	87	Python
14	mrcabbage972/simple-toolformer A Python implementation of Toolformer using Huggingface Transformers	31	Emerging	14	Python
15	Awni00/abstract_transformer This is the project repo associated with the paper "Disentangling and...	30	Emerging	6	Jupyter Notebook
16	tobifinn/ensemble_transformer Official PyTorch implementation of "Self-Attentive Ensemble Transformer:...	27	Experimental	14	Jupyter Notebook
17	ghosthamlet/transformers-rs Rust Implemention of paper: Attention Is All You...	21	Experimental	7	Rust
18	Nemesis-12/multihead-latent-attention Implementation of Multi-head Latent Attention (MLA) from DeepSeek-V2	17	Experimental	—	Python
19	cnygaard/FractalHTransformer Fractal Hierarchical Transformer: multi-resolution causal attention patterns...	14	Experimental	—	Python
20	wiedersehne/Paramixer Parameterizing Mixing Links in Sparse Factors Works Better than Dot-Product...	14	Experimental	20	Python
21	adi-mish/miniformer Miniformer is a lightweight PyTorch transformer library for researchers,...	13	Experimental	—	Python
22	romizone/simulasiLLM 🧠 Interactive LLM Attention Simulation — Visualize how GPT-2 transformers...	13	Experimental	—	JavaScript
23	kesimeg/LORA-turkish-clip Finetuning CLIP using LORA for Turkish language	12	Experimental	6	Jupyter Notebook
24	MaxLSB/linformer linformer implementation and comparison with vanilla transformers	11	Experimental	4	Jupyter Notebook

Comparisons in this category

attention-is-all-you-need-pytorch and transformer (51 vs 35)