keya-desai/Gated-Attention
Implementation of the paper : Not all attention is needed - Gated Attention Network for Sequence Data (GA-Net) [https://arxiv.org/abs/1912.00349]
This project helps machine learning engineers and researchers improve the efficiency and interpretability of their sequence data models. It takes raw sequence data and processes it using a 'gated attention' mechanism, which identifies and focuses only on the most critical parts of the sequence. The output is a more precise and computationally lighter attention mechanism for tasks like natural language processing or time series analysis.
No commits in the last 6 months.
Use this if you are building sequence-based models and need a more efficient way for your model to focus on the most relevant parts of the input, rather than attending to everything.
Not ideal if your primary concern is traditional 'soft attention' where every input token should contribute to the attention output, even if minimally.
Stars
13
Forks
6
Language
Jupyter Notebook
License
—
Category
Last pushed
Aug 20, 2020
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/keya-desai/Gated-Attention"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
xv44586/toolkit4nlp
transformers implement (architecture, task example, serving and more)
luozhouyang/transformers-keras
Transformer-based models implemented in tensorflow 2.x(using keras).
ufal/neuralmonkey
An open-source tool for sequence learning in NLP built on TensorFlow.
graykode/xlnet-Pytorch
Simple XLNet implementation with Pytorch Wrapper
uzaymacar/attention-mechanisms
Implementations for a family of attention mechanisms, suitable for all kinds of natural language...