Rishit-dagli/Nystromformer
An implementation of the Nyströmformer, using Nystrom method to approximate standard self attention
This project helps machine learning engineers build more efficient deep learning models. It takes long sequences of data, such as extensive text documents or lengthy time series, and processes them more quickly than traditional methods. The output is a more scalable and faster-to-train deep learning model, particularly useful for those working with large datasets.
No commits in the last 6 months. Available on PyPI.
Use this if you are a machine learning engineer working with Transformer models and frequently encounter performance issues or high computational costs when processing very long sequences of data.
Not ideal if you are working with short data sequences or do not have prior experience with deep learning model development, as it requires familiarity with TensorFlow and attention mechanisms.
Stars
58
Forks
3
Language
Python
License
Apache-2.0
Category
Last pushed
Aug 21, 2022
Commits (30d)
0
Dependencies
2
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/Rishit-dagli/Nystromformer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
philipperemy/keras-attention
Keras Attention Layer (Luong and Bahdanau scores).
tatp22/linformer-pytorch
My take on a practical implementation of Linformer for Pytorch.
ematvey/hierarchical-attention-networks
Document classification with Hierarchical Attention Networks in TensorFlow. WARNING: project is...
datalogue/keras-attention
Visualizing RNNs using the attention mechanism
thushv89/attention_keras
Keras Layer implementation of Attention for Sequential models