dashstander/block-recurrent-transformer
Pytorch implementation of "Block Recurrent Transformers" (Hutchins & Schlag et al., 2022)
This is a tool for machine learning researchers and practitioners who are experimenting with advanced neural network architectures. It helps you build and train models that can process sequences of information more efficiently than traditional methods, particularly when dealing with very long inputs. You provide your training data, and it helps you produce a trained recurrent transformer model.
No commits in the last 6 months.
Use this if you are developing or researching new deep learning models for sequence processing and want to explore the Block Recurrent Transformer architecture.
Not ideal if you need an out-of-the-box solution for a specific application without delving into model architecture or training details.
Stars
85
Forks
13
Language
Python
License
MIT
Category
Last pushed
May 14, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/dashstander/block-recurrent-transformer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in...
kyegomez/LongNet
Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
pbloem/former
Simple transformer implementation from scratch in pytorch. (archival, latest version on codeberg)
NVIDIA/FasterTransformer
Transformer related optimization, including BERT, GPT
kyegomez/SimplifiedTransformers
SimplifiedTransformer simplifies transformer block without affecting training. Skip connections,...