dingo-actual/infini-transformer
PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)
This project offers a sophisticated deep learning model for working with extremely long text inputs in natural language processing. It takes large textual datasets, like books or very long documents, and processes them to enable tasks such as understanding content, answering questions, or generating new text. It is designed for researchers and engineers building advanced NLP systems that need to handle extensive contexts without memory limitations.
298 stars. No commits in the last 6 months.
Use this if you are developing AI models that need to process and understand very long text sequences, such as entire books, legal documents, or complex scientific papers, where traditional models struggle with memory constraints.
Not ideal if your primary need is processing short, fixed-length text snippets or if you are looking for an out-of-the-box solution without deep learning development experience.
Stars
298
Forks
23
Language
Python
License
MIT
Category
Last pushed
May 04, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/dingo-actual/infini-transformer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
lucidrains/x-transformers
A concise but complete full-attention transformer with a set of promising experimental features...
kanishkamisra/minicons
Utility for behavioral and representational analyses of Language Models
lucidrains/simple-hierarchical-transformer
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
lucidrains/dreamer4
Implementation of Danijar's latest iteration for his Dreamer line of work
Nicolepcx/Transformers-in-Action
This is the corresponding code for the book Transformers in Action