jshuadvd/LongRoPE
Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper
This project extends the context window of large language models (LLMs) like LLaMA2 and Mistral, allowing them to process and understand much longer texts. It takes a pre-trained LLM and, through a progressive extension strategy, enables it to handle inputs up to 2 million tokens while maintaining accuracy. This is designed for AI practitioners and researchers who need LLMs to analyze or generate content from extremely long documents or conversations.
151 stars. No commits in the last 6 months.
Use this if you need your LLM to effectively process and reason over very long documents, extensive dialogues, or large sets of information, far beyond typical context limits.
Not ideal if your application only deals with short texts or if you are not working with large language models.
Stars
151
Forks
14
Language
Python
License
—
Category
Last pushed
Jul 20, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/jshuadvd/LongRoPE"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
openvinotoolkit/nncf
Neural Network Compression Framework for enhanced OpenVINO™ inference
huggingface/optimum
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers...
NVIDIA/Megatron-LM
Ongoing research training transformer models at scale
huggingface/optimum-intel
🤗 Optimum Intel: Accelerate inference with Intel optimization tools
eole-nlp/eole
Open language modeling toolkit based on PyTorch