VITA-Group/Ms-PoE
"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.
This project helps improve how Large Language Models (LLMs) find and use important details when they're given a very long piece of text. It takes an existing LLM and enhances its ability to pinpoint relevant information, especially if that information is buried in the middle of a long document or conversation. Researchers and AI engineers working with LLMs for complex tasks like summarization or long-form Q&A would find this useful.
No commits in the last 6 months.
Use this if your Large Language Models struggle to accurately extract or act upon key information located in the middle of extremely long text inputs.
Not ideal if you are not working with Large Language Models or if your primary concern is not long-context understanding.
Stars
31
Forks
4
Language
Python
License
MIT
Category
Last pushed
May 07, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/VITA-Group/Ms-PoE"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ZHZisZZ/dllm
dLLM: Simple Diffusion Language Modeling
pengzhangzhi/Open-dLLM
Open diffusion language model for code generation — releasing pretraining, evaluation,...
EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. ACM...
THUDM/LongWriter
[ICLR 2025] LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs
AIoT-MLSys-Lab/SVD-LLM
[ICLR 2025🔥] SVD-LLM & [NAACL 2025🔥] SVD-LLM V2