zhiyuanhubj/LongRecipe
LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
This project offers a method to enhance Large Language Models (LLMs) to understand and generate much longer text sequences, beyond their original design. It takes an existing LLM and your specific text data, then processes it through a multi-stage training 'recipe' to output a new, more capable LLM. This is for AI engineers or researchers who need to adapt LLMs for tasks requiring extensive context, such as analyzing long documents or complex conversations.
No commits in the last 6 months.
Use this if you need to train a Large Language Model to effectively process and generate extremely long text inputs and outputs without losing its core capabilities.
Not ideal if you are an end-user simply looking to use an off-the-shelf LLM, as this project requires significant technical expertise in model training and infrastructure.
Stars
79
Forks
4
Language
Python
License
—
Category
Last pushed
Oct 16, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/zhiyuanhubj/LongRecipe"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ZHZisZZ/dllm
dLLM: Simple Diffusion Language Modeling
pengzhangzhi/Open-dLLM
Open diffusion language model for code generation — releasing pretraining, evaluation,...
EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. ACM...
THUDM/LongWriter
[ICLR 2025] LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs
AIoT-MLSys-Lab/SVD-LLM
[ICLR 2025🔥] SVD-LLM & [NAACL 2025🔥] SVD-LLM V2