li-plus/flash-preference
Accelerate LLM preference tuning via prefix sharing with a single line of code
This tool helps machine learning engineers accelerate the process of fine-tuning large language models (LLMs) based on human preferences. By efficiently sharing common parts of input sequences, it speeds up both forward and backward passes during training. This is ideal for ML engineers who are working with techniques like Direct Preference Optimization (DPO) or Reward Modeling.
No commits in the last 6 months.
Use this if you are an ML engineer training LLMs with preference data and want to significantly reduce computation time and memory usage without compromising model accuracy.
Not ideal if you are not directly involved in training or fine-tuning large language models, or if your tasks do not involve preference-based learning.
Stars
51
Forks
—
Language
Python
License
MIT
Category
Last pushed
Jul 04, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/li-plus/flash-preference"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
stair-lab/mlhp
Machine Learning from Human Preferences
princeton-nlp/SimPO
[NeurIPS 2024] SimPO: Simple Preference Optimization with a Reference-Free Reward
uclaml/SPPO
The official implementation of Self-Play Preference Optimization (SPPO)
general-preference/general-preference-model
[ICML 2025] Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment...
sail-sg/dice
Official implementation of Bootstrapping Language Models via DPO Implicit Rewards