RLHFlow/RLHF-Reward-Modeling
Recipes to train reward model for RLHF.
This project provides methods to train a 'reward model' for large language models (LLMs), which learns human preferences by comparing model responses. You input pairs of responses to a prompt, indicating which one is preferred, and the output is a model that can then score future LLM responses. This is for AI researchers and machine learning engineers who are developing or fine-tuning LLMs.
1,520 stars. No commits in the last 6 months.
Use this if you are building an advanced LLM and need to train a robust reward model that accurately reflects human preferences to guide its behavior, or if you want to experiment with state-of-the-art reward modeling techniques.
Not ideal if you are looking for a pre-trained, ready-to-use LLM without needing to customize its preference alignment, or if you lack machine learning development experience.
Stars
1,520
Forks
108
Language
Python
License
Apache-2.0
Category
Last pushed
Apr 24, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/RLHFlow/RLHF-Reward-Modeling"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
agentscope-ai/Trinity-RFT
Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement...
OpenRLHF/OpenRLHF
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO &...
zjunlp/EasyEdit
[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
huggingface/alignment-handbook
Robust recipes to align language models with human and AI preferences
hyunwoongko/nanoRLHF
nanoRLHF: from-scratch journey into how LLMs and RLHF really work.