dannylee1020/openpo
Building synthetic data for preference tuning
This project helps AI developers and researchers create high-quality synthetic datasets to fine-tune their large language models (LLMs). It takes in prompts and generates diverse responses from over 200 different LLMs. The output is a dataset of these responses, often paired with evaluations to indicate preference, which is crucial for training more helpful and accurate AI models.
No commits in the last 6 months. Available on PyPI.
Use this if you are a machine learning engineer or AI researcher who needs to generate and evaluate a large volume of synthetic text data from various LLMs for model training or research.
Not ideal if you're looking for a user-facing application to directly interact with or fine-tune LLMs without writing code, or if you only need to use a single LLM for basic text generation.
Stars
27
Forks
—
Language
Python
License
Apache-2.0
Category
Last pushed
Dec 26, 2024
Commits (30d)
0
Dependencies
9
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/dannylee1020/openpo"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
codelion/pts
Pivotal Token Search
DtYXs/Pre-DPO
Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model
RLHFlow/Directional-Preference-Alignment
Directional Preference Alignment
pspdada/Uni-DPO
[ICLR 2026] Official repository of "Uni-DPO: A Unified Paradigm for Dynamic Preference...
liushunyu/awesome-direct-preference-optimization
A Survey of Direct Preference Optimization (DPO)