microsoft/private-benchmarking
A platform that enables users to perform private benchmarking of machine learning models. The platform facilitates the evaluation of models based on different trust levels between the model owners and the dataset owners.
This platform helps researchers and organizations evaluate machine learning models against datasets without either party fully revealing their proprietary model or sensitive data. It takes in a model from a 'Model Owner' and a dataset from a 'Dataset Owner', then produces performance benchmarks. This is for AI/ML researchers or data scientists who need to compare model performance under strict privacy constraints.
No commits in the last 6 months.
Use this if you need to benchmark the performance of an AI/ML model using a dataset that cannot be fully shared with the model owner, or vice versa, due to privacy or proprietary concerns.
Not ideal if you are looking for a production-ready system to regularly benchmark models, as this is an academic prototype and not yet hardened for real-world deployment.
Stars
12
Forks
2
Language
Python
License
MIT
Category
Last pushed
Sep 16, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/mlops/microsoft/private-benchmarking"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.