Model Inference Serving MLOps Tools

There are 35 model inference serving tools tracked. 1 score above 70 (verified tier). The highest-rated is feast-dev/feast at 84/100 with 6,793 stars. 2 of the top 10 are actively maintained.

Get all 35 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=mlops&subcategory=model-inference-serving&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	feast-dev/feast The Open Source Feature Store for AI/ML	84	Verified	6,793	Python
2	clearml/clearml-serving ClearML - Model-Serving Orchestration and Repository Solution	67	Established	161	Python
3	lakehq/sail LakeSail's computation framework with a mission to unify batch processing,...	64	Established	1,183	Rust
4	PaddlePaddle/Serving A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）	61	Established	925	C++
5	SeldonIO/MLServer An inference server for your machine learning models, including support for...	61	Established	875	Python
6	sustainable-computing-io/kepler-model-server Model Server for Kepler	53	Established	29	Python
7	pytorch/serve Serve, optimize and scale PyTorch models in production	51	Established	4,361	Java
8	raptor-ml/raptor Transform your pythonic research to an artifact that engineers can deploy easily.	48	Emerging	161	Go
9	sustainable-computing-io/kepler-model-db Repository containing up-to-date models to be used by the kepler-model-server	45	Emerging	3	—
10	tugraz-isds/systemds An open source ML system for the end-to-end data science lifecycle	42	Emerging	37	Java
11	george0st/qgate-model ML/AI meta-model, used in MLRun/Iguazio/Nuclio, see qgate-sln-	40	Emerging	414	Python
12	fuseml/fuseml-core FuseML APIs and core service. This repo include the FuseML client useful to...	35	Emerging	5	Go
13	Kenza-AI/kenza Open-Source Machine Learning Platform	34	Emerging	7	Go
14	bioinformatist/cml A Framework for Production-Ready Continuous Machine Learning	33	Emerging	6	Rust
15	aporia-ai/inferencedb 🚀 Stream inferences of real-time ML models in production to any data lake...	31	Emerging	81	Python
16	eora-ai/inferoxy Service for quick deploying and using dockerized Computer Vision models	31	Emerging	95	Python
17	gasparian/ml-serving-template Serving large ml models independently and asynchronously via message queue...	27	Experimental	16	Python
18	datamass-io/ml-kraken Machine-Learning orchestration framework. Cloud-based models management environment.	27	Experimental	8	TypeScript
19	mmziyad/flink-ms Serving layer for large machine learning models on Apache Flink	25	Experimental	4	Java
20	puneethkotha/Falcon Production ML inference platform. Multi-worker · Nginx load balancing ·...	22	Experimental	—	Python
21	JohnJTK/crucible_train 🚀 Accelerate ML training on the BEAM with CrucibleTrain's unified...	21	Experimental	—	Elixir
22	HighviewOne/ml-model-registry ML Model Registry & Deployment Dashboard - AI Dev Tools Zoomcamp 2025	21	Experimental	—	TypeScript
23	gdroguski/MLServe Docker-based Machine Learning models serving	18	Experimental	2	Python
24	ameron-ai/model-serving-sidecar-service-example A simple Python example of a Model Service that can be fronted by the Model Sidecar	17	Experimental	1	Python
25	ameron-ai/model-serving-sidecar A lightweight adapter that handles all the cross-cutting concerns for model serving	17	Experimental	1	Java
26	North-Shore-AI/crucible_model_registry ML model registry for the Crucible ecosystem. Artifact storage, model...	17	Experimental	—	Elixir
27	North-Shore-AI/crucible_train ML training orchestration for the Crucible ecosystem. Distributed training,...	17	Experimental	—	Elixir
28	North-Shore-AI/crucible_feedback ML feedback loop management for the Crucible ecosystem. Quality monitoring,...	17	Experimental	—	Elixir
29	North-Shore-AI/crucible_deployment ML model deployment for the Crucible ecosystem. vLLM and Ollama integration,...	17	Experimental	—	Elixir
30	galafis/distributed-model-inference-engine Distributed model inference engine with REST/gRPC serving, circuit breaker,...	14	Experimental	—	Python
31	man4ish/omnibioai-model-registry Production-grade model registry for the OmniBioAI ecosystem, providing...	14	Experimental	—	Python
32	arthurhzna/Golang_AI_Pipeline A scalable AI task queue and processing pipeline in Go, integrating Redis,...	13	Experimental	—	Go
33	narphu/modelcache-operator Model caching in Kubernetes	13	Experimental	—	Go
34	marekgalovic/photon ML model serving service.	12	Experimental	6	Go
35	markdouthwaite/apogee-server Probabilistic Graphical Models on Kubernetes	11	Experimental	—	Go

Comparisons in this category

kepler-model-server and kepler-model-db (53 vs 45)