ONNX Model Deployment ML Frameworks

Tools and frameworks for converting, optimizing, and deploying ONNX models across different runtime environments and hardware platforms. Does NOT include general model training, framework-specific inference (e.g., pure TensorFlow or PyTorch inference), or non-ONNX deployment solutions.

There are 120 onnx model deployment frameworks tracked. 5 score above 70 (verified tier). The highest-rated is microsoft/onnxruntime at 93/100 with 19,534 stars and 474 monthly downloads. 8 of the top 10 are actively maintained.

Get all 120 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=onnx-model-deployment&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	microsoft/onnxruntime ONNX Runtime: cross-platform, high performance ML inferencing and training...	93	Verified	19,534	C++
2	onnx/onnx Open standard for machine learning interoperability	85	Verified	20,477	Python
3	PINTO0309/onnx2tf Self-Created Tools to convert ONNX files (NCHW) to TensorFlow/TFLite/Keras...	78	Verified	931	Python
4	NVIDIA/TensorRT NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on...	74	Verified	12,784	C++
5	onnx/onnxmltools ONNXMLTools enables conversion of models to ONNX	73	Verified	1,143	Python
6	microsoft/onnxconverter-common Common utilities for ONNX converters	69	Established	295	Python
7	jolibrain/deepdetect Deep Learning API and Server in C++14 support for PyTorch,TensorRT, Dlib,...	68	Established	2,549	C++
8	NVIDIA/DALI A GPU-accelerated library containing highly optimized building blocks and an...	67	Established	5,643	C++
9	pytorch/TensorRT PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT	67	Established	2,955	Python
10	onnxsim/onnxsim Simplify your onnx model	66	Established	4,307	C++
11	onnx/onnx-tensorrt ONNX-TensorRT: TensorRT backend for ONNX	63	Established	3,194	C++
12	onnx/tensorflow-onnx Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX	63	Established	2,519	Jupyter Notebook
13	nntrainer/nntrainer NNtrainer is Software Framework for Training and Inferencing Neural Network...	60	Established	204	C++
14	kraiskil/onnx2c Open Neural Network Exchange to C compiler.	58	Established	368	C
15	NVIDIA/MinkowskiEngine Minkowski Engine is an auto-diff neural network library for high-dimensional...	58	Established	2,883	Python
16	sdpython/onnx-extended New operators for the ReferenceEvaluator, new kernels for onnxruntime, CPU, CUDA	57	Established	35	Python
17	aphrodite-engine/aphrodite-engine Large-scale LLM inference engine	57	Established	1,673	C++
18	triton-inference-server/dali_backend The Triton backend that allows running GPU-accelerated data pre-processing...	57	Established	141	C++
19	Quantco/spox Pythonic framework for building ONNX graphs	55	Established	94	Python
20	VeriSilicon/tflite-vx-delegate Tensorflow Lite external delegate based on TIM-VX	54	Established	48	C++
21	dotnet/infer Infer.NET is a framework for running Bayesian inference in graphical models	54	Established	1,607	C#
22	justinchuby/onnx-shape-inference Symbolic shape inference for ONNX	51	Established	6	Python
23	Tencent/TNN TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep...	51	Established	4,626	C++
24	hailo-ai/hailort An open source light-weight and high performance inference framework for...	51	Established	172	C++
25	onnx/onnx-tensorflow Tensorflow Backend for ONNX	51	Established	1,325	Python
26	xboot/libonnx A lightweight, portable pure C99 onnx inference engine for embedded devices...	51	Established	647	C
27	OAID/Tengine Tengine is a lite, high performance, modular inference engine for embedded device	51	Established	4,510	C++
28	amir-abdi/keras_to_tensorflow General code to convert a trained keras model into an inference tensorflow model	51	Established	1,652	Python
29	facebookarchive/caffe2 Caffe2 is a lightweight, modular, and scalable deep learning framework.	51	Established	8,398	Shell
30	OAID/Caffe-HRT Heterogeneous Run Time version of Caffe. Added heterogeneous capabilities to...	50	Established	269	C++
31	wuba/dl_inference 通用深度学习推理工具，可在生产环境中快速上线由TensorFlow、PyTorch、Caffe框架训练出的深度学习模型。	50	Established	415	Java
32	solrex/caffe-mobile Optimized (for size and speed) Caffe lib for iOS and Android with...	50	Established	313	C++
33	gmalivenko/pytorch2keras PyTorch to Keras model convertor	49	Emerging	862	Python
34	iwatake2222/play_with_tflite Sample projects for TensorFlow Lite in C++ with delegates such as GPU,...	49	Emerging	381	C++
35	gizatechxyz/orion ONNX Runtime in Cairo 1.0 for verifiable ML inference using STARK	49	Emerging	176	Cairo
36	NervanaSystems/ngraph nGraph has moved to OpenVINO	49	Emerging	1,343	C++
37	PINTO0309/simple-onnx-processing-tools A set of simple tools for splitting, merging, OP deletion, size compression,...	49	Emerging	304	Python
38	iwatake2222/InferenceHelper C++ Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite,...	49	Emerging	297	C++
39	NVIDIA/nvImageCodec A nvImageCodec library of GPU- and CPU- accelerated codecs featuring a...	49	Emerging	143	Jupyter Notebook
40	NVIDIA/tensorrt-laboratory Explore the Capabilities of the TensorRT Platform	48	Emerging	262	C++
41	microsoft/nn-Meter A DNN inference latency prediction toolkit for accurately modeling and...	48	Emerging	362	Python
42	zjhellofss/KuiperInfer 校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2...	48	Emerging	3,353	C++
43	kibae/onnxruntime-server ONNX Runtime Server: The ONNX Runtime Server is a server that provides TCP...	48	Emerging	181	C++
44	AlexanderLutsenko/nobuco Pytorch to Keras/Tensorflow/TFLite conversion made intuitive	47	Emerging	343	Python
45	olilarkin/ort-builder ONNX Runtime static library builder	46	Emerging	74	C++
46	cyrusbehr/tensorrt-cpp-api TensorRT C++ API Tutorial	46	Emerging	793	C++
47	iwatake2222/play_with_tensorrt Sample projects for TensorRT in C++	46	Emerging	197	C++
48	OAID/Tengine-Convert-Tools Tengine Convert Tool supports converting multi framworks' models into tmfile...	46	Emerging	92	C++
49	xmba15/onnx_runtime_cpp small c++ library to quickly deploy models using onnxruntime	45	Emerging	385	C++
50	alrevuelta/cONNXr Pure C ONNX runtime with zero dependancies for embedded devices	45	Emerging	216	C
51	NobuoTsukamoto/tensorrt-examples TensorRT Examples (TensorRT, Jetson Nano, Python, C++)	45	Emerging	100	Jupyter Notebook
52	luxonis/modelconverter Model converter for Luxonis' cameras. Convert your model from ONNX, TF, ......	45	Emerging	13	Python
53	AsuharietYgvar/AppleNeuralHash2ONNX Convert Apple NeuralHash model for CSAM Detection to ONNX.	45	Emerging	1,536	Python
54	OAID/MXNet-HRT Heterogeneous Run Time version of MXNet. Added heterogeneous capabilities to...	45	Emerging	72	C++
55	intel/npu-nn-cost-model Library for modelling performance costs of different Neural Network...	44	Emerging	34	C++
56	Adlik/Adlik Adlik: Toolkit for Accelerating Deep Learning Inference	44	Emerging	807	C++
57	Phoenix8215/A-White-Paper-on-Neural-Network-Deployment 模型部署白皮书(CUDA\|ONNX\|TensorRT\|C++)🚀🚀🚀	44	Emerging	244	—
58	emmtrix/emx-onnx-cgen Deterministic ONNX-to-C compiler for embedded and safety-critical systems,...	44	Emerging	10	Python
59	Tencent/Forward A library for high performance deep learning inference on NVIDIA GPUs.	44	Emerging	555	C++
60	merrymercy/tvm-mali Optimizing Mobile Deep Learning on ARM GPU with TVM	43	Emerging	183	C
61	hshatti/TONNXRuntime TOnnxRuntime is a Microsoft ONNXRuntime AI and Machine Learning Library for...	42	Emerging	65	Pascal
62	MegEngine/MegFlow Efficient ML solution for long-tailed demands.	42	Emerging	407	Rust
63	kibae/pg_onnx pg_onnx: ONNX Runtime integrated with PostgreSQL. Perform ML inference with...	42	Emerging	58	C++
64	RobertBiehl/caffe2-ios ☕️ Caffe2Kit. A simple one step integration of Caffe2 for iOS.	42	Emerging	71	C++
65	MaitreChen/openvino-lenet-sample 本仓库包含了完整的深度学习应用开发流程，以经典的手写字符识别为例，基于LeNet网络构建。推理部分使用torch、onnxruntime以及openvino框架💖	41	Emerging	17	Python
66	sdpython/onnx-array-api Array APIs to write ONNX Graphs	41	Emerging	11	Python
67	qbxlvnf11/convert-pytorch-onnx-tensorrt Converting weights of Pytorch models to ONNX & TensorRT engines	41	Emerging	50	Python
68	yuanmu97/InFi InFi is a library for building input filters for resource-efficient inference.	40	Emerging	41	Python
69	iree-org/onnxruntime-ep-iree IREE based OnnxRuntime Execution Provider	40	Emerging	5	C++
70	ysh329/deep-learning-model-convertor The convertor/conversion of deep learning models for different deep learning...	40	Emerging	3,247	—
71	RidgeRun/r2inference RidgeRun Inference Framework	40	Emerging	27	C++
72	zerollzeng/tiny-tensorrt Deploy your model with TensorRT quickly.	39	Emerging	764	C++
73	fateshelled/OnnxGraphQt ONNX model visualizer	38	Emerging	88	Python
74	TeiaCare/TeiaCareInferenceClient TeiaCareInferenceClient is a C++ inference client library that implements...	37	Emerging	3	C++
75	RunEdgeAI/coreflow Graph-based C++ runtime for building and executing AI, ML, and computer...	37	Emerging	12	C++
76	haobosang/TinyTensor TinyTensor is a tool for running already trained NN (Neural Network) models...	37	Emerging	141	C++
77	mshr-h/onnx2fx Turn ONNX models into inspectable, transformable, and trainable PyTorch FX...	37	Emerging	10	Python
78	yuzawa-san/onnxruntime-java A type-safe, lightweight, modern, and performant binding Java binding of...	37	Emerging	29	Java
79	vibeswithkk/ZENITH An open-source bridge for faster ML inference. Supports PyTorch, JAX, and...	34	Emerging	4	Python
80	athrva98/polyinfer Unified deployment pipeline	33	Emerging	9	Python
81	luchangli03/onnxsim_large_model simplify >2GB large onnx model	33	Emerging	70	Python
82	becauseofAI/caffe-plus-plus Caffe++: assemble new features to enhance Caffe☕️	33	Emerging	11	C++
83	smarter-project/armnn_tflite_backend TensorFlow Lite backend with ArmNN delegate support for Nvidia Triton	33	Emerging	8	C++
84	xiaochus/DeepModelDeploy Deploy deep learning model on difference hardware and framework....	32	Emerging	13	C++
85	victorbadenas/SimpleOnnxInference A simple c++ onnx application for inferencing images	31	Emerging	3	C++
86	domenicostefani/deep-classf-runtime-wrappers Classification wrappers for 4 Deep Learning Embedded Runtimes for Music...	30	Emerging	6	C++
87	aws-samples/sagemaker-cv-preprocessing-training-performance SageMaker training implementation for computer vision to offload JPEG...	30	Emerging	21	Python
88	zpye/SimpleInfer A simple neural network inference framework	30	Emerging	25	C++
89	RyanCCC/Deployment 深度学习应用部署	28	Experimental	8	C++
90	loong64/onnxruntime ONNX Runtime: cross-platform, high performance ML inferencing and training...	26	Experimental	1	Shell
91	Orange20000922/Filerestore_CLI 一个 NTFS 文件恢复工具，支持 MFT 扫描、签名搜索、智能覆盖检测和高性能文件恢复	26	Experimental	3	C++
92	zhehaoxu/deep-deploy deploy deep learning model with MNN	24	Experimental	3	C++
93	veera-adithya-d/hardware-aware-algorithm Inference module of Imagenet	23	Experimental	3	C++
94	odeliyach/tiny-ml-runtime Generic neural network inference engine in pure C. 258x faster than PyTorch...	22	Experimental	—	C
95	brandonviaje/Infera lightweight inference engine/runtime	22	Experimental	1	C++
96	Dimitrios-Kafetzis/EdgeNN Lightweight, zero-allocation C11 library for neural network inference on ARM...	21	Experimental	—	C
97	jazpeak/NanoNet A lightweight, runtime C++ inference engine for running simple ONNX models.	21	Experimental	2	C++
98	torinos-yt/NNOnnx Using CUDA for Faster Machine Learning Inference on Unity	20	Experimental	6	C#
99	DarkStarStrix/Nexa_Inference A inference application to serve Scientific Models	20	Experimental	8	Python
100	MrSyabro/luaort ONNX Runtime binding for Lua	19	Experimental	4	C
101	regel/onnxruntime_server An easy way to start an ONNX inferencing server for prediction with GRPC endpoints	19	Experimental	3	CMake
102	brandon-gong/siliconnn Feedforward neural networks in pure ARM64 assembly for Apple Silicon.	19	Experimental	3	Assembly
103	Darth-Kronos/trt-custom-plugins TensorRT plugins for custom operators	18	Experimental	2	C++
104	glzbcrt/tf-onnx-sample Sample Tensorflow model to classify images and infer using ONNX.	18	Experimental	2	Jupyter Notebook
105	karthi0804/Pytorch-ResNet-CPP-Inference ResNet based Image classification using Pytorch.	18	Experimental	2	C++
106	mlomb/onnx2code Convert ONNX models to plain C++ code (without dependencies)	18	Experimental	22	Python
107	HichemMaiza/onnxruntime-extensions-java Microsoft onnxruntime-extensions builds for Java	17	Experimental	1	—
108	selpoG/ort-sample A Minimal Sample to use onnxruntime from C++.	17	Experimental	1	CMake
109	Konstantina155/InferONNX Lightweight TEE-based system for secure ONNX model inference using Intel SGX...	17	Experimental	5	C
110	fancyovo/XingLing-cpp 🚀 High-performance C++/CUDA Inference Engine for XingLing-0.68B....	14	Experimental	1	Cuda
111	HexaForge-1/highperf-ai-ml-inference High-perf C++ AI/ML inference engine with ONNX Runtime & LibTorch. CPU...	13	Experimental	—	C++
112	Natfii/onnxruntime-gpu-blackwell Pre-built onnxruntime-gpu 1.24.1 with Blackwell sm_120 CUDA kernels (RTX...	13	Experimental	—	—
113	Vitgracer/PyTorch2Cpp-Inference Tutorial that shows how to train a PyTorch model in Python and run it in C++	13	Experimental	7	Python
114	idsia-robotics/navground_onnx Provides a C++ navigation behavior that evaluates a ONNX model.	13	Experimental	—	C++
115	Klaus-Chow/Model-Deployment-And-Inference 涉及到pytorch模型移动端的部署，集成一些主流的目标检测、文本检测和文本识别算法，提供了torch模型到onnx模型的通用接...	13	Experimental	9	C++
116	yas-sim/OpenVINO_Asynchronous_API_Performance_Demo This project demonstrates the high performance of OpenVINO asynchronous inference API	12	Experimental	7	Python
117	tzhenghao/onnxdecoder Convert a large ONNX model file into smaller JSON chunks	11	Experimental	—	Python
118	OneAndZero24/TRTTL TensorRT C++ Template Library	11	Experimental	—	C++
119	rastri-dey/Inference_System Inference system for classification built using Nvidia's Deep Learning tools	11	Experimental	—	—
120	aquapapaya/TVM-SSD-multibox-operator TVM API "scope_attr" is order-sensitive	10	Experimental	2	C++

Comparisons in this category

onnxruntime and onnx (93 vs 85) onnx and onnx-tensorrt (85 vs 63) onnx and tensorflow-onnx (85 vs 63) onnxruntime and onnx-tensorrt (93 vs 63) onnx and onnxmltools (85 vs 73) onnxruntime and onnx2c (93 vs 58) onnxruntime and onnxconverter-common (93 vs 69) TensorRT and onnx-tensorrt (74 vs 63) DALI and dali_backend (67 vs 57) onnxmltools and tensorflow-onnx (73 vs 63)