triton-inference-server/pytriton

PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.

/ 100

Emerging

This tool helps machine learning engineers and data scientists deploy their Python-based machine learning models for predictions in production. You provide your trained model as a Python function, and it outputs a high-performance serving endpoint accessible via HTTP or gRPC, ready to handle inference requests. It's designed for those who need to put their models into real-world use.

835 stars. No commits in the last 6 months.

Use this if you need to quickly deploy your Python machine learning model (from frameworks like PyTorch, TensorFlow, or JAX) and serve predictions efficiently.

Not ideal if you are looking for a tool to train models or if your primary need is data preprocessing rather than model serving.

ML model deployment real-time inference AI productization model serving MLOps

Stale 6m No Package No Dependents

Maintenance 2 / 25

Adoption 10 / 25

Maturity 16 / 25

Community 15 / 25

How are scores calculated?

Stars

835

Forks

Language

Python

License

Apache-2.0

Higher-rated alternatives

triton-inference-server/server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

gpu-mode/Triton-Puzzles

Puzzles for learning Triton

hailo-ai/hailo_model_zoo

The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment

open-mmlab/mmdeploy

OpenMMLab Model Deployment Framework

hyperai/tvm-cn

TVM Documentation in Chinese Simplified / TVM 中文文档

Explore ML Frameworks

All categories Trending ML Framework directory Insights