ELS-RD/transformer-deploy
Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
This tool helps data science and machine learning engineers deploy Hugging Face Transformer models into production systems with sub-millisecond response times. It takes your existing Transformer models and outputs highly optimized versions ready for use with enterprise-grade inference servers like Nvidia Triton. This is perfect for engineers building real-time applications such as semantic search engines, content recommendation systems, or chatbots.
1,688 stars. No commits in the last 6 months.
Use this if you need to serve Hugging Face Transformer models for tasks like classification, named entity recognition, or text generation with extremely low latency in a production environment.
Not ideal if you are just prototyping models or don't require high-performance, low-latency inference for your applications.
Stars
1,688
Forks
152
Language
Python
License
Apache-2.0
Category
Last pushed
Oct 23, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/ELS-RD/transformer-deploy"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
modelscope/modelscope
ModelScope: bring the notion of Model-as-a-Service to life.
basetenlabs/truss
The simplest way to serve AI/ML models in production
Lightning-AI/LitServe
A minimal Python framework for building custom AI inference servers with full control over...
deepjavalibrary/djl-serving
A universal scalable machine learning model deployment solution
tensorflow/serving
A flexible, high-performance serving system for machine learning models