smarter-project/armnn_tflite_backend
TensorFlow Lite backend with ArmNN delegate support for Nvidia Triton
This tool helps machine learning engineers efficiently deploy and run TensorFlow Lite models on Arm-based systems. It takes your TFLite serialized models and outputs optimized model inference, allowing you to serve predictions faster. It's designed for developers managing inference servers on edge devices or embedded systems.
No commits in the last 6 months.
Use this if you need to accelerate TensorFlow Lite model inference using ArmNN or XNNPACK on an Nvidia Triton server running on Linux Arm64 hardware.
Not ideal if you are working with non-TFLite models or deploying to x86-based servers, or if you don't use Nvidia Triton as your inference serving solution.
Stars
8
Forks
2
Language
C++
License
MIT
Category
Last pushed
Sep 03, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/smarter-project/armnn_tflite_backend"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
microsoft/onnxruntime
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
onnx/onnx
Open standard for machine learning interoperability
PINTO0309/onnx2tf
Self-Created Tools to convert ONNX files (NCHW) to TensorFlow/TFLite/Keras format (NHWC). The...
NVIDIA/TensorRT
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This...
onnx/onnxmltools
ONNXMLTools enables conversion of models to ONNX