usc-isi/PipeEdge
PipeEdge: Pipeline Parallelism for Large-Scale Model Inference on Heterogeneous Edge Devices
This framework helps machine learning engineers and researchers efficiently run large neural network models like transformers on multiple, diverse edge devices. It takes your trained model and automatically splits its layers across these devices, optimizing how fast it processes data. The output is a highly performant inference system, even with limited computing resources, ideal for specialized ML deployments.
No commits in the last 6 months.
Use this if you need to run large AI models efficiently on a network of edge devices and want to maximize throughput without manual configuration.
Not ideal if you are working with small models that don't require distributed processing or if you only deploy to powerful, single-node servers.
Stars
38
Forks
28
Language
Python
License
BSD-3-Clause
Category
Last pushed
Jan 31, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/usc-isi/PipeEdge"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
tensorflow/tensorflow
An Open Source Machine Learning Framework for Everyone
microsoft/tensorwatch
Debugging, monitoring and visualization for Python Machine Learning and Data Science
KomputeProject/kompute
General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics...
hailo-ai/hailort-drivers
The Hailo PCIe driver is required for interacting with a Hailo device over the PCIe interface
NVIDIA/nvshmem
NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM...