serialscriptr/Orange-PI-5-Pro-MLC-LLM
Guide I wrote mostly for myself on how to run mlc-llm on the Orange Pi 5 Pro
This guide shows you how to set up your Orange Pi 5 Pro to run powerful large language models (LLMs) like those found in chatbots, right on your device. It walks you through installing the necessary operating system and drivers, and preparing your hardware. The output is a fully configured Orange Pi 5 Pro capable of running local LLMs, perfect for hobbyists or developers looking for an affordable, localized AI solution.
No commits in the last 6 months.
Use this if you want to run large language models directly on an Orange Pi 5 Pro or similar Rockchip-based single-board computer, rather than relying on cloud services.
Not ideal if you prefer to use pre-built, ready-to-run LLM applications without needing to configure a low-level operating system and drivers.
Stars
23
Forks
3
Language
—
License
GPL-3.0
Category
Last pushed
Aug 15, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/serialscriptr/Orange-PI-5-Pro-MLC-LLM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
thu-pacman/chitu
High-performance inference framework for large language models, focusing on efficiency,...
sophgo/LLM-TPU
Run generative AI models in sophgo BM1684X/BM1688
NotPunchnox/rkllama
Ollama alternative for Rockchip NPU: An efficient solution for running AI and Deep learning...
Deep-Spark/DeepSparkHub
DeepSparkHub selects hundreds of application algorithms and models, covering various fields of...
howard-hou/VisualRWKV
VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle...