arm-education/Advanced-AI-Hardware-Software-Co-Design
Hands-on course materials for ML engineers to master extreme model quantization and on-device LLM deployment: PyTorch, llama.cpp, Android (educational)
This course helps machine learning engineers and researchers shrink large AI models and run them efficiently on devices like Android smartphones. It provides hands-on exercises to reduce model size and speed up inference, taking raw deep learning models and outputting highly optimized versions that run directly on edge hardware. It's for professionals working to deploy advanced AI in real-world applications.
Use this if you need to deploy large language models (LLMs) and other generative AI models directly onto edge devices with limited resources, while maintaining performance.
Not ideal if you are new to deep learning, Python, or PyTorch, or if your primary goal is cloud-based AI deployment.
Stars
34
Forks
9
Language
Jupyter Notebook
License
—
Category
Last pushed
Dec 08, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/arm-education/Advanced-AI-Hardware-Software-Co-Design"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
quic/efficient-transformers
This library empowers users to seamlessly port pretrained models and checkpoints on the...
ManuelSLemos/RabbitLLM
Run 70B+ LLMs on a single 4GB GPU — no quantization required.
alpa-projects/alpa
Training and serving large-scale neural networks with auto parallelization.
IST-DASLab/marlin
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes...
deepreinforce-ai/CUDA-L2
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning