retkowsky/foundry-local
Foundry Local is an on-device AI inference solution that you use to run AI models locally through a CLI, SDK, or REST API
Foundry Local helps developers run generative AI models directly on their own computers (Windows, macOS, or servers) instead of relying on cloud services. It takes AI models as input and outputs the results of their inference, allowing for tasks like real-time chat or content generation. Developers, data scientists, and engineers who build AI-powered applications will use this.
Use this if you need to run AI models on-device for applications requiring strict data privacy, offline capabilities, low latency, or cost efficiency, without sending data to the cloud.
Not ideal if you prefer a fully managed cloud-based AI service or don't have the local hardware resources (e.g., sufficient RAM or GPU/NPU) to run models efficiently.
Stars
9
Forks
4
Language
Jupyter Notebook
License
—
Category
Last pushed
Feb 26, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/generative-ai/retkowsky/foundry-local"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
openvinotoolkit/model_server
A scalable inference server for models optimized with OpenVINO™
madroidmaq/mlx-omni-server
MLX Omni Server is a local inference server powered by Apple's MLX framework, specifically...
NVIDIA-NeMo/Guardrails
NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based...
generative-computing/mellea
Mellea is a library for writing generative programs.
rhesis-ai/rhesis
Open-source platform & SDK for testing LLM and agentic apps. Define expected behavior, generate...