richardanaya/epistemology
A simple and clear way of hosting llama.cpp as a private HTTP API using Rust
This tool helps developers and IT professionals host a private, local AI assistant using 'llama.cpp' models. It takes your chosen 'llama.cpp' model and executable as input, turning it into a local HTTP API. This allows you to integrate AI capabilities like text completion and embeddings directly into your applications while keeping all data on your machine. It's ideal for those building AI-powered tools who prioritize data privacy and local control.
No commits in the last 6 months.
Use this if you need to run AI models on your own machine, want to ensure complete data privacy, and need a local HTTP endpoint for your applications to interact with these models for tasks like text generation or data embedding.
Not ideal if you need a cloud-hosted solution, prefer pre-built AI services, or require extensive logging and monitoring capabilities for your AI deployments.
Stars
26
Forks
2
Language
Rust
License
MIT
Category
Last pushed
Jun 22, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/richardanaya/epistemology"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
trymirai/uzu
A high-performance inference engine for AI models
justrach/bhumi
⚡ Bhumi – The fastest AI inference client for Python, built with Rust for unmatched speed,...
lipish/llm-connector
LLM Connector - A unified interface for connecting to various Large Language Model providers
keyvank/femtoGPT
Pure Rust implementation of a minimal Generative Pretrained Transformer
ShelbyJenkins/llm_client
The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from...