mithril-security/blind_llama_client
Zero-trust AI APIs for easy and private consumption of open-source LLMs
This project helps developers integrate open-source Large Language Models (LLMs) like Llama2 into their applications while ensuring the privacy of user data. It takes user input and securely processes it with an LLM, returning the model's output without exposing the data to the service provider. The primary users are developers building applications that need to leverage LLMs for sensitive information.
No commits in the last 6 months.
Use this if you are a developer building an application that needs to use open-source LLMs but requires strong guarantees that user data remains confidential and is not exposed to the AI service provider.
Not ideal if you are a non-technical end-user looking for a ready-to-use LLM chat interface, or if your application does not handle sensitive data that requires advanced privacy protection.
Stars
41
Forks
4
Language
Python
License
Apache-2.0
Category
Last pushed
Jul 25, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/mithril-security/blind_llama_client"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Arize-ai/openinference
OpenTelemetry Instrumentation for AI Observability
vndee/llm-sandbox
Lightweight and portable LLM sandbox runtime (code interpreter) Python library.
apache/hertzbeat
An AI-powered next-generation open source real-time observability system.
traceloop/openllmetry
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
utkuozdemir/nvidia_gpu_exporter
Nvidia GPU exporter for prometheus using nvidia-smi binary