OutofAi/ChitChat
Modal LLM LLama.cpp based model deployment as part of series of Model as a Service (MaaS)
This project helps developers quickly set up and deploy a custom large language model (LLM) as an inference endpoint. You provide a Llama.cpp-compatible model, and it outputs a web address that allows other applications to send text prompts to your model and receive responses. This is primarily for AI/ML developers or researchers who need to serve their own LLMs.
Use this if you are a developer looking for a straightforward and cost-effective way to deploy a Llama.cpp-compatible LLM and get an API endpoint for it.
Not ideal if you are a non-technical user looking for a ready-to-use chatbot or if you need to deploy models that are not compatible with Llama.cpp.
Stars
17
Forks
—
Language
Python
License
MIT
Category
Last pushed
Feb 27, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/OutofAi/ChitChat"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
containers/ramalama
RamaLama is an open-source developer tool that simplifies the local serving of AI models from...
av/harbor
One command brings a complete pre-wired LLM stack with hundreds of services to explore.
RunanywhereAI/runanywhere-sdks
Production ready toolkit to run AI locally
runpod-workers/worker-vllm
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
foldl/chatllm.cpp
Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)