Maximilian-Winter/llama-cpp-agent
The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.
This framework helps developers integrate Large Language Models (LLMs) into their applications, even if the models weren't specifically trained for structured outputs. It takes raw text or conversational prompts and produces either chat responses, structured data (like a book's details from unstructured text), or the results of function calls. Developers building intelligent applications would use this to add AI capabilities.
620 stars. Available on PyPI.
Use this if you are a developer looking to add advanced LLM interaction, such as structured output or function calling, to your applications using locally run or private models.
Not ideal if you are looking for a commercial-grade, fully managed LLM API integration framework.
Stars
620
Forks
69
Language
Python
License
—
Category
Last pushed
Mar 09, 2026
Commits (30d)
0
Dependencies
5
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/Maximilian-Winter/llama-cpp-agent"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
mozilla-ai/any-llm
Communicate with an LLM provider using a single interface
CliDyn/climsight
A next-generation climate information system that uses large language models (LLMs) alongside...
ShishirPatil/gorilla
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
OoriData/OgbujiPT
Client-side toolkit for using large language models, including where self-hosted
onnx/turnkeyml
No-code CLI designed for accelerating ONNX workflows