nicholasyager/llama-cpp-guidance
A guidance compatibility layer for llama-cpp-python
This is a tool for developers who are building applications powered by large language models (LLMs) locally. It allows you to use a specific type of local LLM setup (llama-cpp-python) with a popular framework for controlling LLM output (Guidance). This helps you create more reliable and structured text generation for your applications.
No commits in the last 6 months.
Use this if you are a developer building an application that leverages local LLMs and you need to precisely control the structure and content of the text generated by the model.
Not ideal if you are an end-user looking for a ready-to-use application, or if you are working with cloud-based LLMs rather than local ones.
Stars
36
Forks
4
Language
Python
License
—
Category
Last pushed
Sep 11, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/nicholasyager/llama-cpp-guidance"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
beehive-lab/GPULlama3.java
GPU-accelerated Llama3.java inference in pure Java using TornadoVM.
gitkaz/mlx_gguf_server
This is a FastAPI based LLM server. Load multiple LLM models (MLX or llama.cpp) simultaneously...
srgtuszy/llama-cpp-swift
Swift bindings for llama-cpp library
JackZeng0208/llama.cpp-android-tutorial
llama.cpp tutorial on Android phone
awinml/llama-cpp-python-bindings
Run fast LLM Inference using Llama.cpp in Python