absadiki/pyllamacpp
Python bindings for llama.cpp
This project helps developers integrate large language models (LLMs) like LLaMA, Alpaca, or GPT4All into their Python applications. It allows you to run these powerful models directly on a CPU, even on Apple silicon, taking model files as input and generating text responses. It's for developers building AI-powered chat applications, content generators, or language understanding tools without needing specialized GPU hardware.
No commits in the last 6 months.
Use this if you are a Python developer who wants to run large language models on your CPU with good performance and minimal dependencies.
Not ideal if you are an end-user looking for a ready-to-use application or if you require the absolute fastest inference speeds offered by dedicated GPU hardware.
Stars
68
Forks
24
Language
C++
License
MIT
Category
Last pushed
Feb 29, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/absadiki/pyllamacpp"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ludwig-ai/ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
withcatai/node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema...
mudler/LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and...
zhudotexe/kani
kani (カニ) is a highly hackable microframework for tool-calling language models. (NLP-OSS @ EMNLP 2023)
SciSharp/LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.