withcatai/node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
This project helps JavaScript and TypeScript developers integrate advanced AI capabilities directly into their applications by running large language models (LLMs) on their own machines. Developers input a language model and prompts, and the tool outputs structured text, function calls, or embeddings, enabling features like smart chatbots, data summarization, or advanced search within their applications. It's designed for developers building AI-powered features without relying on external cloud services.
1,942 stars. Used by 6 other packages. Actively maintained with 2 commits in the last 30 days. Available on npm.
Use this if you are a developer looking to embed AI models directly into your JavaScript/TypeScript applications, especially when you need precise control over model output formatting like JSON.
Not ideal if you are an end-user who just wants to chat with an AI or if you prefer using cloud-based AI services without local setup.
Stars
1,942
Forks
176
Language
TypeScript
License
MIT
Category
Last pushed
Mar 12, 2026
Commits (30d)
2
Dependencies
28
Reverse dependents
6
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/withcatai/node-llama-cpp"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related models
ludwig-ai/ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
mudler/LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and...
zhudotexe/kani
kani (カニ) is a highly hackable microframework for tool-calling language models. (NLP-OSS @ EMNLP 2023)
SciSharp/LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
kaito-project/aikit
🏗️ Fine-tune, build, and deploy open-source LLMs easily!