HyperMink/inferenceable
Scalable AI Inference Server for CPU and GPU with Node.js | Utilizes llama.cpp and parts of llamafile C/C++ core under the hood.
This is a server that allows you to easily run large language models (LLMs) and vision models on your own computers, using either your CPU or GPU. You can input text or images, and the server will generate text responses or descriptions. It's designed for developers who need to integrate AI inference capabilities into their applications without complex setup.
No commits in the last 6 months.
Use this if you are a developer looking for a straightforward, scalable server to deploy and manage AI language and vision models for your applications.
Not ideal if you are an end-user without development experience who just wants to chat with an AI model without any technical setup.
Stars
15
Forks
2
Language
JavaScript
License
Apache-2.0
Category
Last pushed
May 30, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/HyperMink/inferenceable"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
alibaba/MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering...
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source,...
tensorzero/tensorzero
TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM...