feiyun0112/Local-LLM-Server
quick way to build a private large language model server and provide OpenAI-compatible interfaces | 快速搭建私有大语言模型（LLM）服务，提供OpenAI兼容接口
This project helps developers and data scientists quickly set up a private server for large language models (LLMs) on their own machines. It takes downloaded LLM files as input and provides an OpenAI-compatible interface, allowing for local execution of chat and embedding models. This is ideal for those needing to run LLMs without internet access or with strict data privacy requirements.
No commits in the last 6 months.
Use this if you need to run large language models locally and securely, without relying on external cloud services or public APIs.
Not ideal if you prefer to use managed LLM services, do not have local computing resources, or require support for a very wide range of less common models out-of-the-box.
Stars
34
Forks
8
Language
Python
License
Apache-2.0
Category
Last pushed
Jan 06, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/feiyun0112/Local-LLM-Server"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
containers/ramalama
RamaLama is an open-source developer tool that simplifies the local serving of AI models from...
av/harbor
One command brings a complete pre-wired LLM stack with hundreds of services to explore.
RunanywhereAI/runanywhere-sdks
Production ready toolkit to run AI locally
runpod-workers/worker-vllm
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
foldl/chatllm.cpp
Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)