soulteary/docker-llama2-chat
Play LLaMA2 (official / 中文版 / INT4 / llama2.cpp) Together! ONLY 3 STEPS! ( non GPU / 5GB vRAM / 8~14GB vRAM)
This project helps developers easily set up and experiment with large language models (LLMs) like LLaMA2 and its Chinese versions on their local machines. It simplifies the process of getting an LLM chat interface running, taking model files as input and providing a web-based chat application as output. This tool is ideal for machine learning engineers, researchers, or software developers who want to test or integrate these models without complex setup.
538 stars. No commits in the last 6 months.
Use this if you need to quickly deploy and interact with a LLaMA2 or Chinese LLaMA2 model locally for development or testing.
Not ideal if you are looking for a production-ready, highly scalable LLM deployment solution or do not have experience with Docker.
Stars
538
Forks
84
Language
Python
License
Apache-2.0
Category
Last pushed
Aug 14, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/soulteary/docker-llama2-chat"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ludwig-ai/ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
withcatai/node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema...
mudler/LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and...
zhudotexe/kani
kani (カニ) is a highly hackable microframework for tool-calling language models. (NLP-OSS @ EMNLP 2023)
SciSharp/LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.