fbaldassarri/llama-cpp-container
Docker image to deploy a llama-cpp container with conda-ready environments
This helps data scientists and AI researchers deploy large language models (LLMs) on their own Linux machines with NVIDIA GPUs. It provides a ready-to-use Docker container for running `llama-cpp` models, allowing you to quickly get LLMs up and running without complex setup. You provide the GPU-enabled Linux environment, and it gives you a containerized LLM inference server.
No commits in the last 6 months.
Use this if you are a data scientist or AI researcher who needs to quickly deploy and experiment with `llama-cpp` based large language models on your local GPU-equipped Linux workstation.
Not ideal if you don't have a Linux machine with an NVIDIA GPU, or if you need to run LLMs in a cloud environment or on different hardware.
Stars
17
Forks
1
Language
Dockerfile
License
MIT
Category
Last pushed
Apr 27, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/fbaldassarri/llama-cpp-container"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ludwig-ai/ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
withcatai/node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema...
mudler/LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and...
zhudotexe/kani
kani (カニ) is a highly hackable microframework for tool-calling language models. (NLP-OSS @ EMNLP 2023)
SciSharp/LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.