chelsea0x3b/llama-dfdx
LLaMa 7b with CUDA acceleration implemented in rust. Minimal GPU memory needed!
This tool allows developers to integrate the LLaMa 7b large language model directly into their Rust applications. It takes the LLaMa model weights and a text prompt as input, and outputs generated text or engaging in a chat-like interaction. It is designed for Rust developers who need to embed powerful language generation capabilities directly within their software.
111 stars. No commits in the last 6 months.
Use this if you are a Rust developer looking to embed a performant LLaMa 7b language model, leveraging CUDA for acceleration, directly within your application.
Not ideal if you are not a Rust developer or do not have access to a CUDA-compatible GPU, as CPU performance without CUDA is not supported.
Stars
111
Forks
6
Language
Rust
License
MIT
Category
Last pushed
Jul 27, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/chelsea0x3b/llama-dfdx"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ludwig-ai/ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
withcatai/node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema...
mudler/LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and...
zhudotexe/kani
kani (カニ) is a highly hackable microframework for tool-calling language models. (NLP-OSS @ EMNLP 2023)
SciSharp/LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.