danielgrittner/llama2-rs
LLaMA2 + Rust
This is a tool for developers who want to experiment with or integrate LLaMA2 models within Rust applications. It takes a pre-trained LLaMA2 model and a text prompt, then generates new, coherent text based on the input. This is useful for backend engineers or systems programmers working on applications requiring local, fast text generation capabilities.
No commits in the last 6 months.
Use this if you are a Rust developer looking for a foundational implementation of LLaMA2 for integration into your projects, particularly where performance and control over the model execution are key.
Not ideal if you are an end-user looking for a ready-to-use application for text generation, or if you need to train or fine-tune models.
Stars
13
Forks
—
Language
Rust
License
—
Category
Last pushed
Aug 08, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/danielgrittner/llama2-rs"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
trymirai/uzu
A high-performance inference engine for AI models
justrach/bhumi
⚡ Bhumi – The fastest AI inference client for Python, built with Rust for unmatched speed,...
lipish/llm-connector
LLM Connector - A unified interface for connecting to various Large Language Model providers
keyvank/femtoGPT
Pure Rust implementation of a minimal Generative Pretrained Transformer
ShelbyJenkins/llm_client
The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from...