KerfuffleV2/smolrsrwkv
A relatively basic implementation of RWKV in Rust written by someone with very little math and ML knowledge. Supports 32, 8 and 4 bit evaluation. It can also directly load PyTorch RWKV models.
This tool helps developers who want to experiment with or integrate RWKV (Recurrent pWKv) language models into their applications. It takes pre-trained RWKV model files (PyTorch or SafeTensors format) and a tokenizer, then processes text prompts to output a list of probabilities for the next possible tokens. It's designed for those working with Rust, offering a CPU-only solution without large machine learning frameworks.
No commits in the last 6 months.
Use this if you are a developer looking for a lightweight, CPU-based Rust implementation to run RWKV language model inference with direct PyTorch model loading.
Not ideal if you need high-performance, GPU-accelerated inference or extensive model quantization capabilities, as it's not optimized for speed and only quantizes on the fly.
Stars
94
Forks
4
Language
Rust
License
MIT
Category
Last pushed
Sep 02, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/vector-db/KerfuffleV2/smolrsrwkv"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.