rbitr/llm.f90
LLM inference in Fortran
This project allows developers to run large language models (LLMs) on their own computers using Fortran. It takes a pre-trained LLM model file (like a GGUF file) and a text prompt as input, then generates text completions. The output is the generated text and performance metrics. This is for developers or researchers who want direct control over LLM inference on CPU without complex frameworks.
No commits in the last 6 months.
Use this if you are a developer who needs to run LLM inference on a CPU with minimal dependencies, desire high performance from a simple, hackable codebase, and want to integrate or customize the language model at a low level.
Not ideal if you are a non-developer seeking an out-of-the-box application for general LLM use without programming, or if you require extensive multi-platform support or GPU acceleration directly from this tool.
Stars
64
Forks
8
Language
Fortran
License
MIT
Category
Last pushed
May 30, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/rbitr/llm.f90"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
alibaba/MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering...
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source,...
tensorzero/tensorzero
TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM...