K024/llm-sharp
Language models in C#
llm-sharp is a tool that allows C# developers to run and serve language models like Llama2, Qwen, and Bert directly within their C# applications. It takes pre-trained language model files (like safetensors) and tokenizer data as input, and provides an API or command-line interface for generating text completions or embeddings. This is ideal for developers building C# applications that need to incorporate AI language capabilities without relying on Python-specific environments for model inference.
No commits in the last 6 months.
Use this if you are a C# developer who wants to integrate popular language models and their inference capabilities directly into your .NET applications for production use.
Not ideal if you are not a C# developer or if you require advanced model parallelism features beyond what's currently implemented, such as tensor parallelism.
Stars
50
Forks
7
Language
C#
License
MIT
Category
Last pushed
Dec 30, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/K024/llm-sharp"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ludwig-ai/ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
withcatai/node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema...
mudler/LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and...
zhudotexe/kani
kani (カニ) is a highly hackable microframework for tool-calling language models. (NLP-OSS @ EMNLP 2023)
SciSharp/LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.