haormj/llama2.go
Inference Llama 2 in one file of pure go
This tool helps Go developers integrate the Llama 2 large language model directly into their Go applications. It takes a pre-trained Llama 2 model file and a tokenizer file as input and allows the application to perform text generation and other inference tasks. Go developers who need to embed Llama 2's capabilities into their Go projects without external dependencies would use this.
No commits in the last 6 months.
Use this if you are a Go developer building an application and want to include local Llama 2 model inference capabilities directly within your Go codebase.
Not ideal if you are not a Go developer or if you need to fine-tune Llama 2 models, as this project focuses solely on inference.
Stars
16
Forks
1
Language
Go
License
MIT
Category
Last pushed
Jul 25, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/haormj/llama2.go"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ludwig-ai/ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
withcatai/node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema...
mudler/LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and...
zhudotexe/kani
kani (カニ) is a highly hackable microframework for tool-calling language models. (NLP-OSS @ EMNLP 2023)
SciSharp/LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.