fly-apps/bumblebee-model-harness
Minimal Elixir application that can host AI models on Fly.io GPUs and make them available via WireGuard to remote clustered Elixir applications for improved development experiences.
This project offers a way for Elixir developers to quickly access powerful AI models like Llama 2, Zephyr, or Mistral, running on cloud GPUs. It allows your local Elixir application to send requests to a model hosted in the cloud, receiving AI-generated responses back. This is designed for Elixir developers who want to integrate large language models into their applications without managing complex GPU infrastructure locally.
No commits in the last 6 months.
Use this if you are an Elixir developer building an application that needs to use large AI models, and you want to keep your local development fast while offloading the heavy computation to cloud GPUs.
Not ideal if you are not an Elixir developer, or if your application doesn't require access to large language models.
Stars
52
Forks
7
Language
Elixir
License
MIT
Category
Last pushed
Apr 16, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/fly-apps/bumblebee-model-harness"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
BerriAI/litellm
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with...
vava-nessa/free-coding-models
Find, benchmark and install in CLI 158 FREE coding LLM models across 20 providers in real time
envoyproxy/ai-gateway
Manages Unified Access to Generative AI Services built on Envoy Gateway
theopenco/llmgateway
Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
Portkey-AI/gateway
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with...