makit/makit-llm-lambda
Example showing how to run a LLM fully inside an AWS Lambda Function
This helps developers integrate and run a Large Language Model (LLM) like GPT4ALL directly within an AWS Lambda serverless function. You provide the pre-trained LLM model file and a text prompt via an HTTP call, and it returns a generated text response. This is primarily for backend developers or cloud architects looking to deploy custom LLMs without managing dedicated servers.
No commits in the last 6 months.
Use this if you are a backend developer or cloud architect who needs to host a custom, open-source LLM for basic text generation in a cost-effective, serverless environment.
Not ideal if you need a high-performance, low-latency LLM for complex, real-time applications or if your model is too large for Lambda's memory constraints.
Stars
23
Forks
4
Language
Dockerfile
License
MIT
Category
Last pushed
Jan 13, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/makit/makit-llm-lambda"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the...
josStorer/RWKV-Runner
A RWKV management and startup tool, full automation, only 8MB. And provides an interface...
waybarrios/vllm-mlx
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models...
jordanhubbard/nanolang
A tiny experimental language designed to be targeted by coding LLMs
akivasolutions/tightwad
Pool your CUDA + ROCm GPUs into one OpenAI-compatible API. Speculative decoding proxy gives you...