ziozzang/Mac_mlx_phi-2_server
Test server code for Phi-2 model. support OpenAI API spec
This project provides a local server that lets you interact with the Phi-2 language model on your Apple Silicon Mac, mimicking the OpenAI API. You feed it text prompts, and it generates responses, just like using an external AI service. It's designed for researchers or developers who want to experiment with a powerful small language model locally.
No commits in the last 6 months.
Use this if you want to quickly test and integrate the Phi-2 language model on your Apple Silicon Mac using an OpenAI-compatible API.
Not ideal if you need a production-ready server or are looking for a tool to run Phi-2 on non-Apple hardware.
Stars
18
Forks
2
Language
Python
License
MIT
Category
Last pushed
Dec 15, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/ziozzang/Mac_mlx_phi-2_server"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the...
josStorer/RWKV-Runner
A RWKV management and startup tool, full automation, only 8MB. And provides an interface...
waybarrios/vllm-mlx
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models...
jordanhubbard/nanolang
A tiny experimental language designed to be targeted by coding LLMs
akivasolutions/tightwad
Pool your CUDA + ROCm GPUs into one OpenAI-compatible API. Speculative decoding proxy gives you...