oMLX : Run Local AI Models on Apple Silicon with Persistent KV Cache

oMLX is a dedicated inference server for Apple Silicon, optimized to handle complex, long-running AI tasks by solving operational bottlenecks like context window management, memory pressure, and client request conflicts. By employing persistent, tiered KV caching, it effectively handles large contexts while separating memory management for active inference from the storage of long-term state on SSDs. This ensures reliability for long sessions and allows concurrent requests to share common context prefixes, drastically improving throughput and efficiency. The tool supports multiple installation methods, including a native macOS app, Homebrew, and source, and exposes an OpenAI-compatible API to facilitate integration with existing coding assistants and applications. Ultimately, oMLX balances performance, resource management, and compatibility to make running sophisticated models on Apple hardware more reliable and manageable.