Buoyant Launches Open Source Kubernetes Control Plane for AI Inference
San Francisco-based firm Buoyant has released Kinference, an open-source tool designed to manage and optimize AI model inference on Kubernetes clusters.
Buoyant, the developers behind the service mesh Linkerd, has announced the general availability of Kinference. The software is designed to function as an inference control plane for organizations running generative AI models on Kubernetes, providing routing, load balancing, and policy management.
As businesses increasingly shift toward self-hosting open-weight models to reduce API costs and maintain data control, the challenge of managing GPU resources at scale has grown. Kinference acts as a layer atop the vLLM engine, allowing platform teams to implement governance, cost attribution, and intent-based routing across diverse model environments.
According to the company, the tool aims to improve GPU utilization by analyzing real-time metrics such as cache data and queue status to minimize latency. Benchmark tests cited by Buoyant suggest the software significantly reduces time-to-first-token compared to standard Kubernetes load balancing, though actual performance will vary by infrastructure configuration.
Kinference is now available under an Apache 2.0 license. The release marks a strategic effort by Buoyant to apply its background in infrastructure management to the expanding market for local AI deployment.