A 40MB Platform to Manage a Thousand GPU Servers: GPUWorks Is Now Open Source

Nov 4, 2026

Shanghai, China, November 4, 2026 -- Is it worth building a complex infrastructure stack just for a small GPU fleet? The platform supports heterogeneous hardware. Which card can run which model? Compute is limited — how do multiple large models share the same machines?

These questions, raised by real users, are the starting point for GPUWorks. Enterprises need an inference platform that is simple to deploy, explicit about hardware compatibility, efficient in its use of compute, and able to keep scaling as the business grows.

SHANGHAI, Nov. 4, 2026 - Today, ZStack is officially open-sourcing GPUWorks, a lightweight, enterprise-grade AI inference platform, under the Apache License 2.0.

A small footprint does not mean limited capability. In tests on physical hardware, one GPUWorks Server managed 1,003 GPU servers and brought up more than 500 inference replicas in 4 minutes, delivering 69,000 tokens per second, while the Server itself used only 9 CPU cores and 1.6 GB of memory.

Ultra-Lightweight: a 40+MB Platform, Deployed in 60 Seconds

GPUWorks lowers the barrier to deployment and gets models into production faster. No pre-existing Kubernetes cluster is required. It installs with a single command. From a browser, administrators onboard nodes, view available compute, and deploy models. Models and container images can also be imported offline within the corporate network.

Heterogeneous compatibility is defined for each model-chip pairing. Nearly 400 models and 1,000+ launch configurations bundle the container image, inference engine, weights, and parameters into reusable profiles. Paired with VRAM estimation, these profiles reduce environment setup and repeated debugging.

Time-Shared Scheduling: Run Multiple Large Models on a Single GPU Pool, on Demand

A single GPU pool can be time-shared efficiently across multiple large models. Enterprises do not need to reserve a dedicated pool of compute for each model over the long term. This lets teams deploy more models on hardware they already have and serve different business needs.

Administrators can configure priority, yield-on-idle (release resources when idle), and scheduled-rotation policies. The platform automatically coordinates model sleep, resource release, and wake-up. Models that require multi-GPU coordination are switched as a whole GPU group, which reduces manual stop/start work and resource reconfiguration.

Model service wake-up is about 200× faster than a cold start. On 8× H20, a cold start that reads weights from NFS takes 16–17.5 minutes. L1 memory wake-up restores service in about 4 seconds; DeepSeek-V4.1-Flash returns in as little as 3.6 seconds. Weights stay resident in host memory, so recovery skips engine restarts and repeated weight loading from storage. The wait is substantially shorter when large models take turns.

Thousand-Node Scale: Unified Management of Scattered Compute as the Business Grows

Administrators can start with a small fleet and scale to unified management of 1,000+ nodes. GPUWorks supports NVIDIA and AMD, as well as China-made chips including Ascend, T-Head PPU, and Hygon DCU. Bare metal, virtual machines, and cloud containers across on-prem data centers, private clouds, and public clouds sit on a single management plane. This reduces the burden of maintaining each environment separately.

When a model outgrows the capacity of a single host, it can be served in parallel across multiple machines. As request volume grows, administrators can spin up additional inference replicas to share the requests. Prefill-decode disaggregation makes it possible to allocate resources separately for input processing and content generation, based on actual load, so scale-out targets the performance bottleneck.

From enterprise knowledge bases and question answering to voice interaction and content generation, GPUWorks provides a unified entry point for model serving. Administrators can view GPU health, request latency, and usage, and diagnose failures with an AI assistant. The basis for each scheduling decision can be inspected, so teams can allocate resources and run operations with greater confidence.

Make Any GPU Work. Try GPUWorks, report issues, and contribute inference templates and chip adaptations, so more models can run on more compute.

Repository: github.com/flitai-io/gpuworks

About ZStack

Founded in 2015, ZStack is a cloud infrastructure software provider serving more than 5,000 enterprise customers across over 36 countries and regions. Its product portfolio spans virtualization, private cloud, hyperconverged infrastructure, software-defined storage and AI infrastructure. ZStack is recognized in Gartner's Market Guide for Server Virtualization Platforms as a representative vendor. Learn more at https://www.zstack-cloud.com

Contact Info:
Name: Ge, Rainsy
Email: Send Email
Organization: ZStack
Website: https://www.zstack-cloud.com

Release ID: 89208472

Should you come across any errors, concerns, or inconsistencies within this press release's content, we urge you to reach out without delay by contacting [email protected] (it is important to note that this email is the authorized channel for such matters, sending multiple emails to multiple addresses does not necessarily help expedite your request). Our committed team will promptly address your feedback within 8 hours and take appropriate measures to resolve any identified issues or guide you through the removal process. Providing accurate and dependable information remains our utmost priority.