Kubernetes resource utilisation in the AI era

Cast AI’s 2026 State of Kubernetes Optimisation Report examines Kubernetes resource utilisation and identifies areas of inefficiency in GPU, CPU, and memory usage across clusters.

Cast AI has recently published its 2026 State of Kubernetes Optimisation Report, providing an analysis of GPU, CPU, and memory utilisation across non-optimised Kubernetes clusters. The report is based on data from tens of thousands of clusters and offers insights into how resources are being used.

One of the key findings is the underutilisation of GPUs relative to their cost, with average usage recorded at 5%. The report notes that expected efficiency gains associated with Kubernetes at scale are not consistently being realised, and that there is a widening gap between organisational spend and actual resource utilisation.

While Kubernetes adoption continues to grow across organisations, the data shows relatively low resource utilisation. In 2025, average CPU utilisation across clusters was 8%, and memory utilisation was 20%.

The report also highlights the increasing use of GPU-equipped nodes as organisations run more AI and machine learning workloads on Kubernetes. Despite this, GPU utilisation remains low at around 5%, indicating a level of unused capacity within AI infrastructure investments.

As organisations expand their use of AI, the report points to the need for ongoing optimisation to manage GPU usage more effectively and reduce waste within cloud environments.

Cast AI also identifies a common limitation in Kubernetes environments: configuration is often treated as a one-time activity at deployment. However, workloads and traffic patterns change over time, meaning initial configurations may become less effective. The same applies to areas such as Spot Instance selection, autoscaler configuration, commitment usage, and node lifecycle management, all of which require ongoing adjustment that can be difficult to maintain manually at scale.

Cohesity receives Google Cloud Partner of the Year Award for infrastructure modernisation in...
UK business leaders report increased concern about AI-related cybersecurity risks, alongside...
Hyland has introduced new solutions aimed at helping organisations manage enterprise content and...
GTT Communications plans to expand its global network services in 2026, including AI enhancements...
Exploring the gap between AI adoption and data governance in organisations amid rising concerns...
SUSE and NVIDIA have collaborated on an AI Factory designed to support enterprises in deploying and...
The new CIS Companion Guides provide security guidance for emerging AI environments, including LLMs...
ServiceNow’s planned acquisition of Armis aims to expand its market position and add capabilities...