GPU utilisation rates routinely sit well below 50%. Some workloads burst to full capacity for hours, then idle. Others need two GPUs for inference but are locked to a server provisioned with eight. The hardware is expensive. The waste is significant.
