production metrics like gpu-utilization and gpu-memory-utilization

[geraldstanje]

hi,

i looked at the following but dont see metrics like gpu-utilization and gpu-memory-utilization ... why is that?

• https://docs.vllm.ai/en/latest/serving/metrics.html
• https://github.com/vllm-project/vllm/blob/main/vllm/engine/metrics.py

are metrics like gpu-utilization and gpu-memory-utilization not useful for autoscaling or routing to different vllm engines?

Thanks,
Gerald