Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

Changelog

GPU temperature and utilization metrics

Date

You can now track three additional GPU metrics in the Metrics tab for your Dedicated Inference deployments:

  • GPU temperature: Temperature in degrees Celsius.

  • GPU tensor core utilization: Tensor core activity as a percentage, on GPUs with profiling support.

  • GPU memory bandwidth utilization: Memory read and write activity as a percentage.

Compare these graphs with request load and latency to investigate performance bottlenecks. Use Customize view in the Metrics tab to show the graphs you need.

For more information, see our docs.