Analyzing Disk Latency in Azure Virtual Machines - Part 2

Analyzing Disk Latency in Azure Virtual Machines – Part 2

Luke Campbell Azure February 14, 2024 | 0

“I thought the cloud was supposed to be fast. Why are things moving like a sloth?” Have you heard someone complain about the speed of an application after moving it to the Microsoft Azure? Azure is a good environment, but things are not always fast in the cloud. To illustrate, in this post, we’ll review an Azure VM experiencing high disk latency and determine if the bottleneck is due to the VM size or the disk size. This is part 2 of Analyzing Disk Latency in Azure Virtual Machines.

The specs for the VM we’re using are:

Standard B4ms (4 vcpus, 16 GiB memory) – Supports up to 2880 IOPs or 35 MBps.
Target disk – Standard SSD E10 – 500 IOPs or 60 MBps. This drive is capable of bursting to 600 IOPs or 150 MBps.

We’ll use Microsoft’s diskspd utility to generate the load using the 8kb block size.

Using Azure VM Metrics, you can review numerous counters related to the VM and create alerts based on a set threshold. We’re focused on the following counters within the Virtual Machine Host namespace. Three separate charts are used. FYI, filters can be added if you’re only interested in a specific disk. Identify the disk LUN and then add the LUN filter to the chart.

Disk Throughput / Sec – Write and Read

Data Disk Write Bytes/sec
Data Disk Read Bytes/sec

Disk Operations / Sec (IOPs) – Write and Read

Data Disk Write Operations/sec
Data Disk Read Operations/sec

Disk Latency

Data Disk Latency (Preview)

The screenshot below shows each one of the counters. The time range has been set to the last 30 minutes, which refreshes every 5 minutes.

In the example above, we’ve maxed out on read IOPs, leading to severe latency (above 1 second). By maxing out on IOPs, we’re also limiting in how much data can be read from the disk. At 8kb blocks * 524 IOPs, the disk maxes out at 4.1 MB/s. It’s important to note that if your application reads or writes in larger block sizes it is possible to increase throughput at the cost of lowering IOPs.

Conclusion

The example above represents a bottleneck encountered at the disk vs. the VM size. We’re nowhere near the VM’s limit of 2,880 IOPs or 35 MBps. Azure VM Metrics is a fantastic way to review performance counters directly from within the Azure portal.

For more information

If you’d like some assistance assessing your SQL Server workload on an Azure Virtual Machine, reach out. We’re happy to help.

In the meantime, here are some other posts you may find interesting.

Database Administration Database Development IT Manager Microsoft Azure Sero Sero Group Serogroup Shared Disks SQL SQL Assessment SQL Events SQL Server SQL Server Consultant SQL Server Management SQL Training TempDB The Sero Group

Analyzing Disk Latency in Azure Virtual Machines – Part 2