Storage IOPS update with Storage Spaces Direct

Hello, Claus here again. I played one of my best rounds of golf in a while at the beautiful TPC Snoqualmie Ridge yesterday. While golf is about how low can you go, I want to give an update on how high can you go with Storage Spaces Direct.
Once again, Dan and I used a 16-node rig attached to a 32 port Cisco 3132 switch. Each node was equipped with the following hardware:

2x Xeon E5-2699v4 2.3Ghz (22c44t)
4x 800GB Intel P3700 NVMe (PCIe 3.0 x4)
1x LSI 9300 8i
20x 1.2TB Intel S3610 SATA SSD
1x Chelsio 40GbE iWARP T580-CR (Dual Port 40Gb PCIe 3.0 x8)

Using VMFleet we stood up 44 virtual machines per node, for a total of 704 virtual machines. Each virtual machine was configured with 1vCPU. We then used VMFleet to run DISKSPD in each of the virtual machines with 1 thread, 4KiB random read with 32 outstanding IO.

As you can see from the above screenshot, we were able to hit ~5M IOPS in aggregate into the virtual machines. This is ~7,000 IOPS per virtual machine!
We are not done yet…. If you are attending Microsoft Ignite, please stop by my session “BRK3088 Discover Storage Spaces Direct, the ultimate software-defined storage for Hyper-V” and say hello.
Let us know what you think.
Dan & Claus