CPT Virtualization Model

CPT ArcGIS Server Virtualization Model 42nd Edition

ArcGIS Virtualization

ArcGIS software deployment on virtual server platforms has become standard for most IT departments. This article provides background on updates to the Esri capacity planning models (introduced at the Arc15CapacityPlanning0101 release) implemented to support planning for ArcGIS for Server deployments in virtual server environments.

2011 Virtual Server Performance
Performance and scalability of ArcGIS for Server capacity planning models in virtualized environments was initially established from Esri joint testing with VMware with results published in an Esri ArcGIS Server 10 for VMware Infrastructure deployment and technical considerations guide in January 2011.

Results from the initial VMware testing showed virtual server machine (VM) hypervisor overhead loads increasing based on the number of VM core. Multiple smaller VMs would provide more throughput than one single high capacity VM with the same number of core.

2015 Capacity Planning Tool virtualization performance model
Esri capacity planning models were developed based on the benchmark test results representing the performance profile shown in Figure C-1.1. Arc15CapacityPlanning0715 CPT results are generated using 2013 platform technology in order to compare results between benchmarks discussed later in this paper.
 * Physical server configuration
 * - Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - 4-core disabled for test purposes (8-core active)


 * Virtual server configurations
 * - Host platform: Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - Four virtual server machine configurations, each with total of 8 core.


 * Performance results
 * - Physical 8-core server provided highest throughput = 55.6
 * - Virtual Server VM peak throughput
 * Eight 1-core VMs = 50.4
 * Four 2-core VMs = 50.4
 * Two 4-core VMs = 46.3
 * One 8-core VM = 42.8
 * - Multiple smaller VMs provided higher peak throughput
 * - Virtual Server VM peak throughput with a 12-core host platform configuration provided the same results as the 8-core host platform.
 * Model was based on hypervisor loads supporting the virtual server machines.
 * Hypervisor load did not leverage the additional host platform core.

2013 Virtual Server Performance
Esri conducted a benchmark test with VMware in 2013 to evaluate technology advances with the new virtualization environments. A new ArcGIS for Server 10.1 and 10.2 VMware vSphere Deployment Guide was published sharing the benchmark results.

Results of the 2013 benchmark showed a different performance profile than the earlier testing. VMware recommended a new test configuration, where extra host platform processor resources were reserved for hypervisor processing overhead. As a result, the virtual server VM processing loads were able to match the physical server processing loads with the same number of core (test results did not show the hypervisor processing overhead).

Figure C-1.2 shows the results of the 2013 benchmarks. The Xeon X5650 platform technology was used in the 2013 testing.
 * Physical server configuration
 * - Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - 4-core disabled for test purposes (8-core active)


 * Virtual server configurations
 * - Host platform: Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - Four virtual server machine configurations, each with total of 8 core.


 * Performance results
 * - Physical 8-core server provided highest throughput = 57.4
 * - Virtual Server VM peak throughput
 * Eight 1-core VMs = 35.6
 * Four 2-core VMs = 43.5
 * Two 4-core VMs = 51.0
 * One 8-core VM = 57.4
 * - Single 8-core VM provided higher peak throughput.
 * 8-core VM matched throughput of 8-core physical server.
 * Hypervisor loads were not reflected in VM loads.
 * - Hypervisor loads were supported by extra 4 host platform core.

2013 ArcGIS for Server Site communication overhead
ArcGIS 10.1 for Server deployment introduced a new GIS Server Site architecture. This new architecture included a new site aware service manager load balancing capability integrated with the ArcGIS for Server site deployment. This new capability simplified ArcGIS for Server site deployment and management, providing an adaptive ArcGIS for Server site capability that shared a common file based configuration store.

Results of the 2013 VMware test identified a performance overhead in supporting multi-machine ArcGIS for Server site configurations due to ArcGIS for Server site communications. The ArcGIS for Server site communications increased with the number of machines in the site, reducing peak transaction throughput from the multiple machine configurations.

2016 Capacity Planning Tool virtualization performance model
The CPT performance model introduced in January 2016 incorporates the updated VMware virtualization load distribution and the ArcGIS for Server site communication overhead loads reflected in the 2013 VMware benchmarks.

Figure C-1.3 shows the results of the 2013 benchmarks generated by the 2016 CPT performance models. The Xeon X5650 platform technology was used in generating the CPT results.


 * Physical server configuration
 * - Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - 4-core disabled for test purposes (8-core active)


 * Virtual server configurations
 * - Host platform: Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - Four virtual server machine configurations, each with total of 8 core.


 * Performance results
 * - Physical 8-core server provided highest throughput = 57.4
 * - Virtual Server VM peak throughput
 * Eight 1-core VMs = 35.6
 * Four 2-core VMs = 43.5
 * Two 4-core VMs = 51.0
 * One 8-core VM = 57.4
 * - Single 8-core VM provided higher peak throughput.
 * 8-core VM matched throughput of 8-core physical server.
 * Hypervisor loads were not measured..
 * - Hypervisor loads were supported by extra 4 host platform core.
 * Multiple VM throughput degradation is due to the GIS Server site internal communication loads.

2013 Virtualization hypervisor processing overhead
The 2013 VMware virtualization benchmark showed ArcGIS for Server performance where an 8-core physical server provided the same throughput as an 8-core virtual server machine (VM), with all the virtualization processing overhead supported by 4 separate host platform processor core.

The virtualization processing overhead was not measured in the VMware benchmark – the results showed only the 8-core VM peak throughput and the 8-core physical peak throughput, and both were the same. The 12-core host platform had 4-core that were not assigned to the 8-core VMs, and hypervisor processing loads were supported by those 4 extra host platform core.

Additional internal testing was completed by Esri staff comparing throughput of an 8-core physical machine with 8-core virtual machine configurations with the same 8-core host platform. The physical machine configuration showed over 50 percent more throughput that the virtual machine configuration, suggesting that the virtualization overhead was roughly 50 percent of the loads generated by the virtual machines. The results of our internal test was consistent with what we saw in the VMware test – the virtualization overhead would not impact VM throughput if we reserve 4 extra core on the host platform for hypervisor processing.

Figure C-1.4 shows the results of an 8-core host platform supporting the 8-core VM configurations. 8-core physical machine was used for the physical throughput benchmark. These results were generated by the new January 2016 CPT performance models. The Xeon X5650 platform technology was used in generating the CPT results.


 * Physical server configuration
 * - Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - 4-core disabled for test purposes (8-core active)


 * Virtual server configurations
 * - Host platform: Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - 4-core disabled for test purposes (8-core active)
 * - Four virtual server machine configurations, each with total of 8 core.


 * Performance results (throughput in transactions per second)
 * - Physical 8-core server provided highest throughput = 57.4
 * - Virtual Server VM peak throughput
 * Eight 1-core VMs = 26.4
 * Four 2-core VMs = 31.1
 * Two 4-core VMs = 35.2
 * One 8-core VM = 38.3 (67% of 57.4)
 * - Single 8-core VM provided higher peak throughput.
 * 8-core VM matched throughput of 8-core physical server.
 * Hypervisor loads are reflected by the VM throughput loss.
 * - 8-core VM peak throughput 33 percent less than 8-core physical server.
 * - Reduced VM throughput was due to virtualization load contention.
 * Multiple VM throughput degradation is due to the GIS Server site internal communication loads.
 * - Single 8-core VM machine provides highest VM throughput.

The one 8-core VM throughput was 38.3, 67 percent of the physical 8-core server. The hypervisor accounted for 33 percent of the host platform processing load. In this case, the hypervisor load is 50 percent of the VM service load (33/67). The 2016 CPT model includes 50 percent of the total VM service load for hypervisor processing on the host platform.

2016 ArcGIS for Server Single Cluster mode


ArcGIS 10.1.2 for Server introduced a new single cluster mode configuration. ArcGIS for Server single-cluster site configuration provides improved throughput and scalability by removing the ArcGIS Server site load balancing capability.
 * ArcGIS for Server service handler disabled (no load balancing).
 * Reduces internal site network traffic.
 * Improves site multiple machine peak throughput (improved scalability).

The ArcGIS Server single cluster mode supports linear scalability within a single ArcGIS for Server site.

Figure C-1.5 shows the results of the ArcGIS Server single cluster mode generated by the 2016 CPT performance models. The Xeon X5650 platform technology was used in generating the CPT results.


 * Physical server configuration
 * - Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - 4-core disabled for test purposes (8-core active)


 * Virtual server configurations
 * - Host platform: Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * 4-core disabled for test purposes (8-core active)
 * - Four virtual server machine configurations, each with total of 8 core.


 * Performance results
 * - Physical 8-core server provided highest throughput = 57.4
 * - Virtual Server VM peak throughput was limited by hypervisor load.
 * Hypervisor load 50 percent of virtual server processing loads.
 * Eight 1-core VMs = 38.3
 * Four 2-core VMs = 38.3
 * Two 4-core VMs = 38.3
 * One 8-core VM = 38.3
 * - All virtual server configurations peak at 67 percent of available processing resources.
 * - Hypervisor and virtual machines shared available host processing resources.

2018 Virtualization Performance Update


Microsoft shared compute benchmark scores for some of their Azure Windows VMs. The Microsoft published Azure VM benchmarks were compared to the published vendor benchmarks for the same physical VM host platform available on SPEC to identify the associated hypervisor overhead loads.

Figure C-1.6 shows the results of the benchmark comparison.

Hypervisor processing loads vary based on the number of virtual machine core (vCPU).
 * 24 percent of service load for 2-core Virtual Machines
 * 32 percent of service load for 4-core Virtual Machines
 * 42 percent of service load for 8-core Virtual Machines
 * 53 percent of service load for 16-core Virtual Machines

Virtual Server peak throughput is reduced by the amount of hypervisor load supporting the virtual configuration.

2018 CPT Virtualization Model Upgrade


The CPT virtualization performance model reduced the hypervisor load from 50 percent to 35 percent of the VM service load based on information provided by the Microsoft Azure published benchmark values. The CPT will use the 35 percent hypervisor load for capacity planning.
 * Most large ArcGIS Enterprise virtual server deployments follow workflow separation best practices, and servers are deployed on multiple 4-core or 2-core virtual machines.
 * Smaller organizations typically support ArcGIS Enterprise base deployments on a single physical machine or 4-core virtual machine environment.
 * Amazon and Azure virtual machines are identified on the CPT Hardware tab, along with specific throughput benchmark values for use in capacity planning. For on-premise environments, the CPT Calculator and Design models apply hypervisor processing loads to the host platform to calculate host platform capacity.

Figure C-1.7 shows results for the ArcGIS Server virtualization performance benchmarks generated by the updated 17CapacityPlanning0301 CPT performance models. The Xeon X5650 platform technology was used in generating the CPT results for comparison purposes.


 * Physical server configuration
 * - Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * - 4-core disabled for test purposes (8-core active)


 * Virtual server configurations
 * - Host platform: Xeon X5650 12 core (2 chip) 2667 MHz platform (SRint2006=26.5/core)
 * 4-core disabled for test purposes (8-core active)
 * - Four virtual server machine configurations, each with total of 8 core.


 * Performance results
 * - Physical 8-core server provided highest throughput = 58.0
 * - Virtual Server VM peak throughput was limited by hypervisor load.
 * Hypervisor load 35 percent of VM service processing loads.
 * Eight 1-core VMs = 42.9
 * Four 2-core VMs = 42.9
 * Two 4-core VMs = 42.9
 * One 8-core VM = 42.9
 * - All virtual server configurations peak at 74 percent of available processing resources.
 * - Hypervisor and virtual machines shared available host processing resources.

Conclusion
The 2016 CPT model includes host platform capacity planning loads for virtualization deployments. 2016 CPT virtualization overhead models are conservative, and consistent with the benchmark results and deployment recommendations provided in the ArcGIS for Server 10.1 and 10.2 VMware Deployment Guide.

The 2018 CPT model updates the host platform capacity planning loads for virtualization deployments. 2018 CPT virtualization overhead models are reasonable, and consistent with the benchmark results and deployment recommendations provided in the ArcGIS Enterprise 10.6 deployment patterns.

"Best Practice: Maintaining peak host platform loads below 95 percent utilization will ensure host platform compute resources are available to support maximum virtual machine service throughput without major hypervisor contention."

Virtualization load profiles can change with new technology releases. New virtualization technology releases should reduce virtualization overhead and improve ArcGIS for Server throughput. CPT models can be adjusted to reflect reduced virtualization overhead when appropriate.

Page Footer Specific license terms for this content System Design Strategies 26th edition - An Esri ® Technical Reference Document • 2009 (final PDF release)