Software Performance
From Wiki.GIS.com
Spring 2012 GIS Software Performance 31st Edition (Under Construction)
This section shares lessons learned about selecting and building effective GIS design solutions that satisfy operational performance and scalability needs. Software technology allows us to model our work processes, and provide these models to computers to optimize user workflow performance. The complexity of these models, the functions selected to generate our display, and how application functions are orchestrated to analyze and present information processing needs have a significant impact on computer system performance and scalability.
For many years we focused our system architecture design consulting efforts toward identifying and establishing a hardware infrastructure that would support a standard implementation of Esri software technology. We developed platform sizing models based on consulting experience and customer implementation success. We updated our sizing models based on relative performance benchmark testing which focused on quantifying changes in critical processing loads introduced with each new software release.
There were examples of GIS deployments that did not take advantage of system architecture design best practices. Systems were deployed with unresolved performance issues, and scalability was not well understood. In some cases performance issues were not identified before the production system was under critical peak loads, and the platform solution or network infrastructure failed to meet performance needs. Resolving performance issues while in production can be expensive, both in terms of lost services and user productivity. Building a system design that addresses capacity planning needs throughout development and deployment can improve user productivity and reduced implementation risk.
Technology is Changing GIS user Productivity
User workflow performance is changing. ARC/INFO was ported to the Windows operating system in 1997 on the Intel Pentium Pro 200 MHz workstation. GIS user map display refresh time was about 10 seconds; with some geographic analysts telling stories about how this same map used to take up to 70 seconds on a Sun SPARCStation 2 Solaris workstation back in 1992. ArcGIS Desktop was introduced in 1998, taking about 12 seconds to render the same map on an Intel Pentium II 300 MHz workstation (ARC/INFO could produce the same map in half the processing time). By 2000 this same map display was rendered in less than 4 seconds on an Intel Pentium III 500 MHz workstation. Web mapping (ArcIMS) was introduced in CY2000 as a way to share geographic information products throughout the organization and with the public - data center map display rendering time was less than 3 seconds.
Figure 3-1 shows the changes in map display performance over the last 10 years. In 2001, Light ArcIMS map displays rendered in less than 2.5 seconds, while medium complexity maps still took almost 5 seconds. GIS software technology patterns expanded in 2004 through 2010, with medium complexity dynamic map services rendered in less than a second by 2008 along with a growing interest in using preprocessed map products delivered from optimized cached map services. Heavier high quality cached basemaps now provide a rich foundation for displaying dynamic business information products.
GIS display performance has changed, and this provides new opportunities for future GIS information products. If we assume a maximum user productivity of 6 – 10 displays per minute and an average user think time of less than 6 seconds, map display processing times of less than 1 second can satisfy most user productivity needs. With 2008 - 2010 technology, we have achieved this level of performance for the full range of traditional ArcGIS Server map publishing deployment patterns. What does this mean?
User display performance expectations in 2000 were around 5 seconds – a challenge for light map displays viewed in the computer room. The same map service today can be rendered in less than 0.25 seconds. These performance improvements open opportunities for more complex dynamic map services, deployment of ArcGIS Server on easier to manage virtual server environments, deployment of Web services on hosted cloud computing environment, or possibly much richer dynamic services that employ more sophisticated statistical analysis or network routing algorithms (2 to 3 times the complexity of current GIS workflow baselines). These opportunities will introduce new challenges – as heavier processing options are introduced, it will be increasingly important to plan, set performance milestones, and manage compliance during system deployment. At the same time we have more opportunities than ever before to reduce the risk of deploying systems that do not meet our performance needs.
We noticed over the years that our test and tuning efforts were often finding and fixing the same performance issues over and over again, and from these lessons learned we identified best practices for building high performance scalable systems. The ArcGIS Server technology today includes a broad range of functionality, from simple cached map services with very high performance and scalability to heavy geoprocessing services that can take hours for a single request. Developers and administrators need to understand what functions are appropriate, and how to deploy a solution that meets user productivity needs during peak system loads.
This section shares our lessons learned, and identifies functional areas within the software that make a difference. Our capacity planning tools are designed to help consultants and developers set appropriate performance targets based on the selected software technology. The tools also identify how to measure compliance during design and development. These same tools can be used to maintain and support deployed operational system performance needs.
This chapter will share system design best practices and associated performance baselines for the most common GIS software deployment patterns and identify several key parameters that impact system performance and scalability. We will also share how to use the available capacity planning tools for setting appropriate performance targets and validating compliance during design and implementation of the selected technology solution.
Map Display Performance
GIS provides users with a geographic view of their business environment, and for many GIS workflows the map display is used as a primary spatial information resource.
Figure 3-3 shows the processing performance for three different ArcGIS Server dynamic map displays, all deployed on the same platform environment. The performance difference can be traced back to the complexity of the map display document. The light display requires about half the amount of map display processing (slightly under 0.2 second) as the medium display, and the heavy display requires over three times the light display processing (more than 0.5 seconds). The design and complexity of the map display make a significant difference in system performance and scalability. Light maps are rendered three times faster than heavy maps. The first step in publishing a map service is to create a map document (MXD) representing the geographic information product you wish to publish.
The following best practices share lessons learned in designing high performance map documents. The complexity of the authored map document will be a primary factor in determining map service performance and scalability. Online help provides several tips for designing maps for optimal performance.
Quality vs. Speed
Figure 3-4 shows the classic tradeoff between quality and performance.
The high quality map is a shaded relief with transparent layers and dynamic Maplex labeling. These are expensive functions that require extra computer processing for each user display. In contrast, the same display on the right uses low resolution relief, solid colors, and simple annotation providing similar information with good display performance.
Optimizing lines and polygons
We discussed earlier the importance of keeping the display functions simple.
Using Esri Optimized symbols can improve display drawing performance by up to 50 percent.
Measuring display complexity
Display complexity is an estimate of the average amount of computer processing required to produce an average display. For GIS mapping workflows, most of the display rendering time is consumed in creating the dynamic map product. The map rendering time can be estimated/observed during the map authoring process. ArcGIS 9.3.1 release introduced a Map Service Publishing Preview tool that measures optimized (MSD) map display rendering time. Esri consultants use the MXDPerfstat performance statistics tool to generate a performance profile of an MXD map document. Map display rendering time can be used as a relative measure of map display complexity.
Map rendering time is measured in seconds, and the measured result will be a function of the map display complexity and performance of the platform rendering the map.
Platform performance is represented as a horizontal line based on vendor published per core SPEC performance baseline. Display complexity (low, medium, high) is shown as a function of platform performance. Intersection of the horizontal platform performance line with the display complexity curve identifies the measured render time on the horizontal x-axis.
Map Service Publishing Preview Tool
The ArcGIS 9.3.1 release introduced some new map optimization tools incorporated in the Map Service Publishing tool bar.
The Map Service Publishing tools use the new MSD rendering engine to analyze and display the selected ArcGIS Desktop map extent. With the ArcGIS Desktop Map Service Publishing tools, results are provided for the current map scale only - so you would need to move the display to each area you would like to evaluate.
It is important for me to state that the average workflow service times used for capacity planning may not be the same value as a specific map display render time. Some workflow displays will be heavier, while other displays are lighter. User productivity (displays per minute) is also a factor in defining workflow processing loads. The workflow service times and user productivity together should represent the average user loads applied to the system over time. It is best to estimate actual workflow service time loads from platform throughput and CPU utilization measurements collected during real user operations - it is very difficult to accurately simulate or estimate these loads by measuring single map display processing times. This does not preclude setting appropriate workflow performance targets for capacity planning purposes. These performance targets will be directly influenced by the number of map display layers, the features per layer, and the complexity of the functions used in building the display (display complexity).
Measuring map document display performance
Figure 3-8 provides a sample output generated by the MXDPerfstat performance measurement tool.
The MXDperfstat tool identifies display refresh times at multiple scales, shows layer refresh times for each map scale, provides layer performance statistics such as number of features, vectors, labeling, and breaks out display time for several key rendering phases (geography, graphics, cursor, and database). The tool also provides some high level recommendations for performance tuning (actually, once you see the layer processing time and the performance metrics the display problems are quite obvious).
MXDperfstat is an excellent tool for measuring map document display performance, since it lists layer statistics (render time, features, edges, projection, etc) for each layer included within a complete series of map scales. The measured results can be used for evaluating map display performance and tuning your map document for optimum display performance.
ArcGIS Server Optimized Map Service Document (MSD)
ArcGIS 9.3.1 introduced a new optimized map service description (MSD).
There are some functional limitations with publishing with the MSD map document. Some of the nice features of the map publishing analysis report is that identified problems with the display include hyperlinks to online Esri Help providing instructions for resolving errors and optimizing performance.
Measuring map service display performance
Figure 3-10 provides a sample output generated by the PerfHeatMap performance measurement tool.
PerfHeatMap results are displayed on a map grid showing color coded rendering time ranges across the map extent. Display rendering times are generated for each map scale, presenting a spatial overview of display processing loads. These results can be useful for identifying specific “hot” areas within the map extent or estimating display complexity based on specific area work profiles.
Selecting Workflow Display Complexity
Figure 3-11 provides an overview of the CPT Calculator workflow display complexity selection and associated performance metrics.
Workflow ArcGIS Desktop, SOC, SDE, and DBMS services times are adjusted based on the display complexity setting. Web application and browser/terminal client processing times are not impacted by the complexity settings.
Selecting the right Image Resolution
The resolution of the map service output (map size) is an important performance consideration.
The resolution of the map display output can be an important consideration for GIS user workflows. A high resolution display with full screen map size is important for users that work with map displays over long periods throughout the day. A display resolution of 1280 x 1024 would be common for GIS power users or data maintenance (GIS editor) workflows. Rich Internet Applications (Flex and Silverlight) present higher resolution map displays. Remote client display performance on a dedicated T-1 connection would be over 2.3 seconds slower than a local user display. A display resolution of 600 x 400 is a common size for Web browser map displays. Remote display performance over T-1 line is within 0.6 seconds of local clients.
Web mapping services produce map images that are sent to the client browser for display. Each user request will generate a new map image that must be delivered to the client browser. The size of the output image varies directly with the number of pixels, so higher resolution images generate much higher client traffic loads. The required amount of traffic per display can have a significant impact on user performance over lower bandwidth. Server processing loads may also increase with higher resolution displays (higher resolution can result in larger map extent increasing the number of features rendered for each client display).
Selecting the right image output format
Web mapping services produce map images that are sent to the client browser for display. Each user request will generate a new map image that must be delivered to the client browser. The selected image type can have a direct impact on the volume of network traffic. Lighter images require less display traffic and heavier images require more display traffic. The required amount of traffic per display can have a significant impact on user performance over lower bandwidth.
Figure 3-14 identifies the amount of data required to support common ArcGIS Server output image types (JPEG, PNG24, PNG8, PDF). The volume of data identified in megabits per display (Mbpd) required to support the same map varies with each image type.
JPEG image types provide the most consistent compression, with a slight variation between raster and vector images. PNG images do much better with vector data than with raster - PNG supports transparencies and is the default ArcGIS Server format. PDF is a heavier output format used for high quality map plotting.
Figure 3-15 provides an overview of the CPT Calculator Density (Vector Only, Raster Image) and Output (Default, JPEG, PNG8, PNG24, PNG32, PDF, Feature) selections.
GIS Dynamic Map Display Processing
GIS displays are normally created one layer at a time, similar to the procedure geographers would follow to layout a map display on a table using Mylar sheets.
Figure 3-16 shows the software processing for building a map display, one layer at a time, joining the features (points, polygons, lines) in each layer sequentially one on top of the other until the final display is complete.
In theory, the second approach generates the display faster. The same amount of processing is required for both methods - in fact the parallel approach requires additional procedures for establishing the parallel display request and then bringing those results back together to produce the final map display.
Hardware vendors are providing computers with an increasing number of processor core per chip, expanding the capacity of the server platforms with reducing expectations for increased performance gains per processor core. Vendors have encourages software developers to take advantage of this increased server capacity by increasing the number of concurrent processes used in generating each user display. Most heavy processing workloads today require sequential processing and a single display generation will not take advantage of multiple processor core. The actual standard map display performance gains for reprogramming software to take advantage of parallel processing may not be worth the extra effort and additional processing loads.
Figure 3-18 compares a traditional COTS map display performance with the performance of a parallel query map display, where the display layers are blended together on the client browser.
The parallel implementation is supported by three ArcGIS Server REST API map services mashed together in a JavaScript API client browser application. Client access was over a 1.5 Mbps DSL Internet connection, requiring over 1 sec to deliver each 200 KB map image over the network connection to the client browser display. The extra network transport time and queue time to support the parallel display build consumed most of the parallel processing display performance gain. Parallel processing may not always improve system performance, and in some cases could reduce overall system performance and scalability.
ArcGIS Server map cache: the Performance Edge
ArcGIS Server technology provides a variety of enhanced solutions to improve production system performance and scalability. The concept of pre-processing map outputs for customer delivery is not new - we used to do this with paper map books before we had computers. The ArcGIS Server 9.3 release introduces a variety of ways to maintain and support pre-processed maps, and to organize the map files in a map cache structure optimized for map publishing.
Figure 3-19 shows the display time for both light and medium complexity dynamic map products in comparison to the display time for a fully cached map.
The optimum Web mapping display combines dynamic map services presented as a transparent image [mashup (business layers)] over a map cache base layer. Dynamic map services are important for geographic analysis, editing, and geoprocessing functions which require access to point, polygon, and line features rendered in a dynamic map display. Map cache tiles provide an optimum base map foundation layer, combining high quality map visualization with high display performance. A mashup of dynamic operational layers over high quality base map reference layer delivers the optimum combination of quality and performance.
Figure 3-20 shows how the Capacity Planning Calculator is designed to encourage cached map services.
The CPT Calculator includes an option to add a map cache data source to the selected dynamic workflow, providing appropriate composite service loads to complete a system design. The "+mapcache" data source selection applies an addition 0.5 Mbpd traffic load for Calculator sizing purposes (this load is included in the display traffic displayed on the Workflow tab). The client traffic identified in cell E4 will turn pink to highlight the composite traffic load setting and +$$ will be added to the Calculator workflow name.
The ArcGIS Server map cache service traffic can vary greatly depending on the workflow activity. Map tiles are downloaded to the client browser Internet cache and subsequent request for the same tile are delivered from local client cache (not over the network). A "well behaved" client (one who works in a small geographic area) will soon have all the map tiles located on the local machine cache, and minimum map tile traffic will be required from the central site over the network. On the other hand, a "world traveler" scenario (each map view from a different world location) can generate very high display traffic over the network connection (several map tiles required for each map display). The average 0.5 Mbpd traffic used by the CPT Calculator is based on average worldwide requests from ArcGIS.com map cache service statistics.
Data Source Performance Parameters
Figure 3-21 compares display performance between using an ArcSDE Geodatabase data source and the available File data sources.
The small Shape File performance is about the same as the ArcSDE Geodatabase. The large Shape File format requires three times the processing required with ArcSDE. The small File Geodatabase loads are about 80 percent of the ArcSDE Geodatabase performance values, while the large File Geodatabase provides performance similar to ArcSDE loads.
The performance test series generated common displays for 100 random maps within the neighborhood area identified in the map above. The ArcSDE geodatabase display performance was roughly the same whether using the full San Diego extent or the small neighborhood area as the data source.
The small Shape File testing used vector layers extracted for the neighborhood extent identified above. The large Shape File data set included the complete San Diego area (the Capacity Planning Tool large shape file load would represent about twice the full San Diego data extent).
Display performance with the File Geodatabase is much improved over the Shape File format (test results are not shown). Tests have been completed accessing a File Geodatabase with up to 1 TB of data, and performance was quite good. The small File Geodatabase performance targets would likely support a data source representing the full San Diego extent. The large File Geodatabase performance targets would data sources up to 1 TB in size.
It is difficult to be precise when evaluating these performance targets. There are many factors in the database design, number of features per layer, and number of layers in the display that can factor into these performance values. Processing loads on the file data server platform are very light (all query processing loads for a file data source are performed by the client application).
Figure 3-23 provides some additional information on the number of display layers, total number of features, and the size of the San Diego data source used during our initial performance validation testing. The same raster layer was used for both tests. Network throughput for the small Shape File data source was half the traffic of the medium Shape File data source.
Figure 3-24 shows the different in display performance for the two Shape File data source.
The large Shape File performance targets included in the Capacity Planning Tool represent 3 times the processing of the small Shape file. These performance targets should be adequate to support twice the extent identified in the test results shown above (9 vector layers with up to 2 million features or possibly twice the number of vector layers at the same extent).
Figure 3.25 identifies the data source selections (SDE DBMS, Small File GDB, Large File GDB, Small Shape file, Large Shape File, Image) available for the CPT Calculator.
Capacity Planning Workflow Recipe
The capacity planning workflow nomenclature was first introduced in the GIS Software Technology section. The workflow name generated by the Capacity Planning Calculator tool provides a recipe for generating the selected workflow performance targets. The selected software technology pattern and key performance parameters selected during GIS planning are documented in the workflow recipe.
The workflow recipe identifies the selected software release version (930, 931, 10, etc) followed by the deployment pattern (Wkstn, WTS Citrix, ADF, SOAP, REST, WMS, Image, etc). The recipe follows with the map document (MXD or MSD), complexity (Lite, ML, Med, MH, Hvy, 2xMed, 3xMed), %Dynamic (1-%DataCache), resolution (4x3, 6x4, 8x6, 10x7, 12x10, 16x12), Density (V or R) and output (JPEG, PNG8, PNG24, PNG32, PDF, ICA, Feature). Calculator generated workflow service times and traffic are displayed on the CPT Workflow tab along with the workflow recipe. The Calculator recipe is the Standard Esri Workflow names provided on the CPT Workflow tab. The CPT Calculator workflow recipe identifies the assumptions made in creating performance targets for use in the CPT Design.
Performance Testing Tools
A variety of performance monitor and testing tools are available for systems administration. Standard tools used for Esri benchmark testing include Microsoft Fiddler for network traffic measurements and Visual Studio for Esri Web applications. Windows performance monitor is used for collecting server performance metrics. The Enterprise GIS Resource Center provides information on performance measurement tools used for Esri benchmark testing.
Software performance summary
Experience suggests we can do a better job selecting and building better software solutions. Understanding software performance can reduce implementation risk and save customer time and money. Projects can be delivered within project cost, time, and performance budgets.
CPT Video: Software Performance
The next section will take a closer look at ArcGIS Server software performance.
Previous Editions
Software Performance 30th Edition
Software Performance 29th Edition
Software Performance 28th Edition
Software Performance 27th Edition
Page Footer
Specific license terms for this content
System Design Strategies 26th edition - An Esri ® Technical Reference Document • 2009 (final PDF release)
del.icio.us
facebook
reddit
twitter
