System Design Process 33rd Edition

Fall 2013 System Design Process 33rd Edition

The purpose of this document is to share a system design methodology that promotes successful deployment of geographic information system (GIS) technology. This system design methodology includes guidelines for identifying business requirements, making appropriate software selection, using properly configured data sources, and providing sufficient hardware to meet user productivity needs. This document focuses on system performance and scalability - building a GIS that will perform during peak operational loads.

Much more can be said about business requirements analysis (GIS User Needs) and available software functionality - this is not the focus of this documentation. Dr Roger Tomlinson (Father of GIS) provides an excellent book that shares a proven framework for comprehensive GIS planning called Thinking about GIS. Understanding the information products you want out of the GIS and identifying the software candidates you might use to produce these information products is a prerequisite for completing your system architecture design. 

What Is System Architecture Design?
System architecture design is a process developed by Esri to promote successful GIS enterprise operations. This process builds on your existing information technology (IT) infrastructure and provides specific recommendations for hardware and network solutions based on existing and projected business (user) needs.

The system architecture design process aligns identified business requirements (user needs) derived from business strategy, goals, and drivers (business processes) with recommended business information systems infrastructure technology (network and platform) recommendations. The computer display transaction is the work unit used to translate business requirements to associated server and network loads. Display service times (software processing) generates platform loads and display traffic generates network loads. Peak throughput loads (business needs) are used to generate platform capacity and network bandwidth requirements.

System design starts with identifying business needs. This includes identifying user locations and required information products, identifying required data resources, and developing appropriate software applications to do the work. Business needs are represented by project workflows that identify the traffic and processing associated with each display transaction.

System architecture design translates business needs to identified IT requirements. Hardware requirements are generated based on peak software processing loads. Network connectivity requirements are generated based on peak data flow. Capacity Planning tools are provided to automate the design analysis.

"Warning: Trying to build a GIS without completing a proper system architecture design can lead to system deployment failure."



Why Is System Architecture Design Important?
System performance is limited by the weakest link in the system design. System architecture design identifies the weak links during the planning process and promotes investment in a balanced system design.

A distributed computer environment must be designed properly to support user performance (productivity) requirements. The weakest "link" in the system will limit performance. The system architecture design process identifies specifications for a balanced hardware solution. Investment in hardware and network components based on a balanced system load model provides the highest possible system performance at the lowest overall cost as represented by the chain in Figure 1.2.

Building a high performance GIS requires more than getting the hardware right. User workflows must be designed to optimize client productivity (simple maps) and efficiently manage heavier geoprocessing loads (service request queue). The geodatabase design and data source selection should be optimized to address system performance and scalability requirements. The selected production platform components (servers, workstations, storage) must have the capacity to handle peak user workflow processing loads within an acceptable service response time. The system architecture design must address performance needs and bandwidth constraints over distributed communication networks—technology and solution architecture must be selected to conserve shared infrastructure resources. System architecture design can provide a solid foundation for building a productive operational environment.

Workflow complexity determines processing and data flow loads that must be handled by the computing infrastructure. The computing architecture must be selected with the appropriate capacity to service the required business loads. Workflow complexity is a measure of the amount of processing loads and network traffic required to refresh the user display. Complexity is imposed on the computing architecture by the following design attributes:


 * Database design and data format: DBMS, geodatabase, and ArcSDE
 * User workflow software design: Application development

The computing infrastructure must provide sufficient capacity to handle peak operational loads.


 * Server platform processor core and deployment architecture must handle peak processing loads.
 * Network bandwidth and remote site connectivity must be adequate to avoid traffic contention.
 * Storage access performance and capacity must be adequate to provide required data access.

Server performance, network capacity, and efficient storage strategies can improve user productivity and reduce system cost.

The system architecture design process can be used to identify specifications for a balanced hardware solution. Investment in hardware and network components based on a balanced system load model provides the highest possible system performance at the lowest overall cost. 

Why we do planning
The primary reasons for planning include identifying business needs, defining project requirements, and reducing implementation risk. In practical terms, we need a plan if we hope to get something done. The plan provides a foundation for successful implementation. Figure 1.3 identifies some of the primary planning objectives.

Business requirements may include a variety of user workflows and Web services that would improve business operations – these workflows and services must be identified early during the planning process. The workflow and service definitions establish a foundation for the rest of the planning.

The peak user workflow and service processing requirements must be identified and translated to appropriate system design loads – server processing times and network traffic. These processing loads provide a foundation for generating appropriate hardware specifications (performance and capacity requirements) and network infrastructure needs.

A project must be defined to procure and implement the required technology. The project will include a budget and schedule along with a variety of project performance milestones. Project approval is required before any major design and/or implementation efforts begin. Project approval is based on identified or perceived benefits that will accrue to the business resulting from the proposed level of time (schedule) and financial (budget) investment.

A good plan can be used to reduce system implementation risk. Delivering a system that will satisfy identified business requirements is fundamental to project success. Understanding key system performance parameters, identifying incremental system performance targets, and establishing a system performance validation plan can chart a solid framework for managing implementation risk.

Planning is an incremental process driven by rapidly changing technology. The most effective plans are designed to adapt to changing business needs and evolving technology opportunities. The established change control process must be agile and effective to ensure project tasks are managed to deliver within established schedule and budget constraints.

What questions are we trying to answer?
There are many questions about system architecture design that we answer on a daily basis. The System Design Strategies documentation deployed on this wiki site is developed and maintained to answer most of these questions, and provide a reference to help customers better understand their design needs.

Here is a short list of some of the more common questions:


 * How many users can I support with my existing hardware?
 * What hardware do I need to purchase?
 * How many servers (cores) do I need?
 * What are the software licensing requirements?
 * What workflow loads should I use for my existing applications?
 * What are my current workflow service times?
 * What is the capacity of my current system?

This documentation, along with the companion Capacity Planning Tools, was developed so you can answer these questions about your GIS, and many more. We have been supporting GIS customers for over 20 years, and much of what we have learned is shared so you can benefit and build a successful GIS. 

What demands does GIS place on the computing infrastructure?
GIS applications place heavy demands on server processing and network bandwidth resources. For most standard mapping operations, these demands push the limits of platform processor (CPU) and network bandwidth (I/O) capacity.

Several key infrastructure components work together to service business processing loads.


 * Network I/O: Network traffic demands on available bandwidth
 * Processing: Platform processor core processing demands
 * Memory: Platform physical memory utilization
 * Graphics: Video graphics card loads
 * Disk I/O: Storage disk access performance

Different workflows place different loads on the system. Relative processing demands for the four use patterns above show how these loads are distributed differently, based on the workflow.
 * Data query and analysis is network I/O intensive.
 * Analysis and processing is compute (processing) intensive.
 * Fly-through and 3D animation can be graphics intensive.
 * Data loading and conversion can be disk I/O intensive.

GIS operations are typically network and/or processing intensive—these are the subsystems that require the closest attention. GIS applications place heavy demands on server processing and network bandwidth resources. For most standard mapping operations, these demands push the limits of platform processor and network bandwidth capacity.

Additional critical design components include the following:


 * Memory is critical if you do not have enough; memory requirements can normally be satisfied by following standard configuration guidelines. Platform memory guidelines are provided in Windows Memory Management appendix.
 * Some GIS desktop applications are graphics intensive (3D animation); video graphics are less critical for GIS than for the video gaming industry, so there is plenty of graphics technology available at reasonable cost to meet GIS needs.
 * Disk I/O can be a concern if you are moving large volumes of data in or out of storage, such as with imagery or map cache generation (normally a background administrative process).

Any one of these components, if configured incorrectly, can become a bottleneck and limit system performance.

System architecture design is about building a balanced solution that satisfies peak performance needs at the lowest possible cost. Each of these critical subsystems should be considered in establishing the right system design. 

Cost of a change
Getting it right from the start is best done by taking the time to understand the technology, quantify user requirements, select the right software technology, and deploy the right hardware. Not getting it right from the start will cost money to make it right later. The cost of change increases exponentially as implementation proceeds as shown in Figure 1.5.

Your GIS plan should include performance validation early in the design process. ArcGIS for Desktop authors should have performance targets they verify when publishing a new map service. GIS programmers should have performance targets they build to when developing a new application. Performance should be a consideration for data administrators, network administrators, and selected storage subsystems. Getting the design right from the start can save money and improve overall business operations. 

What is the System Design Process?
The system design process includes a GIS needs assessment and a system architecture design. The system architecture design is based on user workflow requirements identified in the GIS needs assessment.

Traditional system design process
GIS Needs Assessment: The GIS needs assessment includes a review of user workflow requirements and identifies where GIS technology can improve the quality and productivity of the business process flow. This assessment identifies GIS application and data requirements and an implementation strategy for supporting GIS user needs. The user organization must be actively involved throughout the user needs assessment. A GIS solutions architect familiar with current GIS technology patterns and customer business practices can help facilitate this planning effort.

System Architecture Design: The system architecture design is based on user requirements identified by the GIS needs assessment. Customers must have a clear understanding of their GIS application and data requirements before they are ready to complete their system architecture design. An Enterprise system design consultant familiar with GIS software patterns and system design analysis is typically used to complete the final system design and analysis. Technology purchase decisions should be delayed until after completing the system architecture design analysis.



The system design begins with a technology exchange. The technology exchange provides a foundation for client support throughout the design process. Client participation is a key ingredient in the design process. The design process includes a review of the existing computer environment, GIS user requirements, and system design alternatives. System design capacity planning tools can be used to translate projected peak user workflow requirements to specific platform specifications. An integrated implementation strategy can then be developed to support GIS deployment milestones.

"Warning: Problems can occur when software technology selection is made based on a GIS needs assessment without considering system architecture design implications."

Integrated Business Needs Assessment
Traditionally, the user needs assessment and the system architecture design were two separate efforts. There are some key advantages in completing these efforts together. Figure 1.6 shows the project time line relationship between the traditional user’s needs assessment and the system architecture design process.


 * The GIS software technology selection process should include a discussion of architecture options and available system deployment alternatives.
 * The existing hardware environment, peak user workflow operations, and user location information required for a system design should be identified during the user needs workflow interviews.
 * Software technology selection should consider hardware configuration options, required platform selection, peak system loads for each configuration option, and overall system costs. User productivity may be a primary consideration in making the right software technology choice.
 * And finally the system implementation schedule must consider hardware and application delivery milestones.

A primary goal for developing capacity planning tools is to automate the system architecture design analysis in such a way that GIS solution architects can use the tools to complete a user needs assessment and a system architecture design within an integrated business needs assessment. 

Building a GIS: Implementation strategy
There are several critical deployment stages that support a successful implementation. Understanding the importance of each stage and the key objectives for success leads to more effective enterprise implementations. Figure 1.7 shows a series of typical system deployment stages for building and maintaining successful enterprise GIS operations.

 Requirements phase
 * User needs establish a foundation for completing the design.
 * User location and peak business loads establish a foundation for system architecture design.

Design phase
 * Infrastructure upgrade requirements must be identified to identify deployment costs.
 * Network communication capacity is an important consideration for GIS deployments.
 * Hardware and software procurement requirements must be identified.
 * Software development requirements and data acquisition needs must be identified.

"Best Practice: Business decisions for project funding and procurement authorization are often required for project effort to proceed beyond this phase."

Construction phase
 * System procurement authorization, based on the design budget and deployment timeline.
 * Data acquisition and database design efforts begin.
 * Procurement authorization for application design and development.
 * Prototype testing plans completed and scheduled to validate product delivery within design performance targets.

Implementation phase "Best Practice: Deployment process is repeated incrementally on a periodic schedule to leverage technology change."
 * Initial deployment and operational testing.
 * Final system delivery, user training, and workflow migration complete.
 * System maintenance operations.

The Capacity Planning Tools (CPT) were developed as a framework to promote successful GIS system design and implementation. CPT functions contribute throughout the implementation cycle. The CPT tasks include a review of business needs, establish performance targets, identify user locations, review network suitability, select product architecture, and complete the system architecture design analysis. Additional tools are provided to validate system performance during the design, construction, and implementation phases. CPT models can be easily updated to reflect changes in business requirements and review alternative system deployment strategies, providing an adaptive model for addressing a variety of incremental planning activities.

"Best Practice: Build and maintain a simple system performance model that links user requirements with system design." 

System design strategies overview
Enterprise GIS design includes a broad range of technology that must play together to satisfy identified business needs. The better they work together, the more productive your business can be.

System design topics will be discussed throughout the SDSwiki documentation.


 * Software technology: Chapter 2
 * Software performance: Chapters 3 and 4
 * Data administration: Chapter 5
 * Network communications: Chapter 6
 * Platform architecture: Chapter 7
 * Platform performance: Chapter 8
 * Information security: Chapter 9
 * Performance Management: Chapter 10
 * System Implementation: Chapter 11

All of the SDSwiki content will be brought together in a system architecture design case study for the fictional City of Rome. The City of Rome will bring all the chapters together in a realistic enterprise design.


 * City of Rome case study: Chapter 12

System architecture design terminology
There is a great deal of terminology used within the System Design Strategies wiki that might be new to many readers. System architecture design crosses a variety of disciplines, including business operations, IT operations, and software development. The terminology introduced in this SDSwiki works to build a common vocabulary for communications that bring these groups together—establishing a framework for discussing and addressing system performance problems.

New technology introduces new terminology, and the language evolves with each new idea. Often a word is assigned to express a particular concept or explain a particular role a component plays in the overall design—and often common words are assigned and used to describe new concepts. It is important to take the time to understand how the word is being used within the concept that is being discussed.

The words used in the System Design Strategies wiki documentation share what is being assigned by software development, IT, and the business operations groups the system design is bringing together. Take time to listen to the words, and the context in which they are being used. These words provide a framework for discussions on system architecture design.

The [Acronyms and Glossary] describes many of the key words used within the system design strategies documentation.

System architecture design process
The enterprise GIS system design process aligns identified business requirements (user needs) derived from business strategy, goals, and drivers (business processes) with recommended business information systems infrastructure technology (network and platform) recommendations.
 * 1)  User needs assessment (results of a GIS user needs assessment provides inputs for the system architecture design analysis)
 * 2)  Workflow loads analysis (translate user needs to project workflows with baseline traffic and processing transaction loads based on estimated workflow complexity)
 * 3)  Technical architecture strategy (identify user locations, network connectivity, and data center server locations)
 * 4)  User requirements analysis (translate peak user workflow loads to peak throughput transaction loads)
 * 5)  Network suitability analysis (translate peak site/network throughput loads to peak site/network traffic and compare with available network bandwidth)
 * 6)  Platform architecture selection (Identify data center platform tier configuration and identify platform selection for each tier)
 * 7)  Software configuration (Identify platform assignment for each workflow software component peak transaction processing load)
 * 8)  Enterprise design solution (combine all peak workflow software component processing loads on the assigned platform tier, translate baseline processing load to selected platform processing load, and generate number of nodes required for each platform tier with estimate of capacity utilization)

The Esri Capacity Planning Tool (CPT) was developed to simplify business requirements data collection and automate system design analysis tasks associated with each phase of the development cycle. This section will introduce the available capacity planning tools and their function in completing a system architecture design. Hyperlinks are provided to associated CPT demos located in the Capacity Planning Tool appendix.

The CPT can be used to identify your system design requirements and model performance and scalability of your enterprise design. The CPT provides a framework for integrating business, data, applications, and technical architecture needs required to design, deploy, and manage successful enterprise GIS operations. 

Pre design efforts


Figure 1.10 shows how you can prepare for your system architecture design. Business needs must be understood before you are ready to complete the system architecture design. Business requirements analysis includes a review of the enterprise vision, the existing business architecture, and the user workflow requirements. Each of these areas must be explored in some detail before you begin the design.

Figure 1.11 identifies the CPT tabs for documenting findings from your pre-design efforts.

Review Business Needs
Business needs (user requirements) establish a foundation for building a proper system design. Business requirements are represented by user workflows. The first step in completing a system architecture design is to select the appropriate project workflows. GIS Software Technology chapter provides an overview of the primary GIS workflow patterns along with best practices for making a proper technology decision. Software Performance chapter identifies how you can generate project workflows with appropriate transaction loads to represent your business requirements.


 * CPT Workflow loads analysis

The Project Workflows section on the CPT Workflow tab is located at the top of the workflow list. Project Workflows are selected from the Standard Workflows included on the CPT Workflow tab or custom workflows generated from the CPT Calculator or Test tabs - identifying your project workflows is the process referred to as your user workflow loads analysis. Once your software technology (project workflow) selection is complete, your selected workflow performance targets will be available at the top of the CPT workflow selection list for use in your system architecture design.

System Design Process
Once you have identified your project workflows, you are ready to complete your system design. The CPT is developed for use based on a standard system architecture design process as shown in Figure 1.12. Each cycle of the system architecture design process includes the following steps:


 * Technical architecture strategy—High-level overview showing user site locations, network bandwidth connections, and central data center locations. User location information is collected during the user needs analysis.
 * Workflow loads analysis—CPT Requirements analysis section is configured to represent the site locations, user workflows, peak loads, and network bandwidth for the enterprise design solution.
 * Network suitability analysis—CPT Design completes the network suitability analysis and identifies any communication bottlenecks. Network bandwidth upgrades are identified to complete the network suitability analysis.
 * Platform architecture selection—CPT Design Platform tier is configured to represent the design solution. Identify platform tier nicknames, select platforms, and identify platform rollover settings.
 * Software configuration—CPT Design Software Configuration module is used to assign workflow software to supporting platform tier (software install) and make workflow data source selection.
 * Enterprise design solution—Once configured, the CPT Design tab completes the system architecture design analysis and provides the platform solution.



Figure 1.13 shows the steps to complete the system design. System design process includes the user requirements analysis, network suitability analysis, platform architecture selection, software configuration (installation), and the Enterprise design solution. The CPT was designed to complete the analysis. Once the CPT is properly configured and business requirements are defined, the CPT will complete the system architecture design analysis and display the hardware solution.


 * CPT Design tab: User requirements analysis

Once you identify your business workflows, you are ready to complete a user system loads analysis. The CPT Design tab includes a Requirements Analysis module where you can identify user locations and peak throughput loads.


 * CPT Design tab: Network suitability analysis

Once you configure the CPT Design tab to reflect your user requirements and identify the network connections, the CPT completes a network suitability analysis. Network suitability analysis is completed by the CPT Design tab. The CPT analysis evaluates network bandwidth and latency to ensure adequate capacity to accommodate peak traffic flow loads. You can then increase network bandwidth as required to accommodate peak traffic loads.


 * CPT Design tab: Platform architecture selection

Platform selection is a critical decision in any design process. The selected platform directly contributes to user display performance, platform capacity, and software licensing cost. A faster processor core improves user performance and reduces license cost. Higher capacity servers reduce the total number of server machines required to satisfy business requirements.

"Warning: Platform selection directly impacts overall system cost."


 * CPT Design tab: Software configuration

"Note: CPT Design software configuration will be discussed in more detail in Lesson 7: GIS product architecture."


 * CPT Design tab: Enterprise system design

Once you make your platform selections and install workflow software components (software configuration), the CPT will complete the platform sizing and show the required server configuration (number of platforms and peak utilization levels for each platform tier).

The Design tab provides a snapshot view of your user requirements and associated platform solution as a single integrated information product.

"Note: Chapter 12 provides a case study for the City of Rome, demonstrating how to use the CPT to complete an enterprise system architecture design."


 * CPT Hardware tab: Platform performance

The CPT Hardware tab is used to list SPEC performance benchmark values for use in completing the system architecture design.

Monitor performance compliance
The capacity planning tool (CPT) can be used by project managers to establish specific project performance milestones and measure compliance. Figure 1.14 shows some key opportunities for measuring performance compliance.


 * Measuring display rendering time when authoring a service is the first opportunity to validate performance.
 * Measuring deployed service rendering time is another opportunity to validate performance.

"Best Practice: Monitoring progress in meeting performance milestones can reduce deployment risk and ensure project delivery success."

When performance issues are identified early in deployment, proper adjustments can be made before impacting production workflow productivity. Simpler map displays, simpler data models, and cached data layers are proven ways to reduce workflow complexity. Warnings identified by the map publishing analysis tool identify how to reduce display complexity.

"Best Practice: Identifying and resolving performance issues before they become production level performance problems will promote deployment success."

Figure 1.15 shows some tools that were developed to translate system performance measurements to workflow service times for use in performance validation. The CPT Test tab was designed to help with the validation process, and includes several tools that can be used to translate performance measurements to equivalent workflow service times.


 * Performance validation (CPT Test tab)

Measuring performance during the system build and deployment reduces implementation risk. System design is only the beginning of the process; then you need to build a system that performs within the system architecture design performance targets. If measured performance loads exceed planned budgets, adjustments can be made to workflow complexity during the build process to deliver services within initial performance targets.

Platform capacity calculations
Platform capacity for a standard map display can be calculated for any selected platform. Platform capacity varies based on selected platform throughput performance and published map display complexity.
 * 2013 Baseline System Specifications
 * Arc13 4-core platform performance SPEC baseline = 200 (SPECfp_rate2006 Baseline)
 * ArcGIS 10.2 for Server REST medium complexity workflow
 * Baseline 4-core medium complexity throughput capacity = 70,000 TPH


 * Variations in platform performance and workflow display complexity drives throughput
 * Platform SPEC baseline = 400, medium complexity, throughput = 140,000 TPH
 * Platform SPEC baseline = 100, medium complexity, throughput = 35,000 TPH
 * Platform SPEC baseline = 200, light complexity, throughput = 140,000 TPH
 * Platform SPEC baseline = 200, heavy complexity, throughput = 47,000 TPH

This section ends by introducing a simple tool used for proper hardware platform selection. The Platform Capacity Calculator identifies the single server throughput capacity of the selection platform configuration.


 * Platform Capacity Calculator

The Platform Capacity Calculator provides a peak throughput range for selected workflows, showing medium complexity output in blue and light complexity output in red. This provides a simple tool to help customers select the appropriate platform configuration.

CPT Calculator tab provides a much more accurate platform capacity planning assessment for a single workflow configuration, taking into account software performance variables and hardware configuration strategies.

CPT Design tab provides a much more adaptive enterprise capacity planning assessment integrating system loads for multiple workflows supporting multiple network local and remote sites and a variety of platform configuration strategies.

Concluding remarks
Proper GIS planning is the most important investment any organization can make in building a GIS. Understanding your GIS needs, selecting the right technology at the right time, and establishing documented implementation milestones to measure your progress can ensure your success. This document is focused on sharing how to build and maintain successful GIS operations. The Capacity Planning Tools provide a framework for collecting what you know about your business needs and your system environment. The Capacity Planning Tool models connect what you understand about GIS user requirements with the network and platform loads your IT support teams can measure in the data center.

CPT Video: System Design Process
The next Chapter will review the most common GIS technology patterns and share best practices in making the right technology selection.

Previous Editions
System Design Process 32nd Edition System Design Process 31st Edition System Design Process 30th Edition System Design Process 29th Edition System Design Process 28th Edition System Design Process 27th Edition

Page Footer Specific license terms for this content System Design Strategies 26th edition - An Esri ® Technical Reference  Document • 2009 (final PDF release)