System Design Process

From Wiki.GIS.com

Jump to:navigation, search
System Design Strategies (select here for table of contents)
1. System Design Process 2. GIS Software Technology 3. Software Performance 4. Server Software Performance
5. GIS Data Administration 6. Network Communications 7. GIS Product Architecture 8. Platform Performance
9. Information Security 10. Performance Management 11. System Implementation 12. City of Rome
A1. Capacity Planning Tool B1. Windows Memory Management Preface (Executive Summary) SDSwiki What's New



Spring 2014 System Design Process 34th Edition

The purpose of this document is to share a system design methodology that promotes successful deployment of geographic information system (GIS) technology. This system design methodology includes guidelines for identifying business requirements, making appropriate software selection, using properly configured data sources, and providing sufficient hardware to meet user productivity needs. This document focuses on system performance and scalability - building a GIS that will perform during peak operational loads.

Much more can be said about business requirements analysis (GIS User Needs) and available software functionality - this is not the focus of this documentation. Dr Roger Tomlinson (Father of GIS) provides an excellent book that shares a proven framework for comprehensive GIS planning called Thinking about GIS. Understanding the information products you want out of the GIS and identifying the software candidates you might use to produce these information products is a prerequisite for completing your system architecture design.

Contents

What Is System Architecture Design?

Figure 1.1 System architecture design is an analytic process that generates IT hardware and network capacity requirements from specified peak business workflow performance needs.
System architecture design is a process developed by Esri to promote successful GIS enterprise operations. This process builds on your existing information technology (IT) infrastructure and provides specific recommendations for hardware and network solutions based on existing and projected business (user) needs.

The system architecture design process aligns identified business requirements (user needs) derived from business strategy, goals, and drivers (business processes) with identified business information systems infrastructure technology (network and platform) recommendations. The computer display transaction is the work unit used to translate business requirements to associated server and network loads. Display service times (software processing) generates platform loads and display traffic generates network loads. Peak throughput loads (business needs) are used to generate platform capacity and network bandwidth requirements.

System design starts with identifying business needs. This includes identifying user locations and required information products, identifying required data resources, and developing appropriate software applications to do the work. Business needs are represented by project workflows that identify the traffic and processing associated with each display transaction.

System architecture design translates business needs to identified IT requirements. Hardware requirements are generated based on peak software processing loads. Network connectivity requirements are generated based on peak data flow. Capacity Planning tools are provided to automate the design analysis. Capacity Planning Tools make the process of aligning Business workflows with selected IT resources agile and iterative in nature, rapidly identifying system performance impacts in response to changing business and technology architecture patterns.

Warning: Trying to build a GIS without completing a proper system architecture design can lead to system deployment failure.


Why Is System Architecture Design Important?

Figure 1.2 System performance chain. Overall system performance is limited by the weakest link in the performance chain.
System performance is limited by the weakest link in the system design. System architecture design identifies the weak links during the planning process and promotes investment in a balanced system design.

A distributed computer environment must be designed properly to support user performance (productivity) requirements. The weakest "link" in the system will limit performance. The system architecture design process identifies specifications for a balanced hardware solution. Investment in hardware and network components based on a balanced system load model provides the highest possible system performance at the lowest overall cost as represented by the chain in Figure 1.2.

Building a high performance GIS requires more than getting the hardware right. User workflows must be designed to optimize client productivity (simple maps) and efficiently manage heavier geoprocessing loads (service request queue). The geodatabase design and data source selection should be optimized to address system performance and scalability requirements. The selected production platform components (servers, workstations, storage) must have the capacity to handle peak user workflow processing loads within an acceptable service response time. The system architecture design must address performance needs and bandwidth constraints over distributed communication networks—technology and solution architecture must be selected to conserve shared infrastructure resources. System architecture design can provide a solid foundation for building a productive operational environment.

Workflow complexity determines processing and data flow loads that must be handled by the computing infrastructure. The computing architecture must be selected with the appropriate capacity to service the required business loads. Workflow complexity is a measure of the amount of processing loads and network traffic required to refresh the user display. Complexity is imposed on the computing architecture by the following design attributes:

The computing infrastructure must provide sufficient capacity to handle peak operational loads.

Server performance, network capacity, and efficient storage strategies can improve user productivity and reduce system cost.

The system architecture design process can be used to identify specifications for a balanced hardware solution. Investment in hardware and network components based on a balanced system load model provides the highest possible system performance at the lowest overall cost.

Why we do planning

Figure 1.3 Planning is required to quantify business requirements.

The primary reasons for planning include identifying business needs, defining project requirements, and reducing implementation risk. In practical terms, we need a plan if we hope to get something done. The plan provides a foundation for successful implementation. Figure 1.3 identifies some of the primary planning objectives.

Business requirements may include a variety of user workflows and Web services that would improve business operations – these workflows and services must be identified early during the planning process. The workflow and service definitions establish a foundation for the rest of the planning.

The peak user workflow and service processing requirements must be identified and translated to appropriate system design loads – server processing times and network traffic. These processing loads provide a foundation for generating appropriate hardware specifications (performance and capacity requirements) and network infrastructure needs.

A project must be defined to procure and implement the required technology. The project will include a budget and schedule along with a variety of project performance milestones. Project approval is required before any major design and/or implementation efforts begin. Project approval is based on identified or perceived benefits that will accrue to the business resulting from the proposed level of time (schedule) and financial (budget) investment.

A good plan can be used to reduce system implementation risk. Delivering a system that will satisfy identified business requirements is fundamental to project success. Understanding key system performance parameters, identifying incremental system performance targets, and establishing a system performance validation plan can chart a solid framework for managing implementation risk.

Planning is an incremental process driven by rapidly changing technology. The most effective plans are designed to adapt to changing business needs and evolving technology opportunities. The established change control process must be agile and effective to ensure project tasks are managed to deliver within established schedule and budget constraints.

What questions are we trying to answer?

There are many questions about system architecture design that we answer on a daily basis. The System Design Strategies documentation deployed on this wiki site is developed and maintained to answer most of these questions, and provide a reference to help customers better understand their design needs.

Here is a short list of some of the more common questions:

This documentation, along with the companion Capacity Planning Tools, was developed so you can answer these questions about your GIS, and many more. We have been supporting GIS customers for over 20 years, and much of what we have learned is shared so you can benefit and build a successful GIS.

What demands does GIS place on the computing infrastructure?

Figure 1.4 These charts show relative performance demands on the basic infrastructure components, and how these performance demands vary with four very different processing scenarios.

GIS applications place heavy demands on server processing and network bandwidth resources. For most standard mapping operations, these demands push the limits of platform processor (CPU) and network bandwidth (I/O) capacity.

Several key infrastructure components work together to service business processing loads.

Different workflows place different loads on the system. Relative processing demands for the four use patterns above show how these loads are distributed differently, based on the workflow.

GIS operations are typically network and/or processing intensive—these are the subsystems that require the closest attention. GIS applications place heavy demands on server processing and network bandwidth resources. For most standard mapping operations, these demands push the limits of platform processor and network bandwidth capacity.

Additional critical design components include the following:

Any one of these components, if configured incorrectly, can become a bottleneck and limit system performance.

System architecture design is about building a balanced solution that satisfies peak performance needs at the lowest possible cost. Each of these critical subsystems should be considered in establishing the right system design.

Cost of a change

Figure 1.5 The cost of making a change increases rapidly as you purchase and apply GIS implementation resources.
Getting it right from the start is best done by taking the time to understand the technology, quantify user requirements, select the right software technology, and deploy the right hardware. Not getting it right from the start will cost money to make it right later. The cost of change increases exponentially as implementation proceeds as shown in Figure 1.5.

Your GIS plan should include performance validation early in the design process. ArcGIS for Desktop authors should have performance targets they verify when publishing a new map service. GIS programmers should have performance targets they build to when developing a new application. Performance should be a consideration for data administrators, network administrators, and selected storage subsystems. Getting the design right from the start can save money and improve overall business operations.

What is the System Design Process?

The system design process includes a GIS needs assessment and a system architecture design. The system architecture design is based on user workflow requirements identified in the GIS needs assessment.

Traditional system design process

GIS Needs Assessment: The GIS needs assessment includes a review of user workflow requirements and identifies where GIS technology can improve the quality and productivity of the business process flow. This assessment identifies GIS application and data requirements and an implementation strategy for supporting GIS user needs. The user organization must be actively involved throughout the user needs assessment. A GIS solutions architect familiar with current GIS technology patterns and customer business practices can help facilitate this planning effort.

System Architecture Design: The system architecture design is based on user requirements identified by the GIS needs assessment. Customers must have a clear understanding of their GIS application and data requirements before they are ready to complete their system architecture design. An Enterprise Architect familiar with GIS software patterns and system design analysis is typically used to complete the final system design and analysis. Technology purchase decisions should be delayed until after completing the system architecture design analysis.

Figure 1.6 Integrated business needs assessment promotes proper and timely business system design decisions.

The system design begins with a technology exchange. The technology exchange provides a foundation for client support throughout the design process. Client participation is a key ingredient in the design process. The design process includes a review of the existing computer environment, GIS user requirements, and system design alternatives. System design capacity planning tools can be used to translate projected peak user workflow requirements to specific platform specifications. An integrated implementation strategy can then be developed to support GIS deployment milestones.

Warning: Problems can occur when software technology selection is made based on a GIS needs assessment without considering system architecture design implications.

Integrated Business Needs Assessment

Traditionally, the user needs assessment and the system architecture design were two separate efforts. There are some key advantages in completing these efforts together. Figure 1.6 shows the project time line relationship between the traditional user’s needs assessment and the system architecture design process.

A primary goal behind developing capacity planning tools is to automate the system architecture design analysis in such a way that GIS solution architects can use these tools to complete a user needs assessment and a system architecture design within an integrated business needs assessment. An integrated business needs assessment promotes an adaptive discovery effort, enabling optimum alignment between evolving business and technology architecture patterns.


Building a GIS: Implementation strategy

Figure 1.7 System design implementation strategy moves through several steps that require planning and implementation management.
There are several critical deployment stages that support a successful implementation. Understanding the importance of each stage and the key objectives for success leads to more effective enterprise implementations. Figure 1.7 shows a series of typical system deployment stages for building and maintaining successful enterprise GIS operations.

Requirements phase

Design phase

Best Practice: Business decisions for project funding and procurement authorization are often required for project effort to proceed beyond this phase.

Construction phase

Implementation phase

Best Practice: Deployment process is repeated incrementally on a periodic schedule to leverage technology change.

The Capacity Planning Tools (CPT) were developed as a framework to promote successful GIS system design and implementation. CPT functions contribute throughout the implementation cycle. The CPT tasks include a review of business needs, establish performance targets, identify user locations, review network suitability, select product architecture, and complete the system architecture design analysis. Additional tools are provided to validate system performance during the design, construction, and implementation phases. CPT models can be easily updated to reflect changes in business requirements and review alternative system deployment strategies, providing an adaptive model for addressing a variety of incremental planning activities.

Best Practice: Build and maintain a simple system performance model that links user requirements with system design.


System design strategies overview

Figure 1.8 System design strategies overview showing the principal chapters that will be covered in the SDSwiki documentation and how they interface in building an enterprise design solution.
Enterprise GIS design includes a broad range of technology that must play together to satisfy identified business needs. The better they work together, the more productive your business can be.

System design topics will be discussed throughout the SDSwiki documentation.

All of the SDSwiki content will be brought together in a system architecture design case study for the fictional City of Rome. The City of Rome will bring all the chapters together in a realistic enterprise design.


System architecture design terminology

Figure 1.9 System architecture design brings together a broad range of terminology used for addressing Enterprise system design, performance, and capacity planning.
There is a great deal of terminology used within the System Design Strategies wiki that might be new to many readers. System architecture design crosses a variety of disciplines, including business operations, IT operations, and software development. The terminology introduced in this SDSwiki works to build a common vocabulary for communications that bring these groups together—establishing a framework for discussing and addressing system performance problems.

New technology introduces new terminology, and the language evolves with each new idea. Often a word is assigned to express a particular concept or explain a particular role a component plays in the overall design—and often common words are assigned and used to describe new concepts. It is important to take the time to understand how the word is being used within the concept that is being discussed.

The words used in the System Design Strategies wiki documentation share what is being assigned by software development, IT, and the business operations groups the system design is bringing together. Take time to listen to the words, and the context in which they are being used. These words provide a framework for discussions on system architecture design.

The [Acronyms and Glossary] describes many of the key words used within the system design strategies documentation.

System architecture design process

The enterprise GIS system design process aligns identified business requirements (user needs) derived from business strategy, goals, and drivers (business processes) with recommended business information systems infrastructure technology (network and platform) recommendations.

  1. User needs assessment (results of a GIS user needs assessment provides inputs for the system architecture design analysis)
  2. Workflow loads analysis (translate user needs to project workflows with baseline traffic and processing transaction loads based on estimated workflow complexity)
  3. Technical architecture strategy (identify user locations, network connectivity, and data center server locations)
  4. User requirements analysis (translate peak user workflow loads to peak throughput transaction loads)
  5. Network suitability analysis (translate peak site/network throughput loads to peak site/network traffic and compare with available network bandwidth)
  6. Platform architecture selection (Identify data center platform tier configuration and identify platform selection for each tier)
  7. Software configuration (Identify platform assignment for each workflow software component peak transaction processing load)
  8. Enterprise design solution (combine all peak workflow software component processing loads on the assigned platform tier, translate baseline processing load to selected platform processing load, and generate number of nodes required for each platform tier with estimate of capacity utilization)

The Esri Capacity Planning Tool (CPT) was developed to simplify business requirements data collection and automate system design analysis tasks associated with each phase of the development cycle. This section will introduce the available capacity planning tools and their function in completing a system architecture design. Hyperlinks are provided to associated CPT demos located in the Capacity Planning Tool appendix.

The CPT can be used to identify your system design requirements and model performance and scalability of your enterprise design. The CPT provides a framework for integrating business, data, applications, and technical architecture needs required to design, deploy, and manage successful enterprise GIS operations.

Pre design efforts

Figure 1.10 Business needs establish the foundation for any Enterprise GIS design. The Enterprise vision, existing business architecture, and user requirements must be understood to select the best GIS solution.

Figure 1.10 shows how you can prepare for your system architecture design. Business needs must be understood before you are ready to complete the system architecture design. Business requirements analysis includes a review of the enterprise vision, the existing business architecture, and the user workflow requirements. Each of these areas must be explored in some detail before you begin the design.

The GIS needs assessment begins with the organization identifying where GIS technology can improve the quality and productivity of the business process flow. This assessment identifies GIS application and data requirements and an implementation strategy for supporting GIS user needs. The user organization must be actively involved throughout the user needs assessment. A GIS solutions architect familiar with current GIS technology patterns and customer business practices can help facilitate this planning effort; the real work must be done by the organization themselves.

Review the Strategic Business Plan

Figure 1.11.1 Planning begins with the organization, the strategic business plan, the management accountability framework, and the business process flows that drive business decisions.
Figure 1-11.1 emphasizes the importance of understanding the business. The real benefits from GIS will be derived from business process improvements.

It all starts with the Organization

Review statements:

Examine business model proposed to achieve goals

Gain a clear sense of direction for GIS planning

Business operations analysis (business process workflows)

Success depends on Knowing what you want to get out of it.

Current state (without GIS)

Represents the current business baseline.

Future state (with GIS)

Information Product Description shows how GIS can make a difference.

Best Practice: Select information products that empower your business operations.

Information products drive your business solution

Figure 1-11.2 GIS data, applications, and system requirements are derived from well defined business information products.
Figure 1-11.2 shows how the design solution is driven by information products that improve business operations. GIS Benefits to the organization come from the information products. Data, applications, and the system infrastructure are costs that must be paid for by benefits realized from the information products.

Planning begins by thinking about what you want out of the system.

Best Practice: Information product benefits determine GIS deployment ROI.



Review Business Needs

Figure 1.11.3 CPT Calculator and Workflow tabs were developed for addressing pre-design planning efforts.

Figure 1.11.3 identifies the CPT tabs for documenting findings from your pre-design efforts.

Business needs (user requirements) establish a foundation for building a proper system design. Business requirements are represented by user workflows. The first step in completing a system architecture design is to select the appropriate project workflows. GIS Software Technology chapter provides an overview of the primary GIS workflow patterns along with best practices for making a proper technology decision. Software Performance chapter identifies how you can generate project workflows with appropriate transaction loads to represent your business requirements.

CPT Workflow loads analysis

The Project Workflows section on the CPT Workflow tab is located at the top of the workflow list. Project Workflows are selected from the Standard Workflows included on the CPT Workflow tab or custom workflows generated from the CPT Calculator or Test tabs - identifying your project workflows is the process referred to as your user workflow loads analysis. Once your software technology (project workflow) selection is complete, your selected workflow performance targets will be available at the top of the CPT workflow selection list for use in your system architecture design.

System Design Process

Figure 1.12 System Architecture Design process provides a logical step by step methodology for using the CPT to complete your System Architecture Design.
Once you have identified your project workflows, you are ready to complete your system design. The CPT is developed for use based on a standard system architecture design process as shown in Figure 1.12. Each cycle of the system architecture design process includes the following steps:
Figure 1.13 The CPT Design tab includes modules to address a variety of functions used during the Enterprise System Architecture Design process.

Figure 1.13 shows the steps to complete the system design. System design process includes the user requirements analysis, network suitability analysis, platform architecture selection, software configuration (installation), and the Enterprise design solution. The CPT was designed to complete the analysis. Once the CPT is properly configured and business requirements are defined, the CPT will complete the system architecture design analysis and display the hardware solution.

CPT Design tab: User requirements analysis

Once you identify your business workflows, you are ready to complete a user system loads analysis. The CPT Design tab includes a Requirements Analysis module where you can identify user locations and peak throughput loads.

CPT Design tab: Network suitability analysis

Once you configure the CPT Design tab to reflect your user requirements and identify the network connections, the CPT completes a network suitability analysis. Network suitability analysis is completed by the CPT Design tab. The CPT analysis evaluates network bandwidth and latency to ensure adequate capacity to accommodate peak traffic flow loads. You can then increase network bandwidth as required to accommodate peak traffic loads.

CPT Design tab: Platform architecture selection

Platform selection is a critical decision in any design process. The selected platform directly contributes to user display performance, platform capacity, and software licensing cost. A faster processor core improves user performance and reduces license cost. Higher capacity servers reduce the total number of server machines required to satisfy business requirements.

Warning: Platform selection directly impacts overall system cost.
CPT Design tab: Software configuration
Note: CPT Design software configuration will be discussed in more detail in Lesson 7: GIS product architecture.
CPT Design tab: Enterprise system design

Once you make your platform selections and install workflow software components (software configuration), the CPT will complete the platform sizing and show the required server configuration (number of platforms and peak utilization levels for each platform tier).

The Design tab provides a snapshot view of your user requirements and associated platform solution as a single integrated information product.

Note: Chapter 12 provides a case study for the City of Rome, demonstrating how to use the CPT to complete an enterprise system architecture design.
CPT Hardware tab: Platform performance

The CPT Hardware tab is used to list SPEC performance benchmark values for use in completing the system architecture design.

Monitor performance compliance

Figure 1.14 Monitoring performance compliance throughout development (map publishing), initial deployment, and in production can reduce implementation risk.
The capacity planning tool (CPT) can be used by project managers to establish specific project performance milestones and measure compliance. Figure 1.14 shows some key opportunities for measuring performance compliance.
Best Practice: Monitoring progress in meeting performance milestones can reduce deployment risk and ensure project delivery success.

When performance issues are identified early in deployment, proper adjustments can be made before impacting production workflow productivity. Simpler map displays, simpler data models, and cached data layers are proven ways to reduce workflow complexity. Warnings identified by the map publishing analysis tool identify how to reduce display complexity.

Best Practice: Identifying and resolving performance issues before they become production level performance problems will promote deployment success.
Figure 1.15 The CPT includes additional tools for performance validation and platform sizing.
Figure 1.15 shows some tools that were developed to translate system performance measurements to workflow service times for use in performance validation. The CPT Test tab was designed to help with the validation process, and includes several tools that can be used to translate performance measurements to equivalent workflow service times.
Performance validation (CPT Test tab)

Measuring performance during the system build and deployment reduces implementation risk. System design is only the beginning of the process; then you need to build a system that performs within the system architecture design performance targets. If measured performance loads exceed planned budgets, adjustments can be made to workflow complexity during the build process to deliver services within initial performance targets.

Platform capacity calculations

Platform capacity for a standard map display can be calculated for any selected platform. Platform capacity varies based on selected platform throughput performance and published map display complexity. Vendor published throughput benchmarks (SPECint_rate2006 Baseline) available on the SPEC Web site shows relative platform throughput for available server platform configurations.

  • Arc13 4-core platform performance SPEC baseline = 200 (SPECint_rate2006 Baseline)
  • ArcGIS 10.2 for Server REST medium complexity workflow
  • Baseline 4-core medium complexity throughput capacity = 70,000 TPH
  • Platform SPECint_rate2006 baseline = 400, medium complexity, throughput = 140,000 TPH
  • Platform SPECint_rate2006 baseline = 100, medium complexity, throughput = 35,000 TPH
  • Platform SPECint_rate2006 baseline = 200, light complexity, throughput = 140,000 TPH
  • Platform SPECint_rate2006 baseline = 200, heavy complexity, throughput = 47,000 TPH

This section ends by introducing a simple tool used for proper hardware platform selection. The Platform Capacity Calculator identifies the single server throughput capacity of the selection platform configuration.

Platform Capacity Calculator

The Platform Capacity Calculator provides a peak throughput range for selected workflows, showing medium complexity output in blue and light complexity output in red. This provides a simple tool to help customers select the appropriate platform configuration.

CPT Calculator tab provides a much more accurate platform capacity planning assessment for a single workflow configuration, taking into account software performance variables and hardware configuration strategies.

CPT Design tab provides a much more adaptive enterprise capacity planning assessment integrating system loads for multiple workflows supporting multiple network local and remote sites and a variety of platform configuration strategies.

Concluding remarks

Proper GIS planning is the most important investment any organization can make in building a GIS. Understanding your GIS needs, selecting the right technology at the right time, and establishing documented implementation milestones to measure your progress can ensure your success. This document is focused on sharing how to build and maintain successful GIS operations. The Capacity Planning Tools provide a framework for collecting what you know about your business needs and your system environment. The Capacity Planning Tool models connect what you understand about GIS user requirements with the network and platform loads your IT support teams can measure in the data center.

CPT Capacity Planning videos

The next Chapter will review the most common GIS technology patterns and share best practices in making the right technology selection.

Previous Editions

System Design Process 33rd Edition
System Design Process 32nd Edition
System Design Process 31st Edition
System Design Process 30th Edition
System Design Process 29th Edition
System Design Process 28th Edition
System Design Process 27th Edition

System Design Strategies (select here for table of contents)
1. System Design Process 2. GIS Software Technology 3. Software Performance 4. Server Software Performance
5. GIS Data Administration 6. Network Communications 7. GIS Product Architecture 8. Platform Performance
9. Information Security 10. Performance Management 11. System Implementation 12. City of Rome
A1. Capacity Planning Tool B1. Windows Memory Management Preface (Executive Summary) SDSwiki What's New


Page Footer
Specific license terms for this content
System Design Strategies 26th edition - An Esri ® Technical Reference Document • 2009 (final PDF release)

Navigation
Need Help
Toolbox
Share This Page