GIS Product Architecture (CPT Calculator Demos) 43rd Edition

Arc18CapacityPlanning0901 release

Figure A1-9a.1 shows an overview of the ArcGIS Enterprise technical architecture. GIS Product Architecture provides a foundation for understanding the software components and platform configuration options available for distributed GIS operations. Understanding application architecture alternatives and associated configuration strategies provides a foundation for selecting an appropriate distributed GIS design.

This section will show how to configure the CPT Calculator tab for the available ArcGIS Desktop and ArcGIS Server hardware configurations.

ArcGIS Desktop workflows
 * Solutions include workstation (Enterprise GDB, File GDB, and Feature service data sources), virtual application sessions in physical and Virtual Server configurations, high availability (HA) virtual application sessions, and virtual desktop infrastructure configurations (ArcMap and ArcGIS Pro).

ArcGIS Server workflows 
 * Solutions include single-tier, two-tier, three-tier physical configurations, three-tier Virtual Server configurations, and three-tier HA Virtual Server configurations.

ArcGIS Desktop: CPT Calculator configurations
This section shows how to configure the following CPT Calculator ArcGIS Desktop workflow patterns.
 * Workstation (Enterprise GDB, File GDB, and Feature service data source)
 * Virtual Application Sessions on Physical server.
 * Virtual Application Sessions on Virtual Servers.
 * High availability Virtual Application Sessions on Virtual Servers.
 * ArcMap and ArcGIS Pro Virtual Desktop Infrastructure.

CPT Calculator supports the following workflow configurations. 
 * Direct Connect to Enterprise Geodatabase platforms.
 * Direct Connect to supported tabular DBMS platforms.
 * Network connection to a selected File data source.
 * Web connection to an ArcGIS Server feature service.

CPT Calculator ArcGIS Desktop workstation database connect workflows
ArcGIS Desktop direct connect to an Enterprise Geodatabase or supported tabular DBMS platforms.
 * Wkstn software selection in cell E3:F3
 * ArcMap or Pro selection in cell G3.
 * DB DBMS data source selection in cell J6:K6.
 * Minimum selection in cell A10.

Direct connect architecture platform solution output in rows 16–29. 
 * Database server and utilization in rows 26-29.
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator ArcGIS Desktop workstation with file data source
Figure A1-9a.3 shows platform solution for workstation direct connect to a selected file data source.
 * Wkstn software selection in cell E3:F3.
 * ArcMap or Pro selection in cell G3.
 * File data source selection in cell J6:K6.
 * Minimum selection in cell A10.

File source architecture platform solution output in rows 16–29. 
 * File server utilization is not shown (very light processing load).
 * File server capacity determined by traffic loads (NIC card).
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator ArcGIS Desktop with feature service data source
Figure A1-9a.4 shows ArcGIS Desktop connection to a registered Portal feature services.
 * Wkstn FSvc software selection in cell E3:F3.
 * ArcGIS Desktop selection in cell G3.
 * VPortal selection in I3.
 * DB DBMS data source selection in cell J6:K6.
 * 3 tier selection in cell A9.
 * Minimum selection in cell A10.

Architecture platform solution output in rows 16–29. 
 * Portal and Web Server in rows 18-21.
 * GIS Server load in rows 22-25.
 * Data Store load in rows 26-29.
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator ArcGIS Desktop feature service access with cached basemap tiles
Figure A1-9a.5 shows ArcGIS Desktop connection to ArcGIS Server feature service workflow with local cached basemap files. This workflow can be used with raster basemaps and with vector tile maps. This workflow assumes a percentage of the map display is provided by pre-cached tiles, and these tiles are downloaded to the local client for mashup with additional dynamic feature service layers. Client, network, and server processing loads are reduced based on the percentage of display that is cached.

Creating a tile cache workflow performance target. 
 * Select Software technology performance factors for AGDwkstnFSvc workflow (Cell E3:F3) based on display complexity.
 * Select percentage of display (%DataCache) supported by cached basemap tiles (Cell F6).
 * Select +mapcache (cell K5) to account for tile download traffic. This assumes basemap tiles are downloaded once, and follow-on requests for the same tiles based on map extent will use local cached tiles.
 * Tiles are cached in local client internet browser cache. Local browser file cache can be deleted to refresh basemap tiles.

CPT Calculator ArcGIS Desktop Pro feature service with client feature caching
Figure A1-9a.6 shows ArcGIS Pro with client feature cache enabled connecting to a registered ArcGIS Server feature service.

Creating an ArcGIS Desktop Web GIS workflow with client feature caching

 * Select Software technology performance factors for AGD wkstn$ FSvc (Cell E3:F3) workflow based on display complexity.
 * Select percentage of features (%DataCache) cached on the client workstation (Cell F6).
 * Cached features are downloaded once.
 * follow-on requests for cached features use local feature cache.
 * no network or server query loads when accessing local cached features.
 * Feature edits will clear local feature cache (feature cache refreshed from service queries).
 * Percentage of features in local cache (%DataCache) will vary based on the client workflow.

How caching works

 * ArcGIS Pro uses a cache to optimize performance when working in previously visited extents.
 * When working with web feature layers there are only two options for caching: Clear the cache when the session ends, or Do not cache.
 * Branch versioning. When the web feature layer has been published and the version management service (VMS) capability is enabled the edit application manages the cache. In this scenario the option to use the cache and clear it when the session ends is enabled by default and cannot be changed.
 * When editing in a named branch version, the cache cannot be cleared if edits have been made but not saved or discarded. Refreshing the version also clears the cache in the case where edits have occurred on the server but are not yet reflected in the current ArcGIS Pro session.



CPT Calculator ArcMap Virtual Application Sessions with physical servers
Figure A1-9a.7 shows the CPT Calculator configuration for an ArcGIS Desktop ArcMap physical server virtual application session direct Connect to Enterprise Geodatabase or supported tabular DBMS platforms'''.
 * AGD Citrix Software selection in cell E3:F3.
 * ArcMap selection in cell G3.
 * Data source selection in cell J6:K6.
 * 2 tier selection in cell A9.
 * Minimum selection in cell A10.

Direct connect architecture platform solution output in rows 16–29. 
 * Physical application servers and utilization in rows 22-24.
 * Physical database server utilization in rows 26-29.
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator ArcMap Virtual Application Sessions with Virtual Servers
Figure A1-9a.8 shows ArcGIS Desktop ArcMap virtual server virtual application sessions with direct connect to an Enterprise Geodatabase or supported tabular DBMS platforms'''.
 * Citrix Software selection in cell E3:F3 (same as above).
 * ArcMap selection in G3.
 * Data source selection in cell J6:K6 (same as above).
 * 2 tier selection in cell A9 (same as above).
 * Minimum selection in cell A10 (same as above).

Direct connect architecture platform solution output in rows 16–29. 
 * Select Vserver in cell K25.
 * Select Virtual Server vCPU configuration in cell K24.
 * Physical database server configuration in rows 26-29 (same as above).
 * Host platform servers and utilization in rows 34-35.
 * Minimum host platform selection in cell A33.
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator High Availability ArcMap Virtual Application Sessions with Virtual Servers
Figure A1-9a.9 shows ArcGIS Desktop ArcMap virtual application session with direct connect to Enterprise Geodatabase or supported tabular DBMS platforms in a high availability Virtual Server configuration.
 * Citrix Software selection in cell E3:F3 (same as above).
 * ArcMap selection in G3.
 * Data source selection in cell J6:K6 (same as above)
 * 2 tier selection in cell A9 (same as above).
 * High Avail selection in cell A10.

Direct connect architecture platform solution output in rows 16–29. 
 * Select Vserver in cell K25 (same as above).
 * Select Virtual Server vCPU configuration in cell K24 (same as above).
 * Physical database server configuration in rows 26-29 (same as above).
 * Host platform servers and utilization in rows 34-35 (same as above).
 * High Avail host platform selection in cell A33.
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator ArcMap Virtual Desktop Infrastructure (VDI)
Figure A1-9a.10 shows an ArcGIS Desktop ArcMap Virtual Desktop Infrastructure deployment with direct connect to Enterprise Geodatabase or supported tabular DBMS platforms.
 * Citrix Software selection in cell E3:F3 (same as above).
 * ArcMap selection in cell G3.
 * Data source selection in cell J6:K6 (same as above).
 * 2 tier selection in cell A9 (same as above).
 * Minimum selection in cell A10.

Direct connect architecture platform solution output in rows 16–29. 
 * Select VDI in cell K25.
 * Select virtual desktop vCPU configuration in cell K24.
 * Physical database server configuration in rows 26-29 (same as above).
 * Host platform servers and utilization in rows 34-35 (same as above).
 * High Avail host platform selection in cell A33 (same as above).
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator ArcGIS Pro Virtual Desktop Infrastructure (VDI)
Figure A1-9a.11 shows an ArcGIS Desktop Pro Virtual Desktop Infrastructure deployment with direct connect to Enterprise Geodatabase or supported tabular DBMS platforms.
 * Citrix Software selection in cell E3:F3 (same as above).
 * ArcPro selection in cell G3.
 * Data source selection in cell J6:K6.
 * Select ArcGIS Pro VDI platform in cell D10 (Esri recommends 28 core server).
 * 2 tier selection in cell A9 (same as above).
 * Minimum selection in cell A10.

Direct connect architecture platform solution output in rows 16–29. 
 * Physical database server configuration shown in rows 26-29.
 * Select VDI in cell K25.
 * Select virtual desktop vCPU configuration in cell K24 (6 core/node recommended for ArcGIS Pro).
 * Enter number of virtual desktop clients in cell D25 (limited by NVIDIA GRID card capacity).
 * Cell T25 identifies required host platform nodes (based on NVIDIA GRID card capacity).
 * Enter required host platform nodes in cell D36 (based on NVIDIA GRID card capacity).
 * Cell T2 identifies maximum number of ArcGIS Pro background jobs supported by the selected configuration.
 * Host platform servers and utilization in rows 34-35.
 * Workflow performance summary provides relative display performance between user locations.

CPT Calculator ArcGIS Pro Virtual Desktop Infrastructure (VDI) with concurrent batch jobs
Figure A1-9a.12 shows an ArcGIS Desktop Pro Virtual Desktop Infrastructure deployment with direct connect to Enterprise Geodatabase or supported tabular DBMS platforms.

Platform configuration without max concurrent background jobs shown above. 
 * Enter 80 jobs in cell T3 (generates batch loads on VDI tier).
 * Host server utilization shown in cell H33.

CPT Calculator ArcGIS Server platform configurations
The CPT Calculator can be configured for each of the three ArcGIS Server architecture patterns.
 * CPT Calculator single-tier configurations (minimum, high availability, Virtual Server)
 * CPT Calculator two-tier configurations (minimum, high availability, Virtual Server)
 * CPT Calculator three-tier configurations (minimum, high availability, Virtual Server)

CPT Calculator single-tier configuration
Figure A1-9a.13 shows a standard ArcGIS Server REST heavy web mapping workflow in a single-tier platform configuration. Peak throughput is 100,000 TPH. Service configuration includes a direct connect DBMS data source connection.

The CPT Calculator platform architecture configuration is set from a drop-down menu in cell A9. Choices include single, two-tier, and three-tier architecture selections. The single-tier architecture selection supports the Web Adaptor, GIS Server, and DBMS software on a single server platform tier.

Once the CPT Design is properly configured to represent your business requirements and selected hardware, the Workflow Performance Summary shows the software service time distribution and expected client display response time for each workflow and the platform section shows the number of required platform nodes and processing loads on the selected platform tier. 

CPT Calculator two-tier configuration
Figure A-9a.14 shows a standard ArcGIS Server REST heavy web mapping workflow in a two-tier platform configuration. Service configuration includes a direct connect DBMS data source connection.

"Note: If two DBMS platforms are required to support peak throughput loads, a higher capacity platform (scale up) or active-active DBMS architecture (special configuration) will be required to support the peak loads. Multiple DBMS platforms can be configured by specifying the number of platform nodes for the DBMS tier in column D." 

CPT Calculator three-tier configuration
Figure A1-9a.15 shows a standard ArcGIS Server REST medium web mapping workflow in a three-tier platform configuration. Service configuration includes a direct connect DBMS data source connection. 

CPT Calculator three-tier Virtual Server configuration
Figure A1-9a.16 shows a standard ArcGIS Server REST heavy web mapping workflow in a three-tier Virtual Server platform configuration. Service configuration includes a direct connect DBMS data source connection. Host platform tier is shown as a bottom tier below the Virtual Server platform tier. All virtual and host platform selections must be the same platform configuration, since Virtual Servers are deployed on the Host platform tier. <br style="clear: both" />

CPT Calculator three-tier high availability Virtual Server configuration
Figure A1-9a.17 shows a standard ArcGIS Server REST heavy web mapping workflow in a three-tier HA Virtual Server platform configuration. HA virtual platform architecture selection in cell A10 and host platform HA selection in cell A30 complete the configuration. Web and GIS Server Virtual Server tier will include a minimum of two (2) servers, and the DBMS virtual tier will be supported in a failover configuration. The Host Platform will be an N+1 configuration, with capacity for full throughput support with failure of one host platform node.

"Note: If two DBMS platforms are required to support peak throughput loads, a higher capacity platform (scale up) or active-active DBMS architecture (special configuration) will be required to support the peak loads. Multiple DBMS platforms can be configured by specifying the number of platform nodes for the DBMS tier in column D." <br style="clear: both" />

CPT Calculator ArcGIS Enterprise server roles
ArcGIS Enterprise server licensing roles. <br style="clear: both" />
 * ArcGIS Image Server (image services)
 * ArcGIS Image Server (raster analytics)
 * ArcGIS GeoAnalytics Server
 * ArcGIS GeoEvent Server

CPT Calculator GIS Server Role
Figure A1-9a.18 shows the CPT Calculator GIS Server role configuration for dynamic web services.

The ArcGIS Enterprise GIS Server role supports a variety of web mapping services and feature services. CPT Calculator Software Technology Performance Factors (Software, Graphics, Density/Portal, Complexity, %DataCache, Resolution, and Output) can be selected to generate performance targets for a variety of GIS Server capabilities.

"Workflow separation best practice: Critical system of record dynamic web services are best supported on a dedicated GIS Server site."

GIS Server can also host geoprocessing services. CPT Calculator can be configured for GIS Server geoprocessing services following the procedures identified for the ArcGIS Enterprise GeoAnalytics Server role.

"Workflow separation best practice: Heavy ArcGIS geoprocessing services are best supported on a dedicated GIS Server or GeoAnalytics Server site." <br style="clear: both" />

CPT Calculator ArcGIS Image Server license role (image services)
Figure A1-9a.19 shows the CPT Calculator ArcGIS Image Server configuration for dynamic image services.

The ArcGIS Enterprise Image Server supports a variety of dynamic image services. CPT Calculator Software Technology Performance Factors (Software, Graphics, Density/Portal, Complexity, %DataCache, Resolution, and Output) can be selected to generate performance targets for a variety of Image Server dynamic image service capabilities.

The following CPT Calculator selections are appropriate for dynamic image services.
 * Software = AGS Imagery (Cell E3).
 * Density/Portal = Raster Image (Cell I3). Select RPortal if Image Server is registered with Portal for ArcGIS.
 * Data Source = select imagery data source (Cell J6).
 * Platform Architecture = MDS EGDB (Cell A11). Selection includes mosaic dataset load on DBMS tier when hosted by the Enterprise Geodatabase. Select MDS FGDB if mosaic dataset is hosted on a file share.

"Workflow separation best practice: Dynamic image services are best supported on a dedicated Image Server site."

ArcGIS Image Server can also host raster analytics. CPT Calculator can be configured for Image Server raster analytics following the procedures identified for ArcGIS Image Server (raster analytics). <br style="clear: both" />

CPT Calculator Image Server (raster analytics)
Figure A1-9a.20 shows the CPT Calculator ArcGIS Image Server configuration for raster analytics.

The ArcGIS Image Server supports a variety of raster analytics services. CPT Calculator Software Technology Performance Factors (Software, Graphics, Density/Portal, Complexity, %DataCache, Resolution, and Output) can be selected to generate performance targets for a variety of ArcGIS Image Server raster analytics capabilities.

The following CPT Calculator Software Technology Performance Factors are appropriate for raster analytics.
 * Software = AGS Imagery (Cell E3).
 * Density/Portal = Raster Imagery (Cell I3). ArcGIS Image Server raster analytics are executed by service instances assigned within the ArcGIS Image Server site.  Once processing is complete, the final output appears as a reference layer in the Portal for ArcGIS named user content.  Output data is stored in the ArcGIS relational and tile cache data store. Shared content is published by the Portal for ArcGIS Hosted GIS Server.

CPT Calculator User Requirements settings. "Best practice. Managed Services Max Instances setting for Image Server Raster Analytics = available platform processor core." "Warning: Limit raster analytics max instances to allow for dynamic services when both are deployed on the same Image Server site."
 * Min Think = 0 (Cell D6). Raster analytics batch processing loads are generated by setting Min Think time = 0.
 * User = Available processor core (Cell A6). Enter Raster Analytics max instance service setting for peak concurrent User throughput.

CPT Calculator Platform Architecture settings.
 * Platform Architecture tier = Single (Cell A9)
 * Data Source = select imagery data source (Cell J6).
 * Platform Architecture = MDS EGDB (Cell A11). Selection includes mosaic dataset load on DBMS tier when hosted by the Enterprise Geodatabase. Select MDS FGDB if mosaic dataset is hosted on a file share.
 * Client load = Batch (Cell J15). Batch selection moves client load to Image Server tier.

"Workflow separation best practice: Batch raster analytics are best supported on a dedicated Image Server site." <br style="clear: both" />

CPT Calculator ArcGIS GeoAnalytics Server
Figure A1-9a.21 shows the CPT Calculator ArcGIS GeoAnalytics Server configuration for geoprocessing services.

The ArcGIS GeoAnalytics Server supports a variety of geoprocessing services. CPT Calculator Software Technology Performance Factors (Software, Graphics, Density/Portal, Complexity, %DataCache, Resolution, and Output) can be selected to generate performance targets for a variety of GeoAnalytics Server geoprocessing capabilities.

The following CPT Calculator Software Technology Performance Factors are appropriate for SOC geoprocessing services.
 * Workflow Source = Workflow tab (Cell C30).
 * Workflow selection = AGS SOC GeoBatch (Cell E30). The AGS SOC GeoBatch Standard Workflow assigns all processing loads to the SOC tier.

CPT Calculator User Requirements settings. "Best practice. Managed Services Max Instances setting for batch geoprocessing services = available platform processor core." "Warning: Limit geoprocessing max instances to allow for dynamic services when both are deployed on the same ArcGIS Server site."
 * Min Think = 0 (Cell D6). GeoAnalytics batch processing loads are generated by setting Min Think time = 0.
 * User = Available processor core (Cell A6). Enter GeoAnalytics max instance service setting for peak concurrent User throughput.

"Workflow separation best practice: Batch geoprocessing services are best supported on a dedicated ArcGIS Server site." <br style="clear: both" />

CPT Calculator ArcGIS GeoEvent Server
GeoNet GeoEvent blog

The ArcGIS GeoEvent Server supports a variety of real time ingestion and streaming services. CPT Calculator Software Technology Performance Factors (Software, Graphics, Density/Portal, Complexity, %DataCache, Resolution, and Output) can be selected to generate performance targets for a variety of GeoEvent Server event ingestion capabilities.

GeoEvent Server supports two different service capabilities.
 * GeoEvent Ingestion. Event inbound traffic and processing loads on GeoEvent Server.
 * GeoEvent streaming services. Event traffic streaming or messaging to client displays.

"Warning. GeoEvent streaming services traffic can cause network contention.''"

"Note. Based on ArcGIS 10.5.1 and prior releases. ArcGIS 10.6 provides a more scalable ArcGIS GeoEvent architecture."

ArcGIS 10.5 GeoEvent Server
Figure A1-9a.22 shows the CPT Calculator ArcGIS 10.5 GeoEvent Server configuration for real time event collection (ingestion) services.

The following CPT Calculator Software Technology Performance Factors are appropriate for GeoEvent ingestion loads (GeoEvent Server capacity planning).
 * Software Source = GeoProcessing (Cell E2).
 * Software = AGS GeoEvent (Cell E3).
 * Output = Feature. Inbound real time vector traffic (feature transactions).

"Warning. GeoEvent Server ingestion load is based on ArcGIS 10.5 medium complexity capacity planning expectations (relatively simple event processing loads).''"

CPT Calculator peak throughput Requirements settings.
 * Min Think = 0.01 (Cell D6). Inbound event traffic can approach 4000 transactions per second (no think time).  Important to set Min Think time > 0 for CPT to include random arrival wait (queue) times.
 * User = number of GeoEvent sources. Enter number of real time locations/vehicles (clients) being tracked.
 * DPM/client = average client transmission rate (B6). Enter client transmission rate (i.e. 60 transactions per minute/client) to generate peak throughput rate (Cell U6).

GeoEvent peak throughput rate may be limited by one of the following non-CPU intensive constraints. "Best practice. GeoEvent Server should be deployed on single machine sites - multi-machine sites do not scale due to very light inbound event transaction loads." "Workflow separation best practice: GeoEvent Server must be deployed on a dedicated machine."
 * RabbitMQ max throughput 4000 events/sec.
 * Maximum geofences must fit in physical memory.
 * JVM memory default may need to be reset to extend memory access.
 * DDR3 RAM max memory speed of 800-2133 MT/sec.
 * DDR4 RAM max memory speed 2100-4266 MT/sec (GeoEvent Server may require DDR4 memory to support peak throughput).

ArcGIS 10.6 GeoEvent Server
Figure A1-9a.23 shows the CPT Calculator ArcGIS 10.6 GeoEvent Server configuration for real time event collection (ingestion) services.

Beginning with the 10.6 release – GeoEvent Server is running an Apache Kafka instance as an event message broker within the ArcGIS GeoEvent Gateway Windows service. The message broker uses on-disk topic queues to manage event records. The event records which have been sent from the message broker to a GeoEvent Server instance for processing are recorded within the broker's associated configuration store (e.g. Apache Zookeeper).

The Kafka message broker provides a transactional message guarantee that the RabbitMQ message broker (used in 10.5.1 and earlier releases) does not provide. If the GeoEvent Gateway on a machine were stopped and restarted, the configuration store will have recorded where event message processing was suspended and will use indexes into the topic queues to resume processing previously received event records.

The CPT Calculator configuration for the ArcGIS 10.6 GeoEvent Server is similar to earlier releases.
 * Software Source = GeoProcessing (Cell E2).
 * Software = AGS 10.6 GeoEvent (Cell E3).
 * Output = Feature. Inbound real time vector traffic (feature transactions).

"Warning. AGS 10.6 GeoEvent Server ingestion load is based on medium complexity capacity planning expectations (relatively simple event processing loads).''"

CPT Calculator peak throughput Requirements settings.
 * Min Think = 0.01 (Cell D6). Inbound event traffic can approach over 12,000 transactions per second (no think time).  Important to set Min Think time > 0 for CPT to include random arrival wait (queue) times.
 * User = number of GeoEvent sources. Enter number of real time locations/vehicles (clients) being tracked.
 * DPM/client = average client transmission rate (B6). Enter client transmission rate (i.e. 60 transactions per minute/client) to generate peak throughput rate (Cell U6).

GeoEvent peak throughput rate may be limited by one of the following non-CPU intensive constraints. "Workflow separation best practice: GeoEvent Server sites must include an odd number of machines (i.e. three machine site) for optimum failover recovery."
 * Kafka message broker max throughput is much higher than previous releases. Maximum test ingestion rates exceeded 12,000 events/sec.
 * Maximum geofences must fit in physical memory.
 * JVM memory default may need to be reset to extend memory access.
 * DDR3 RAM max memory speed of 800-2133 MT/sec.
 * DDR4 RAM max memory speed 2100-4266 MT/sec (GeoEvent Server may require DDR4 memory to support peak throughput).

CPT Project Workflows
Figure A1-9a.24 shows a set of CPT Project Workflows that will be used to demonstrate enterprise data center design solutions in the GIS Product Architecture CPT Design demonstrations.

The CPT Project Workflows are generated by the CPT Calculator tab and moved to the CPT Workflow tab project workflow section to support the CPT Design analysis.

CPT Capacity Planning videos
<br style="clear: both" />

Page Footer Specific license terms for this content System Design Strategies 26th edition - An Esri ® Technical Reference Document • 2009 (final PDF release)