Tracking
Contents
Enterprise Tracking Tools for ESRI Products
There are many questions we can ask about our systems that tracking can answer. For instance, what is the most popular application, what are the most frequently used feature classes, which users are the most active or what are the lowest utilized map services? Besides popularity of components, we can also track performance and answer questions like what is my map service average request time, what is my resource utilization on my map server or application server, could my existing system handle a new application? Without tracking in place there is no way to knowledgeably answer or quantify these valuable questions. These questions can drive investment decisions, resource allocation and strategic planning for any organization administering a GIS.
This page will provide details on how to track four elements of your enterprise system; including the license manager, SDE databases, ArcGIS Server, and web applications. The following will list information on how to deploy tracking, how to analyze the tracking outputs and finally what questions you can answer and actions you can take depending on the results. These three phases will be detailed for each of the four tracking elements.
ESRI License Tracking
Details
The Desktop element is all about tracking the license manager usage. Details of interest include how many licenses desktop users are consuming on average, what extensions they use, when users are in the system most and what the maximum concurrent usage is of licenses. The tracking software is a batch that can be scheduled in Task Scheduler to run at any interval, but recommended to run every fifteen minutes. This batch will run an executable that comes with every installation of Esri’s License Manager and is called lmutil.exe. The executable will be run with the lmstat parameter which will provide the product, user and computer name of all the users that are in the system at that time and write those details to a file.
How to Deploy
1. Click Here to download the resources needed from Github. Use the download ZIP button to get all of the files.
2. Unzip the folder and open the ReadMe.pdf for instruction on where to place the batch file, configure it and set it up to run automatically from task scheduler.
How to Analyze
1. The ReadMe.pdf file details how to process our log files using the Template.xlsm file.
View Analysis Results
1. Open the spreadsheet you created in the Analysis section and view the Results tabs
Note: Make sure you close and open the spreadsheet so the pivot tables refresh automatically
Actions
Questions you can answer
1. Who are my power desktop users and how many hours do they consume of licenses? (Result1 Worksheet)
2. Who are all of the active desktop users in my system and how many are there? (Result2 Worksheet)
3. What department has the most users and uses the most hours? (Result3 and Result4)
4. How many hours of desktop usage has my organization used? (Result5)
5. What is my maximum concurrent license usage ever and what is my average license usage? (Result5)
6. What is my annual user cycle by number of users? (Result6)
7. What is my annual user cycle in hours of usage? (Results7)
Steps You Can Take
Better Understanding of User Community
- Quantitatively show how many people GIS serves and where they are in the City
- Identify trainers that can assist with new users
- Identify department technical leads
- Identify testers for software upgrade or new application testing
- Have accurate list of users that can be used to notify of outages, upgrades, training opportunities and anything pertinent to the GIS operations
Effective Strategic Planning
- Plan outages during times of low usage
- Identify required users for software upgrades
- If departments contribute to license costs, this would quantify how much each department should pay for their portions of usage
- Identify training needs, if there is a department with high usage, but no training this could validate the need for additional training
Show Management Value of GIS
- Show how many users are leveraging GIS and how many hours they spend taking advantage of the software
Negotiate ESRI Purchasing
- Validate the usage of enterprise licenses
- Validate the need for licenses you pay for annually
- Reveal if you are paying for too many licenses annually
Web Applications
Details
The Web Application element is intended to track application usage, with the goal of tracking down to the widget level to understand exactly what applications your users are using and what functionality they are finding most useful. The tracking software chosen for this task is Google Analytics. Ideally, a heatmap tracking software would be perfect for this task, but due to limitations in not being able to capture dynamic content in an application, they are not a good fit at this time. Google analytics provides numerous metrics about your users and it is possible to track at the widget level.
How to Deploy
1. Click Here to download the resources needed from Github. Use the download ZIP button to get all of the files.
2. Unzip the folder and open the ReadMe.pdf for instruction on to get started with Google Analytics and how to incorporate widget level tracking.
How to Analyze and View Results
1. The ReadMe.pdf file details how to navigate the Google Analytics site to view your metrics and visualize your tracking results.
Actions
Questions you can answer
1. What is my most/least popular application?
2. How many hits do my servers get a day?
3. When are my busy times?
4. What is my most used widget?
5. What is the percent usage of each widget?
6. Are users testing staging applications?
7. Are users transitioning from legacy to new applications?
8. How long are people using my application?
Actions
1. Better understanding of critical/less critical applications.
2. Verify users are testing staging environments.
3. Verify that users are transitioning once a new application is released.
4. Understand what are the most popular widgets and required then to be included on new applications.
5. Can ultimately be put into a capacity planning tool to understand the load on the server infrastructure and plan for new applications.
ArcGIS Server
Details
ArcGIS Server tracking is focused around understanding the load on your ArcGIS Servers and optimizing your map services. ArcGIS Server has a freely available System Monitor product that you can download here or they also have a more advanced version that you can find here, which required a professional services contract. With the freely available version you are able to efficiently track ArcGIS Server and configure alerts to monitor the status of the map services. This product uses MongoDB to store your logs on the back end and consists of an agent service and web application. Once you configure System Monitor in a staging environment, or even on your desktop, the ArcGIS Server will be pinged by the agent at whatever interval you set to retrieve the ArcGIS Server log files and write them back to the MongoDB. From there you can interact with the System Monitor Application. System Monitor causes very minimal resource consumption on your ArcGIS Server, since it is just requesting log files and writing them back to where you have System Monitor installed.
How to Deploy
1. There is a detailed installation guide provided in the download. It takes you through step by step how to configure the MongoDB, the windows services and application. Follow the instructions very carefully and do not install this product on a production server.
2. Tips: There are groups and permissions that need to be configured on the servers you wish to track.
How to Analyze and View Results
1. You can view your tracking data in the System Monitor application. It allows you to view specific time ranges, services, errors, throughput, etc.
2. You can also configure Dashboards and automated reports through the System Monitor application.
Actions
Questions you can answer
1. What is my most/least popular service?
2. How are my resources being consumed across the environment, is one server carrying a larger load than the others?
3. Are there consistent errors in my environment?
4. What is my throughput on services?
5. How many transactions do my services process?
Actions
1. Set low utilized services to min instance of 0
2. Increase resources (min instances) on popular services
3. Migrate services so load is equaly balanced across environment
4. Leverage Capacity Planning tool to plan for future services
SDE Table Tracking
Details
The table tracking trigger is aimed at monitoring the database usage and more specifically which feature classes are being used within the database. The tracking for this element includes adding a table to the SDE database and then configuring a database trigger. There are three feature classes within the database that when joined together will give you the user connecting, the computer name the user is connecting from and the feature class name they are using. The three tables within the database include the PROCESS_INFORMATION, TABLE_LOCKS, and TABLE_REGISTRY tables. The PROCESS_INFORMATION table provides all of the users at that time and an SDE_ID. That SDE_ID is joined to the TABLE_LOCKS table as an intermediary to get a REGISTRATION_ID, which is then plugged into the TABLE_REGISTRY table. Once that REGISTRATION_ID is plugged into the TABLE_REGISTRY table, you are able to get the table name that that user is using. The graphic to the right details an example of the joining of the three tables that ultimately shows the GEOWORKS user is using the SERVICEREQUESTS_WAM feature class.
An insert trigger can be placed in the database to execute when a user connects to a table within the database. The trigger can perform the join, capture the user, computer name and feature class as well as the start time and write it to a tracking table within the database. A second delete trigger can also be added to capture the end time when the user disconnects from the database. To complete this set-up all that is needed is a tracking table and the two database triggers detailed below.
How To Deploy - Oracle
1. Click Here to download the resources needed from Github. Use the download ZIP button to get all of the files.
2. Unzip the folder and open the ReadMe.pdf for instruction on to get started deploying the database triggers and testing that they are working properly.
How To Analyze
1. The ReadMe.pdf file details how to process the Table Log using the Template.xlsm file.
View Analysis Results
1. Open the spreadsheet you created during the analysis section
Note: Be sure to close and re-open after you are done with the Analysis section. The pivot tables are set to refresh and pick up the newly processed data on restart of excel
2. The Worksheet tabs named Results1-Results5 contain pivot tables that reference the data and answer different questions about your environment
3. Navigate through the tabs to view the analysis results of your data processing
Actions
Questions you can answer:
1. What are my top ten feature classes? (Result1)
2. What are all the used feature classes in the desktop environment? (Result2)
3. Which feature classes are not used in my environment? (Result3)
4. What feature classes are used by each department? (Result4)
5. When was the last time a feature class was used? (Result5)
Steps you can take:
1. Verify/Notify Users about changes happening to a FC or outages to a FC Archive Inactive Tables
2. Optimize the most used FC’s (run analyze, statistics more frequently)
3. Efficient Data Dissemination Strategy focused around most popular FC’s
