Boosting performance through pre-computing

Boosting performance through pre-computing:

At Esri conferences, we frequently offer workshops on designing faster GIS web services. Perhaps the most important piece of advice that we give in these workshops is to pre-compute when possible. In this post, we’ll explain what we mean by “pre-computing” and offer some simple ways you can speed up your ArcGIS Server services by following this principle.

The importance of pre-computing in a server environment

When you do something in your desktop GIS program, you have the computer all to yourself. You can ask the computer to do computationally-intensive things, and often those things will be accomplished fairly quickly because you have the machine’s entire resources at your disposal.

In the server GIS world, things are a bit different. You usually aren’t the only one requesting things from a GIS server. It may have thousands of other users to accommodate at the same time. And in the web world, most users expect to see their results immediately. If a user has to wait longer than a few seconds for something to happen, the app may be deemed unusable, no matter how amazing its functionality.

The key to pre-computing is to understand what work your server is doing, and complete as much of that work as possible before the user request is made. The less work your server has to do at the time of a user request, the more users it can accommodate at once. Let’s look at some examples.

Map caching

Map caching is a prime example of pre-computing in order to keep your services running fast. You spend some initial time and computing resources drawing map images (tiles) and storing them in a server-side cache. After the cache is built, the service can just hand out the tiles as requested instead of drawing map images on the fly. Usually the tiles are created on a staging server and copied onto the production server during off-hours.
Not only is map caching fast (a server can return a tile in a fraction of a second), it’s also scalable. You can accommodate many more simultaneous users when handing out cache tiles because your server is hardly doing any work.

It can take a long time to create a map cache, especially if the map covers large scales across a broad geographic extent. Fortunately many of the other pre-computing tips in this post can help speed up tile creation.

Projecting data

Projection on the fly is computationally intensive, and it can introduce itself at various tiers in your GIS server architecture. If you can match the coordinate system of your source data, your data frame, and your web map, you can eliminate projection on the fly altogether.

Most people hesitate to match their coordinate system all the way up the stack because their web maps use Web Mercator, an undesirable projection for storing source data. A solution to this is to make a one-way replica of your datasets in Web Mercator to use for web drawing purposes only. The replication action can actually output the data to Web Mercator, which is an easy way to project all your datasets while leaving the parent replica in the original coordinate system.

See Esri Knowledge Base article 34129 to learn more about changing coordinate systems during replication.

Pre-calculating geoprocessing results

If your app contains a geoprocessing model that could be run on a common or known set of features, consider pre-calculating the results and storing them in a database or table. In our workshops, a common example we show is the Solar Boston app, in which the developers pre-calculated the solar potential of all the rooftops in downtown Boston. You can click any rooftop in this app and see the solar potential in a fraction of a second, because the model result is already stored as an attribute of the building. On the other hand, if you use the option to calculate the solar potential of a freehand polygon, you have to wait a while for the server to run the model.

Maintaining spatial and attribute indexes

When ArcGIS draws a map, it has to retrieve all the features that pertain to the current extent of the map. If the datasets in the map have a spatial index built, the features can be found much more quickly, and therefore the draw speed improves. Build spatial indexes and keep them up to date for any datasets whose drawing performance is a concern.

Similarly, when ArcGIS performs a query on a table, an attribute index can help it find the needed records more quickly. If you know that a certain field will be queried often, it’s worth the time to create an attribute index for that field.

Labeling with annotation

Automatically finding an ideal placement for map labels is a computationally intensive problem that has inspired much research. Modern label engines such as Maplex are able to position scores of labels in a tiny map space without overlap. However, these intelligent label placement decisions cost CPU and memory resources. Your server can be spared some labeling work on each map draw request if you use annotation.

Annotation is a technique for saving label information to a database. When you use annotation in your services, each label has a predetermined location; therefore, the server doesn’t have to make any label placement decisions. To get the best of both worlds, you can use Maplex in ArcGIS Desktop to get the best possible placement for your labels, then save those labels to annotation for use in your web services.

Summary

The above examples show that there are a variety of ways that you can pre-compute information to speed up your GIS web services. Many of the pre-computing techniques can also help speed up the creation of map cache tiles, which is the ultimate form of pre-computing.

Do you have any other pre-computing practices that you use to speed up your GIS? We’d like to hear about them as comments to this post.

Contributed by Sterling Quinn of the ArcGIS Server development team