TopoJSON

Next, we will move on to preparing this data relationship for the Web and its spatiotemporal visualization.

TopoJSON is a variant of JSON, which uses the topological relationships between the geometric features to greatly reduce the size of the vector data and thereby improves the browser's rendering performance and reduces the risk of delay due to data transfers.

An example of GeoJSON

The following code is an example of GeoJSON, showing two of our State House Districts. The format is familiar—based on our previous work with JSON—with sets of coordinates that define a polygonal area grouped together. The repeated sections are marked by ellipses (…).

{
  "type": "FeatureCollection",
  "crs": { "type": "name", "properties": { "name":   "urn:ogc:def:crs:OGC:1.3:CRS84" } },

  "features": [
    { "type": "Feature", "properties": { "STATEFP": "42", "SLDLST": "181", "GEOID": "42181", "NAMELSAD": "State House District 181", "LSAD": "LL", "LSY": "2014", "MTFCC": "G5220", "FUNCSTAT": "N", "ALAND": 8587447.000000, "AWATER": 0.000000, "INTPTLAT": "+39.9796799", "INTPTLON": "-075.1533540" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -75.176782, 39.987337 ], [ … ] ]] } }, {…}]}

An example of TopoJSON

The following code is the corresponding representation of the same two State House Districts in TopoJSON, as discussed earlier.

Although this example uses the same coordinate system (WGS84/EPSG:4326) as that used before, it is expressed as simple pairs of abstract space coordinates. These are ultimately transformed into the WGS coordinate system using the scale and translate data in the transform section of the data.

By taking advantage of the shared topological relationships between geometric objects, the amount of data can be drastically reduced from 21K to 7K. That's a reduction of 2/3! You will see in the following code that each polygon is not clearly represented on its own but rather through these topological relationships. The repeated sections are marked by ellipses (…).

{"type":"Topology","objects":{"geojson":{"type":"GeometryCollection","crs":{"type":"name","properties":{"name":"urn:ogc:def:crs:OGC:1.3:CRS84"}},"geometries":[{"type":"Polygon","properties":{"STATEFP":"42","SLDLST":"181","GEOID":"42181","NAMELSAD":"State House District 181","LSAD":"LL","LSY":"2014","MTFCC":"G5220","FUNCSTAT":"N","ALAND":8587447,"AWATER":0,"INTPTLAT":"+39.9796799","INTPTLON":"-075.1533540"},"arcs":[[0,1]]},{"type":"Polygon","properties":{"STATEFP":"42","SLDLST":"197","GEOID":"42197","NAMELSAD":"State House District 197","LSAD":"LL","LSY":"2014","MTFCC":"G5220","FUNCSTAT":"N","ALAND":8251026,"AWATER":23964,"INTPTLAT":"+40.0054726","INTPTLON":"-075.1405813"},"arcs":[[-1,2]]}]}},"arcs":[[[903,5300],[…]]]],"transform":{"scale":[0.0000066593659365934984,0.000006594359435944118],"translate":[-75.176782,39.961088]}}

Vector simplification

Similar to TopoJSON, Vector simplification removes the nodes in a line or polygon layer and will often greatly increase the browser's rendering performance while decreasing the file size and network transfer time.

As the vector shapes can have an infinite level of complexity, in theory, simplification methods can decrease the complexity by more than 99 percent while still preserving the perceivable shape of the geometry. In reality, the level of complexity at which perception becomes significantly affected will almost never be this high; however, it is common to have a very acceptable perception change at 90 percent complexity loss. The more sophisticated simplification methods have improved results.

Simplification methods

A number of simplification methods are commonly in use, each having strengths for particular data characteristics and outcomes. If one does not produce a good result, you can always try another. In addition to the method itself, you will also usually be asked to define a threshold parameter for an acceptable amount of complexity loss, as defined by the percentage of complexity lost, area, or some other measure.

Douglas-Peucker: In this, the threshold affects the distance from which the original lines and edges of the polygons are allowed to change. This is useful when the nodes to be simplified are densely located but can lead to "pointy" simplifications.
Visvalingam / effective area: In this, the point forming the triangle having the least area with two adjacent points is removed. The threshold affects how many times this criterion is applied. This has been described by Mike Bostock, the creator of TopoJSON among other things, as simplification with the criteria of the least perceptible change.
Visvalingam / weighted area: In this, the point forming the vertex with the most acute angle is removed. The threshold affects how many times this criterion is applied. This method provides the "smoothest" result, as it specifically targets the "spikes".

The Visvalingam effective area method is the only method of simplification offered in the TopoJSON command-line tool that we will use. Mapshaper, a web-based tool that we will take a look at offers all these three methods but uses the weighted area method by default.

Other options

Other options affecting the data size are often offered alongside the simplification methods.

Repair intersections: This option will repair the simplifications that cause the lines or polygon edges to intersect.
Prevent shape removal: This option will prevent the simplification that would cause the removal of the (usually small) polygon shapes.
Quantization: Quantization controls the precision of the coordinates. This is easier to think about when we are dealing with the coordinates in linear units. For obvious reasons, you may want to extend the precision to 1/5000 of a mile—getting the approximate foot precision. Also obviously, great precision comes at the cost of greater data size, so you should not overquantize where the application or source does not support such precision.

Simplifying for TopoJSON

Both Web and desktop TopoJSON conversion tools that we will use support these simplification options. That way, you can simplify a polygon at the same time as you reduce the data size through the topological relationship notation.

Simplifying for other outputs

If you wish to produce data other than for TopoJSON, you will need to find another way to do the simplification.

QGIS provides Simplify Geometries out of the box (navigate to Vector | Geometry Tools | Simplify Geometries), which does a Douglas-Peucker simplification. While it is the most popular method, it may not be the most effective one (see the following section for more).

The Simplify plugin offers a Visalvingam method in addition to Douglas-Peuker.

Converting to TopoJSON

There are a few options for writing TopoJSON. We will take a look at one for the desktop, which requires a software installation, and one via the web browser. As you might imagine, the desktop option will be more stable for doing anything in a customized way, which the web browser does not support, and is also more stable with the more complex feature sets. The web browser has the advantage of not requiring an install.

Web mapshaper

You can use the web-based mapshaper software from http://www.mapshaper.org/ to convert from shapefile and other formats to TopoJSON and vice versa. Perform the following steps to convert the State Legislative Districts shapefile to TopoJSON:

Open your browser and navigate to the mapshaper website.
Optionally, select a different simplification method or try other options. This is not necessary for this example.
Browse to select the Philadelphia State Legislative Districts shapefile (c5/data/original/tl_2014_42_sldl/tl_2014_42_sldl.shp) from your local computer or drag a file in. As the page indicates, Shapefile, GeoJSON, and TopoJSON are supported.
Optionally, choose a simplification proportion from the slider bar (again, this is not needed for this example).
Export as TopoJSON by clicking on the Export button at the top.

You will get the following screen in your browser:

The command-line tool

The command-line tool is useful if you are working with a larger or more complicated dataset. The downside is that it requires that you install Node.js as it is a node package. For our purposes, Node.js is similar to Python. It is an interpreter environment for JavaScript, allowing the programs written in JavaScript to be run locally. In addition, it includes a package manager to install the needed dependencies. It also includes a web server—essentially running JavaScript as a server-side language.

Perform the following steps:

Install Node.js from https://nodejs.org/.
Open your OS command line (for example, on Windows, run cmd).
Input the following in the command line:
```
>npm install -g topojson
```
Navigate to cd c:\packt\c5\data\output and input the following:
```
>topojson -p -o house_district.json house_district.shp
```

You will now get the following output: