Building a uniform topojson with two quantized/transformed topojsons?

Question

Background

I'm building a map that displays a map of the US. On this map I am plotting heat circles that correspond to a Nielsen DMA topography.

The first topojson that I'm using, is this Nielsen DMA topojson (from simzhou's repo here) to visually plot these heat circles across the US map.

Below you can see the map, with the DMA heat circles, and also the DMA border lines built completely from the Nielsen DMA topojson.

Problem:

The issue I'm having is trying to draw state border lines, instead of these DMA border lines. I've brought in the "https://unpkg.com/us-atlas@1/us/10m.json" that @mbostock has provided for us. When drawing the state borders via topojson.feature(us, us.objects.states).features (i've tried topojson.mesh too), that's when things go awry. I am 99% sure that this is because the two json files are using different transform values, and therefore the positions/coordinates are transformed on different scales.

Here are the two jsons: Nielsen DMA here and the US here

You can see how the transform object differs below:

Transform object from US Atlas

"transform": {
       "scale": [0.009995801851947097,0.005844667153098606],
       "translate":[-56.77775821661018,12.469025989284091]
}

Transform object from DMA topojson

"transform": {
        "scale": [0.00577894299429943, 0.002484260626062607],
        "translate": [-124.732975, 24.544237]
},

Here's what I've done so far.

Approach:

1. Round trip through GeoJSON

As detailed by Bostock here I've tried making a new topoJSON "via a round trip through GeoJSON."

Quantized → non-quantized, to remove quantization. This is often done temporarily to process data (for example, topojson.presimplify). I suppose you might want this so that you could combine topologies with different quantized transforms, but you could always do this by making a round trip through GeoJSON.

For each of the jsons 1. I converted them from topoJSON to geoJSON. topo2geo nielsen_dma=us-dma-geo.json < us-dma-topo.json Now for each json we have a feature collection with absolute coordinates. 2. With the new geoJSON, I then converted them back to topoJSON via CLI. geo2topo nielsen_dma=us-dma-geo.json > us-dma-topo.json Both JSONs no longer have the transform property, but they do have bbox. 3. Now I have both jsons make a round trip from topojson -> geojson -> topojson. 4. I stripped bbox properties from both jsons, as they are optional. 5. I then simply added over the geometry collection of one to the other. statesJSON.objects.nielsen_dma = dmaJSON.objects.nielsen_dma

I now have a topojson with the nielsen_dma and states geometries. However this still doesn't work, and drawing the state lines brings chaos.

Did I fail to remove the quantization of the coordinates for both jsons during the round trip to geoJSON?

Possibly (ir)relevant Question:

1. The Nielsen DMA map does not include geometries for Alaska and Hawaii. Could this discrepancy between the two jsons lead to this issue?

Answer 1

When I first read this I attempted to address the underlying problem in the comments (for which I missed the mark a bit, as I assumed the code to display the map looked a little different than it likely was - I apologize for the misread). However, on second examination, I realized that while the question is an x,y problem, the attempted solution leads to some interesting questions about D3 and topojson that aren't immediately clear - but the underlying issue remains mixing projected and unprojected data, so I'll attempt to address both the x and the y here:

Topojson

Topojson essentially is a method of storing geojson features or feature classes by encoding topology and, optionally, quantized delta encoded integer coordinates. Topojson itself doesn't change the underlying coordinates, it changes their representation.

In order to use topojson with D3, we need to convert it back to geojson, as D3 geoPaths only accept geojson - topojson.feature() returns geojson.

The Underlying Source of Pain

The ultimate issue doesn't arise from the topojson at all. You have two topojon sources that use different coordinate systems:

• the US json uses two dimensional Cartesian coordinate space (features projected with an Albers projection). The uncompressed coordinates are SVG pixels.

• the Nielsen json uses latitude/longitude coordinate space (points on a 3d globe). The uncompressed coordinates are not Cartesian (unless applying a straight Plate Carree projection: long = x, lat = y).

When the US json is passed through topojson.feature(), the returned geojson will have x,y values within a bounding box of roughly [0,0],[960,600] (in this case). When any unprojected topojson (Nielsen json for exmaple) is passed through topojson.feature(), the returned geojson will have long,lat values within a bounding box of [-180,90],[180,-90] - that is it will have valid long/lat pairs.

These underlying and differing coordinate systems are the issue, not the representation of the coordinates in the topojson.

Combining Topojson

I'm not sure on the details of how you combined the two. A topojson splits geojson into constituent parts - arcs and features (which reference the arcs and hold the feature properties). The arcs are referenced numerically - if you didn't modify the indexes, then you can't just use:

statesJSON.objects.nielsen_dma = dmaJSON.objects.nielsen_dma

But, even if you did modify the arc indexes and bring in the arcs that aren't in the statesJSON but are in the dmaJSON, you'd still run into problems. This is because we are still using different coordinate systems.

The transform that a topojson uses consists of scale and translate - the projected data uses an Albers projection, the unprojected data doesn't use any projection, but could be coerced to x,y with a Plate Carree projection (long=x,lat=y). But a Plate Carree features latitudes that are straight (it is an equirectangular projection), while the Albers features latitudes that are curved (it is a conical projection). Consequently, the shapes of identical features in each coordinate system will be different. Directions too will be different between each coordinate system (East is not to the direct right on most places on an Albers). You cannot modify shape and direction with translate and scale.

Different Topojson Transforms

I am 99% sure that this is because the two json files are using different transform values, and therefore the positions/coordinates are transformed on different scales.

It will be very rare to use two topojson files that have the same transform values, likewise with bounding box. The transform is only critical for reproducing the original coordinates that were used to create the topojson. Its values are only relevant for converting encoded topojson coordinates back to the original (geojson) coordinates. The transform is specific to that topojson's coordinates.

Ultimately, the transform is not relevant when using D3: the original coordinates used to create the topojson are what is actually used by D3, through topojson.feature(). The transform is essentially immaterial - as long as it is constant for the encoding and decoding.

How Do I Fix This?

You have a projected file (the US json) for which you do not know the projection that was used to create it. Without knowing the projection used to create it, you cannot apply the same projection on the Nielsen data. Even if you could run the Nielsen data through a d3 geoProjection so that it overlaid the US data (which uses a null projection) - you shouldn't. Different coordinate systems are cumbersome when zooming, panning, or setting things like centering coordinates - it requires separate code to handle unprojected and projected data, essentially duplicating code

This means that you need to find a new data source for the US. Luckily there are many online sources, for example here are two: excessive size for web or reasonable size for web. You can convert to topojson if you want with mapshaper.org (and simplify the file too).

With a geojson/topojson for the US that is unprojected (long/lat pairs), you now have two files that share the same coordinate system. The topojson transform will be different for each (if working with topojson for both), this is immaterial, what matters is that the underlying coordinates are in the same coordinate space, not that their quantized representation in the topojson uses the same transform.

Here's a basic example using an unprojected US states geojson (from the second linked geojson, converted to topojson):

https://bl.ocks.org/andrew-reid/178445c6acd84aa3b43525076f277157