replace values in xarray dataset with None

Question

I want to replace values in a variable in an xarray dataset with None. I tried this approach but it did not work:

da[da['var'] == -9999.]['var'] = None

I get this error: *** TypeError: unhashable type: 'numpy.ndarray'

Is there something like numpy replace that I could use here? da is xarray dataset. here is what da looks like:

<xarray.Dataset>
Dimensions:  (band: 1, time: 3, x: 4258, y: 2334)
Coordinates:
  * band     (band) int32 1
  * y        (y) float64 4.406e+06 4.406e+06 4.406e+06 4.406e+06 4.406e+06 ...
  * x        (x) float64 1.125e+05 1.126e+05 1.127e+05 1.128e+05 1.129e+05 ...
  * time     (time) datetime64[ns] 2005-12-31 2006-12-31 2007-12-31
Data variables:
    var      (time, band, y, x) float32 dask.array<shape=(3, 1, 2334, 4258), chunksize=(1, 1, 2334, 4258)>

Here is what da.var looks like:

<xarray.DataArray 'var' (time: 3, band: 1, y: 2334, x: 4258)>
dask.array<shape=(3, 1, 2334, 4258), dtype=float32, chunksize=(1, 1, 2334, 4258)>
Coordinates:
  * band     (band) int32 1
  * y        (y) float64 4.406e+06 4.406e+06 4.406e+06 4.406e+06 4.406e+06 ...
  * x        (x) float64 1.125e+05 1.126e+05 1.127e+05 1.128e+05 1.129e+05 ...
  * time     (time) datetime64[ns] 2005-12-31 2006-12-31 2007-12-31
Attributes:
    transform:   (90.0, 0.0, 112500.0, 0.0, -90.0, 4406400.0, 0.0, 0.0, 1.0)
    crs:         +ellps=GRS80 +no_defs +proj=utm +towgs84=0,0,0,0,0,0,0 +unit...
    res:         (90.0, 90.0)
    is_tiled:    1
    nodatavals:  (-9999.0,)

typically, one would replace missing values with `np.nan`. Is that an option here? — jhamman, May 01 '18 at 21:24

score 24 · Accepted Answer · edited May 02 '18 at 01:32

24

The standard way to do this is using where: http://xarray.pydata.org/en/latest/indexing.html#masking-with-where

# Note that I'm going to use `ds` instead of the OP's `da`

# replace all values equal to -9999 with np.nan
ds_masked = ds.where(ds['var'] != -9999.)

edited May 02 '18 at 01:32

shoyer

9,165
1
37
55

answered May 01 '18 at 21:31

jhamman

5,867
19
39

1

thanks @jhamman, somehow this does not seem to fix the issue i.e. I save `ds_masked` to disk and the -9999 is still there – user308827 May 01 '18 at 21:37
4

I edited jhamman's answer -- he should have written `ds.where(ds['var'] != -9999.)`, not `ds.where(ds['var'] == -9999.)` -- which does the opposite of what you want. Hopefully that works for you! – shoyer May 02 '18 at 01:33
2

Thanks @shoyer. Indeed I was mistaken in my comparison. – jhamman May 02 '18 at 02:34

replace values in xarray dataset with None

1 Answers1

Linked