Questions tagged [netcdf4]

Questions related to NetCDF (Network Common Data Form) format and corresponding libraries/implementations may have this tag.

In 2008, the netCDF4 format was added to support per-variable compression, multiple unlimited dimensions, more complex data types, and better performance, by layering an enhanced netCDF access interface on top of the HDF5 format.

At the same time, a fourth format variant, netCDF-4 classic model format, was added for users who needed the performance benefits of the new format (such as compression) without the complexity of a new programming interface or enhanced data mode.

Every netCDF-4 or netCDF-4 classic model file can be read or written by the HDF5 library version 1.8 or later, so in that respect netCDF-4 files are already HDF5 files and need no conversion.

According to netCDF FAQ installing the netCDF4 is required for any of the following situations:

  • you need to access netCDF data that makes use of netCDF-4 compression or chunking
  • you need to access data in all netCDF formats including netCDF-4 or netCDF-4 classic model formats
  • you need to write non-record variables larger than 4GiB or record variables with more than 4GiB per record
  • you are installing netCDF to support other software packages that require netCDF-4 features
  • you want to write data that takes advantage of compression, chunking, or other netCDF-4 features
  • you want to be able to read netCDF-4 classic model data with no changes to your current software except relinking with the new library
  • you want to benchmark your current applications with the new libraries to determine whether the benefits are significant enough to justify the upgrade
  • you need to use parallel I/O with netCDF-4 or netCDF-4 classic files
651 questions
13
votes
1 answer

How best to rechunk a NetCDF file collection to Zarr dataset

I'm trying to rechunk a NetCDF file collection and create a Zarr dataset on AWS S3. I have 168 original NetCDF4 classic files with arrays of dimension time: 1, y: 3840, x: 4608 chunked as chunks={'time':1, 'y':768, 'x':922}. I want to write this…
Rich Signell
  • 14,842
  • 4
  • 49
  • 77
11
votes
3 answers

HDF5 Warnings When Accessing Xarray DataSet

I'd like to understand what is causing the warning messages that I'm getting in the following scenario: In an earlier operation I've created some NetCDF files and saved them to disk using xarray.to_netcdf(). Lazy evaluation of these datasets is…
jpolly
  • 141
  • 9
9
votes
1 answer

ValueError: found the following matches with the input file in xarray's IO backends: ['netcdf4', 'h5netcdf']

So I'm trying to run this script in Spyder (Python 3.9) dset = xr.open_dataset(filename, decode_times=True, use_cftime=True) print(dset) But I keep on getting this error: ValueError: found the following matches with the input file in xarray's IO…
kiwi_kimchi
  • 345
  • 3
  • 12
9
votes
1 answer

xarray/dask - limiting the number of threads/cpus

I'm fairly new to xarray and I'm currently trying to leverage it to subset some NetCDFs. I'm running this on a shared server and would like to know how best to limit the processing power used by xarray so that it plays nicely with others. I've read…
8
votes
3 answers

convert a netcdf time variable to an R date object

I have a netcdf file with a timeseries and the time variable has the following typical metadata: double time(time) ; time:standard_name = "time" ; time:bounds = "time_bnds" ; time:units = "days since 1979-1-1…
ClimateUnboxed
  • 7,106
  • 3
  • 41
  • 86
8
votes
4 answers

Convert NetCDF file to CSV or text using Python

I'm trying to convert a netCDF file to either a CSV or text file using Python. I have read this post but I am still missing a step (I'm new to Python). It's a dataset including latitude, longitude, time and precipitation data. This is my code so…
aliki43
  • 161
  • 1
  • 2
  • 5
8
votes
2 answers

netCDF4 import not being found by Python

I need to use the netCDF4 module in order to do some processing. I have followed the steps on this guide and also tried used anaconda2 as recomended on this post. netCDF4 seems to be installed, as i can run nc-config --version and obtain: netCDF…
Caio César
  • 221
  • 1
  • 4
  • 12
6
votes
1 answer

Convert a numpy dataset to netCDF

I have a numpy array in python with size (16,250,186) representing time, latitude and longitude. I want to convert it to a netCDF file so that I can read the data easily with co-ordinates in future. My numpy array looks like this RZS =…
Ep1c1aN
  • 683
  • 9
  • 25
6
votes
2 answers

\[Errno -101\] NetCDF: HDF error when opening netcdf file

I have this error when opening my netcdf file. The code was working before. How do I fix this ? Traceback (most recent call last): File "", line 1, in ... File "file.py", line 71, in gather_vgt return xr.open_dataset(filename) File…
Florian
  • 874
  • 1
  • 8
  • 17
5
votes
1 answer

Converting generic grid type to a lonlat gridtype using cdo

I am currently trying to use one NetCDF file (open-source data which can be downloaded from https://doi.pangaea.de/10.1594/PANGAEA.828650) to extract specific latitudes from this dataset…
Ellie Nelson
  • 67
  • 1
  • 7
5
votes
3 answers

Monthly sum of wet days from daily data using Climate Data Operators (CDO)

I have climate data with a daily temporal resolution and would like a count of days that have precipitation (e.g., greater than 1mm/day) by month and by year. I've tried eca_pd,1 and eca_rr1, but these commands return wet-day totals for all…
derelict
  • 3,657
  • 3
  • 24
  • 29
5
votes
2 answers

Change grid size of a netCDF file

Let's assume I have 2 netCDF data files with data for the same region (like South America, Africa, etc) but the different grid sizes as 0.5 degrees x 0.5 degrees and 1.0 degrees x 1.0 degrees in another. I want to increase or decrease its grid size…
Ep1c1aN
  • 683
  • 9
  • 25
5
votes
2 answers

Using xarray to change coordinate system in order to Slice operation

I am new here. first on all, I am very thankful for your time and consideration. I have 2 questions regarding to managing 2 different netcdf files in python. I searched a lot but unfortunately I couldn't find a solution. 1- I have a netcdf file…
5
votes
2 answers

Can I write to a HDF5 file from multiple processes/threads?

Does hdf5 support parallel writes to the same file, from different threads or from different processes? Alternatively, does hdf5 support non-blocking writes? If so then is this also supported by NetCDF4, and by the python bindings for either? I am…
benjimin
  • 4,043
  • 29
  • 48
5
votes
3 answers

Extracting data for a specific location from netCDF by python

I am new to using Python and also new to NetCDF, so apologies if I'm unclear. I have an nc file that has several variables and I need to extract data from those nc files in a new order. My nc file has 8 variables (longitude, latitude, time, u10,…
박성우
  • 53
  • 1
  • 1
  • 3
1
2 3
43 44