Need assistance in Appending two netcdf files based choosing time period

Question

I have two netcdf files: 1) BB_001.nc with 337 records with record variable as time; 2) BB_002.nc which is continuation of the simulation with 385 records (record variable is also time). Therefore, these two files have one overlap record.

$ ncdump -h BB_001.nc
netcdf BB_WC {
dimensions:
nele = 278399 ;
node = 143546 ;
siglay = 18 ;
siglev = 19 ;
three = 3 ;
time = UNLIMITED ; // (337 currently)
DateStrLen = 26 ;
maxnode = 11 ;
maxelem = 9 ;
four = 4 ;
...

$ ncdump -h BB_002.nc
netcdf BB_0001 {
dimensions:
nele = 278399 ;
node = 143546 ;
siglay = 18 ;
siglev = 19 ;
three = 3 ;
time = UNLIMITED ; // (385 currently)
DateStrLen = 26 ;
maxnode = 11 ;
maxelem = 9 ;
four = 4 ;
...

I want to append them but the last record of BB_001.nc and first record of BB_002.nc is the same and I need to delete the redundant record.

I tried the following command:

ncks –A –d time,1,385 BB_002.nc BB_001.nc

But it did not work and records of BB_001.nc is still the same (337 instead of 337+384=721).

Alternatively, I tried:

ncrcat BB_001.nc BB_002.nc test.nc

Is works but test.nc has 722 records. How can I get rid of the redundant record?

$ ncdump -h test.nc 
netcdf test {
dimensions:
nele = 278399 ;
node = 143546 ;
siglay = 18 ;
siglev = 19 ;
three = 3 ;
time = UNLIMITED ; // (722 currently)
DateStrLen = 26 ;
maxnode = 11 ;
maxelem = 9 ;
four = 4 ;

Any help is very much appreciated.

Thanks,

did any of these solutions help? – ClimateUnboxed Jun 19 '18 at 15:44 — ClimateUnboxed, Jun 19 '18 at 15:44

score 2 · Accepted Answer · answered Jun 15 '18 at 19:22

Ordinarily people do this in two steps as above, e.g.,

ncks -d time,1, BB_002.nc 2.nc
ncrcat BB_001.nc 2.nc out.nc

People who read the http://nco.sf.net/nco.html are aware of the --record_append option which allows this to be done in one step, i.e., faster. Exercise left for reader.

score 1 · Answer 2 · answered Jun 14 '18 at 17:47

1

Before I answer how to do so in nco, I highly recommend that you pick up python and use the netCDF4 library because manipulating netCDF files is factors easier in my opinion.

First make a new netCDF file and remove the last entry:

ncks -d time,0,335 BB_001.nc BB_001_new.nc

Then, concatenate BB_001_new.nc with BB_002.nc

ncrcat BB_001_new.nc BB_002.nc test.nc

answered Jun 14 '18 at 17:47

user1321988

513
2
6

1

I disagree; although I love Python/netCDF4/xarray, why write a complete Python script if NCO/CDO can do the same with two simple command line calls? The only reason I can think of is flexibility, in case you are dealing with a specific use case which NCO/CDO can't handle. – Bart Jun 14 '18 at 20:30
That's fair, there is certainly overhead in scripting with netCDF4. That being said, the flexibility, capability, and overall benefits in manipulating and processing data make the endeavor worth it IMO. If the user is not a developer and only needs basic commands, you are right. However, the ability to optimize batch processing with netCDF4 speeds up the implementations of even basic operations, especially when working with large multidimensional files. – user1321988 Jun 15 '18 at 20:43

score 0 · Answer 3 · answered Jun 16 '18 at 07:30

0

An equivalent solution using CDO:

cdo delete,timestep=1 BB_002.nc 2.nc
cdo mergetime BB_001.nc 2.nc out.nc

you may be able to pipe the command as follows:

cdo mergetime  BB_001.nc -delete,timestep=1 BB_002.nc out.nc

answered Jun 16 '18 at 07:30

ClimateUnboxed

7,106
3
41
86

Need assistance in Appending two netcdf files based choosing time period

3 Answers3