1

I am trying to extract certain variables from netcdf files. The following code works if I apply it to a single file:

ncks -C -F -d nj_u,151,152,1 ni_u,234,235,1 -v vel_u 20091208000001.nc testU.nc

See also question: Hyperslab of a 4D netcdf variable using ncks for explanation. Now I want to use this code on several .nc files with following names:

20091208000001.nc
20091208000002.nc
20091208000003.nc

I tried the following loop:

# !bin/bash
for ((x=1;x<=3;x+=1))
do
ncks -C -F -d nj_u,151,152,1 ni_u,234,235,1 -v vel_u 2009120800000$x.nc testU.nc
done

I get the error 

ncks: ERROR received 4 filenames; need no more than two

How do I get the loop to only extract from one file at a time and then append the extracted output from all the files into a single output file?

Jellyse
  • 839
  • 7
  • 22
  • For every loop iteration, each of the files from 20091208000001.nc to ..3.nc is processed. So it basically is one file at a time. What is wrong with this – Inian Feb 07 '19 at 10:22
  • I don't know it should work. That's why I posted it. I get this error 4 times in my output. – Jellyse Feb 07 '19 at 10:28

4 Answers4

2

I believe the words ni_u,234,235,1 were mistaken as another filename. You would need another -d before that.

And if you are processing multiple nc files, you might want to rename testU.nc so that they don't overlap, or you could use ncrcat to concatenate into one single file. E.g.

ncrcat -C -F -d nj_u,151,152,1 -d ni_u,234,235,1 -v vel_u 2009120800000?.nc testU.nc
Packard CPW
  • 339
  • 1
  • 5
  • 16
  • If this was the reason (which makes sense, BTW), then the claim made in the question that the code works fine out of the loop would be false :-( – Poshi Feb 08 '19 at 14:16
  • @Poshi yes, I know, it worked fine the day before, I did not see the -d was missing. Lesson learned ... – Jellyse Feb 08 '19 at 15:06
  • That's the reason I was asking you to rerun the command, and me writing the exact command you had to test, for you to just cut and paste and check the statement was true. BTW, it would be nice to correct the question so the answer makes sense. – Poshi Feb 08 '19 at 15:22
1

I see a couple errors in your script, but nothing that could lead to your actual error.

  • The shebang line should not contain space and the path should be absolute

  • There's a comma in the for condition that should be a semicolon

    #!/bin/bash

for ((x=1;x<=3;x+=1))
do
    ncks -C -F -d nj_u,151,152,1 ni_u,234,235,1 -v vel_u 2009120800000$x.nc testU.nc
done

When I prepend echo to the command you want to run, I get this result:

ncks -C -F -d nj_u,151,152,1 ni_u,234,235,1 -v vel_u 20091208000001.nc testU.nc
ncks -C -F -d nj_u,151,152,1 ni_u,234,235,1 -v vel_u 20091208000002.nc testU.nc
ncks -C -F -d nj_u,151,152,1 ni_u,234,235,1 -v vel_u 20091208000003.nc testU.nc

Three invocations with a single file each. That code is working. It looks like there's something else. Are you simplifying your code or showing us the full code?

Poshi
  • 5,332
  • 3
  • 15
  • 32
  • I corrected the mistakes, still the same error. I use bash namescript.sh to run it. – Jellyse Feb 07 '19 at 10:28
  • @Jellyse I updated my answer with the result I'm obtaining. – Poshi Feb 07 '19 at 13:19
  • This is litteraly my complete code in the file testing.sh The folder only has those three files and the testing.sh file. Is it possible the nco is not correctly installed? – Jellyse Feb 07 '19 at 13:33
  • Ok I reinstalled it, still the same error. The only thing I can think of is that somewhere on my computer there is another file with the same name and that's why it's confused. – Jellyse Feb 07 '19 at 13:50
  • @Jellyse, prepend `echo` to your command (`echo ncks -C......`) and check the output. Having other files with the same name in the computer should not cause this error. – Poshi Feb 07 '19 at 13:58
  • I get the exact same thing as you when I do echo ncks ... (no errors) but the file testU.nc does not get created. – Jellyse Feb 07 '19 at 14:10
  • No, with the `echo` you are getting the command that will be executed, but it is not executed, so no creation fo the result. If you are getting the same output than me, it means that it is being generated properly. What happens when you run one of this commands alone? Try just running `ncks -C -F -d nj_u,151,152,1 ni_u,234,235,1 -v vel_u 20091208000001.nc testU.nc`. Did you get the same error message? – Poshi Feb 07 '19 at 14:20
  • it works just fine when I do that. Nicely have my testU.nc with the data I need. Thank you for helping me btw. – Jellyse Feb 07 '19 at 15:18
  • This does not make sense. There should be no difference between running the command yourself or letting the loop run it for you. It is the same command. Last thing I cant advise is that you `set -x` before the loop, run it, and check carefully the debug output. The behaviour you describe cannot be reproduced and conceptually does not make sense either :-/ – Poshi Feb 07 '19 at 15:37
1

@Packard is right on both counts. Moreover, the stride of 1 is default and thus not needed. Hence

ncrcat -C -F -d nj_u,151,152 -d ni_u,234,235 -v vel_u 2009120800000${x}.nc testU${x}.nc

Charlie Zender
  • 5,929
  • 14
  • 19
0

I edited the code above according to the dimensions I wanted (lat, lon) 

ncrcat -C -F -d nj_u,151,152,1 -d ni_u,234,235,1 -v vel_u 2009120800000?.nc testU.nc

and the feedback was this:

HINT: If operation fails, try multislabbing (http://nco.sf.net/nco.html#msa) wrapped dimension using ncks first, and then apply ncrcat to the resulting file

user12920237
  • 1