0

I have a script to loop through a selection of net cdf files. The files are opened, data extracted, then closed again. I have used this many times before and it works with no issue. I was recently sent a new selection of files to run through the same code. I can check the files individually using the ncdf4 package and nc_open() function. The files look fine and are not corrupt. However, when I run through the loop the function will not let me open the files and I get this error:

Error in R_nc4_open: NetCDF: HDF error

When I run though the loop to check, all is fine and the file opens. It just cannot open in the loop. There is no issue with the code.

Has anyone come across this before with non-corrupt net cdf files getting this error only on occasion. Even outside the loop I can run the code and get the error first time, then run it again without changing anything and the connection works.

Not sure how to trouble shoot this one, so just looking for advice as to why this might be happening.

Code snippet:

targetYear <- '2005-2019'
variables <- c('CHL','SSH')
ncNam <- list.files(folderdir, '.nc', recursive = TRUE)

for(v in 1:length((variables)))
{
varNam <- unlist(unique(variables))[v]

# Get names corresponding to variable
varLs <- ncNam[grep(varNam, basename(ncNam))]
varLs <- varLs[grep(targetYear, varLs)]}
varLs <- varLs[1]
export <- paste0(exportdir,varNam,'/')
dir.create(export, recursive = TRUE)

if(varNam == 'Proximity1km' | varNam == 'Proximity200m'| varNam == 
'ProximityCoast'| varNam == 'Bathymetry'){

fileNam <- varLs
ncfilename <- paste0(folderdir, fileNam)
print(ncfilename)
  
# Read ncfile
ncfile <- nc_open(ncfilename)  
  
nc_close(ncfile) 
gc()
  
} else {

fileNam <- varLs
ncfilename <- paste0(folderdir, fileNam)
print(ncfilename)
  
# Read ncfile

ncfile <- nc_open(ncfilename)

nc_close(ncfile) 
gc()}`
mb5572
  • 27
  • 6
  • Unless you provide data and a code snippet that replicates the problem, I doubt this can be answered here. Please read https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example – dww Jun 08 '21 at 11:22

1 Answers1

0

I figured out the issue. It was to do with the error detection filer in the .nc files.

I removed the filter and the files work fine inside the loop. Still a bit strange.

Perhaps the ncdf4 package is not up to date with this filtering.

mb5572
  • 27
  • 6