3

I'm attempting to download a daily ocean temperature reanalysis product (GLORYS2V4 from CMEMS) for the North Atlantic, but the only download option is FTP access with large (2GB) global files. Is it possible to use open_mfdataset on an FTP directory to take the subset I need without downloading all the data to my local machine? If so, how would I set up the path or url?

Currently I'm connecting to the FTP and then trying to give xarray the url, but it's interpreting the url as a path in my working directory. E.g. if I were trying to open just one file for January 1, 2000:

ftp = FTP("my.cmems-du.eu", username, password)
sample_directory = '/Core/GLOBAL_REANALYSIS_PHY_001_031/global-reanalysis-phy-001-031-grepv2-daily/2000/01'
ftp.cwd (sample_directory)

ftp_path = 'ftp://my.cmems-du.eu/Core/GLOBAL_REANALYSIS_PHY_001_031/global-reanalysis-phy-001-031-grepv2-daily/2000/01/grepv2_daily_20000101.nc'
ds = xr.open_dataset (ftp_path)

This yields a FileNotFoundError: [Errno 2] No such file or directory: b'U:\\Documents\\conda_dir\\ftp:\\my.cmems-du.eu\\Core\\GLOBAL_REANALYSIS_PHY_001_031\\global-reanalysis-phy-001-031-grepv2-daily\\2000\\01\\grepv2_daily_20000101.nc' where 'U:\Documents\conda_dir' is my working directory.

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
jmason
  • 31
  • 1

2 Answers2

0

Generally not. I do not know that particular file format. Even if you find a library that can work with that specific file format and can work with remote FTP storage, it can hardly filter the contents without downloading the whole file anyway.

The only exception can be, if the format has some kind of an index, which the library can download, find what parts of the file it needs, and then download only those parts. For example, this way you can download only some files from a remote ZIP archive. But with most file formats this is not possible.

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
0

You will not be able to access CMEMS data via ftp using xarray. If you only want to extract a spatial subset of the data, it is best to use the Wekeo service, which can be accessed through Python. Read about it here: https://wekeo.eu/web/guest/home.

You can also use motuclient: https://marine.copernicus.eu/faq/what-are-the-motu-and-python-requirements/.

Robert Wilson
  • 3,192
  • 11
  • 19