cdsapi error while trying to download era5 data using python script

Question

I'm trying to download ERA5 best-track data (hourly, single level) with CDS API. I'm totally new to get this kind of dataset, so I've running into bunch of errors like this.

This is the script:


import cdsapi
import numpy 
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

c = cdsapi.Client()

# Request parameters
for year in range(1945, 2022):
    c.retrieve(
        'reanalysis-era5-single-levels',
        {
            'variable': 'total_precipitation',
            'format': 'netcdf',
            'year': str(year),
            'month': [
                '01', '02', '03',
                '04', '05', '06',
                '07', '08', '09',
                '10', '11', '12',
            ],
            'day': [
                '01', '02', '03',
                '04', '05', '06',
                '07', '08', '09',
                '10', '11', '12',
                '13', '14', '15',
                '16', '17', '18',
                '19', '20', '21',
                '22', '23', '24',
                '25', '26', '27',
                '28', '29', '30',
                '31',
            ],
            'time': [
                '00:00', '01:00', '02:00',
                '03:00', '04:00', '05:00',
                '06:00', '07:00', '08:00',
                '09:00', '10:00', '11:00',
                '12:00', '13:00', '14:00',
                '15:00', '16:00', '17:00',
                '18:00', '19:00', '20:00',
                '21:00', '22:00', '23:00',
            ],
            'area': [90, -180, -90, 180], # Global
        },
        f'era5_{year}_precipitation.nc',
    )

And getting same error like this.

2023-05-14 17:42:20,413 INFO Welcome to the CDS
2023-05-14 17:42:20,414 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels
2023-05-14 17:42:20,702 INFO Request is queued
2023-05-14 17:42:21,986 INFO Request is failed
2023-05-14 17:42:21,986 ERROR Message: the request you have submitted is not valid
2023-05-14 17:42:21,986 ERROR Reason:  Mars server task finished in error; UserError: Duplicate value for month: 1945-01 [mars]; Error code is -2; Request failed; Some errors reported (last error -2)
2023-05-14 17:42:21,986 ERROR   Traceback (most recent call last):
2023-05-14 17:42:21,986 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 59, in handle_request
2023-05-14 17:42:21,986 ERROR       result = cached(context.method, proc, context, context.args, context.kwargs)
2023-05-14 17:42:21,986 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached
2023-05-14 17:42:21,986 ERROR       result = proc(context, *context.args, **context.kwargs)
2023-05-14 17:42:21,986 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 124, in __call__
2023-05-14 17:42:21,986 ERROR       return p(*args, **kwargs)
2023-05-14 17:42:21,987 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 60, in __call__
2023-05-14 17:42:21,987 ERROR       return self.proc(context, *args, **kwargs)
2023-05-14 17:42:21,987 ERROR     File "/home/cds/cdsservices/services/mars/mars.py", line 48, in internal
2023-05-14 17:42:21,987 ERROR       return mars(context, request, **kwargs)
2023-05-14 17:42:21,987 ERROR     File "/home/cds/cdsservices/services/mars/mars.py", line 20, in mars
2023-05-14 17:42:21,987 ERROR       execute_mars(context, requests, info)
2023-05-14 17:42:21,987 ERROR     File "/home/cds/cdsservices/services/mars/execute_mars.py", line 20, in execute_mars
2023-05-14 17:42:21,987 ERROR       exception=MarsException)
2023-05-14 17:42:21,987 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/context.py", line 209, in run_command
2023-05-14 17:42:21,987 ERROR       raise exception(call, proc.returncode, output)
2023-05-14 17:42:21,987 ERROR   home.cds.cdsservices.services.mars.__init__.py.exceptions.MarsException: Mars server task finished in error; UserError: Duplicate value for month: 1945-01 [mars]; Error code is -2; Request failed; Some errors reported (last error -2)
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
~/RSCH/Globe/data2/ERA5_rainfall.py in <module>
      9 # Request parameters
     10 for year in range(1945, 2022):
---> 11     c.retrieve(
     12         'reanalysis-era5-single-levels',
     13         {

/usr/local/miniconda3/lib/python3.8/site-packages/cdsapi/api.py in retrieve(self, name, request, target)
    346
    347     def retrieve(self, name, request, target=None):
--> 348         result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
    349         if target is not None:
    350             result.download(target)

/usr/local/miniconda3/lib/python3.8/site-packages/cdsapi/api.py in _api(self, url, request, method)
    504                         break
    505                     self.error("  %s", n)
--> 506                 raise Exception(
    507                     "%s. %s."
    508                     % (reply["error"].get("message"), reply["error"].get("reason"))

Exception: the request you have submitted is not valid. Mars server task finished in error; UserError: Duplicate value for month: 1945-01 [mars]; Error code is -2; Request failed; Some errors reported (last error -2).

I really don't know what the problem is here. How can I download the data without this error?

You are missing product_type. In this case the error message is incorrect, so there is a bug in the cdsapi, as they are incorrectly saying there is an error with the months — Robert Wilson, May 14 '23 at 13:15

score 1 · Answer 1 · answered May 14 '23 at 13:14

You have not defined the product type in your call. Adding it in should work, at least it does for me:

for year in range(1945, 2022):
    c.retrieve(
    'reanalysis-era5-single-levels',
    {
        'product_type': 'reanalysis',
        'variable': 'total_precipitation',
        'format': 'netcdf',
        'year': str(year),
        'month': [
            '01', '02', '03',
            '04', '05', '06',
            '07', '08', '09',
            '10', '11', '12',
        ],
        'day': [
            '01', '02', '03',
            '04', '05', '06',
            '07', '08', '09',
            '10', '11', '12',
            '13', '14', '15',
            '16', '17', '18',
            '19', '20', '21',
            '22', '23', '24',
            '25', '26', '27',
            '28', '29', '30',
            '31',
        ],
        'time': [
            '00:00', '01:00', '02:00',
            '03:00', '04:00', '05:00',
            '06:00', '07:00', '08:00',
            '09:00', '10:00', '11:00',
            '12:00', '13:00', '14:00',
            '15:00', '16:00', '17:00',
            '18:00', '19:00', '20:00',
            '21:00', '22:00', '23:00',
        ],
        'area': [90, -180, -90, 180], # Global
    },
    f'era5_{year}_precipitation.nc',
)

oops, missed that - strange it was missing, the webpage won't let you cut and paste an incomplete retrieval request. — ClimateUnboxed, May 14 '23 at 21:20

cdsapi error while trying to download era5 data using python script

1 Answers1