Filtering out time object in netcdf data

[gateuxx]

What happened?

I work with netcdf data and need to convert time variable to datetimeindex before using it with another package. However, function to_datetimeindex() raises an error since there are non-sensible dates such as '2091-02-29 12:00:00' which can't be coerced, as they don't exist. I struggle to filter these values out, either programmatically or manually by stating the value.

What did you expect to happen?

I expected this command to work:
data = data.drop_sel(time='2091-02-29 12:00:00')

however, I get the following error:
KeyError: "['2091-02-29 12:00:00'] not found in axis"

Snapshot of the data

<xarray.Dataset>
Dimensions:       (rlat: 84, rlon: 82, time: 1800, bnds: 2)
Coordinates:
    height        float64 ...
    lat           (rlat, rlon) float64 ...
    lon           (rlat, rlon) float64 ...
  * rlat          (rlat) float64 -4.895 -4.785 -4.675 ... 4.015 4.125 4.235
  * rlon          (rlon) float64 -9.015 -8.905 -8.795 ... -0.325 -0.215 -0.105
  * time          (time) object 2091-01-01 12:00:00 ... 2095-12-30 12:00:00
Dimensions without coordinates: bnds
Data variables:
    rotated_pole  |S1 ...
    time_bnds     (time, bnds) object 2091-01-01 00:00:00 ... 2096-01-01 00:0...
    wsgsmax       (time, rlat, rlon) float32 12.6 12.5 11.58 ... 20.03 19.42
Attributes: (12/26)
    Conventions:                    CF-1.4
    contact:                        Erik van Meijgaard, KNMI, Regional Climat...
    experiment:                     RCP2.6 run
    experiment_id:                  rcp26
    realization:                    1
    driving_experiment:             MOHC-HadGEM2-ES,rcp26,r1i1p1
                            ...
    knmi_version_comment:           v2: replaces withdrawn results with RCMVe...
    knmi_grib_path:                 mos.knmi.nl:/climreg/CXEUR12/eSSTcorr-v49...
    creation_date:                  2016-08-21T13:22:10Z
    tracking_id:                    3a225db3-bc59-4692-a354-54f036c36d95
    history:                        Wed Jan 24 09:08:39 2024: ncks -d rlat,-4...
    NCO:                            netCDF Operators version 5.1.9 (Homepage ...

Peek into the variable

<xarray.DataArray 'time' (time: 1800)>
array([cftime.Datetime360Day(2091, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(2091, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(2091, 1, 3, 12, 0, 0, 0, has_year_zero=True), ...,
       cftime.Datetime360Day(2095, 12, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(2095, 12, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(2095, 12, 30, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
    height   float64 ...
  * time     (time) object 2091-01-01 12:00:00 ... 2095-12-30 12:00:00
Attributes:
    standard_name:  time
    long_name:      time
    bounds:         time_bnds

Ideal scenario

I would like either to automatically get rid of all dates that don't exist, or at least be able to filter them manually with the drop_sel() function (or similar one).

[spencerkclark]

Thanks for the very clear question! I recommend taking a look at the convert_calendar method—I think the following should do what you are looking for:

data.convert_calendar("standard", dim="time", align_on="date")

Regarding drop_sel, for data indexed by a CFTimeIndex, xarray currently only accepts explicit cftime objects. Right now it will not automatically convert strings to dates, but we could consider changing that:

data.drop_sel(time=cftime.Datetime360Day(2091, 2, 29, 12))

[gateuxx]

Hi Clark, both commands solve exactly my problem - the former automatically solved all errors, while the other one works on case basis. Thanks a lot, appreciate your prompt answer.