cdutil.averager could do a sanity check of the delta of the dimension bounds

[jypeter]

@dnadeau4 One of our interns was a victim of the following side effect. The time bounds were not correctly defined in a file she was using, with the bounds of each time step having the same value as the time step

 time_counter = 86400, 172800, 259200, 345600, 432000, 518400, 604800,

 time_counter_bounds =
  86400, 86400,
  172800, 172800,
  259200, 259200,
  345600, 345600,
  432000, 432000, 

The correct values should have been

 time_counter = t0, t1, ...
 time_counter_bounds =
  t0 - delta/2, t0 + delta/2,
  t1 - delta/2, t1 + delta/2,

but we had

 time_counter_bounds =
    t0, t0,
    t1, t1, 

This led to crazy (big) values when averaging over the time axis, probably due to a division by zero or something similar

>>> import cdms2, cdutil, genutil
>>> f = cdms2.open('/home/scratch01/jypeter/time_counter_bounds_pb.nc')
>>> U = f('U')
>>> import genutil
>>> genutil.minmax(U)
(-31.189350128173828, 23.517915725708008)
>>> U_avg = cdutil.averager(U, axis='t')
>>> genutil.minmax(U_avg)
(1.7976931348623157e+308, -1.7976931348623157e+308)

Let me know if you need a sample data file to reproduce this bug (or side effect)

[doutriaux1]

@jypeter it is really a user error. I guess we could add a check for empty bounds. Not sure it's really the way to go. I recommend you use cdutil.times.setTImeBoundsMonthly(data) or similar.

@dnadeau4 what do you think?

[jypeter]

I agree that it's a user (or rather meta-data) error, and that the user could call cdutil.times.setTImeBounds* if he knew that the problem was due to this. It would actually be easier for the user if there were no time bounds, that is if they were missing rather than incorrect (what you call empty)

The trouble here is that what the user sees after the averaging above, is that all the data is masked, which makes it a bit hard for a casual user to understand what has happened

>>> genutil.minmax(U_avg)
(1.7976931348623157e+308, -1.7976931348623157e+308)
>>> U_avg.shape
(1, 90, 180)
>>> import MV2
>>> MV2.count(U_avg)
0
>>> U_avg
variable_5
masked_array(data =
 [[[-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]
  ...,
  [-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]
  [-- -- -- ..., -- -- --]]],
             mask =
 [[[ True  True  True ...,  True  True  True]
  [ True  True  True ...,  True  True  True]
  [ True  True  True ...,  True  True  True]
  ...,
  [ True  True  True ...,  True  True  True]
  [ True  True  True ...,  True  True  True]
  [ True  True  True ...,  True  True  True]]],
       fill_value = 1e+20)

Hmmm, the values reported by genutil.minmax are also a bit strange and misleading, and I have created a genutil issue about that (CDAT/genutil#20)

>>> genutil.minmax(U_avg)
(1.7976931348623157e+308, -1.7976931348623157e+308)
>>> U_avg.min()
masked
>>> U_avg.max()
masked

Anyway, you can download the input data if you have time to get a closer look

[dnadeau4]

That is a bug in genutil, it should not returns any values, and write back a message that all data are masked.

@doutriaux1 are you in charge of genutil, I can see that you wrote code in 2010?

[dnadeau4]

@doutriaux1 I will look into this today or tomorrow.

[jypeter]

Note: new correct link for time_counter_bounds_pb.nc test file