Correct Unexpected floats when reading LI L2 LFL

[ClementLaplace]

Makes sure that the method apply_fill_value is not applied when _FillaValue is not into the attributes. It is fixing the bug mentioned there #2854 .

• Closes Unexpected floats when reading LI L2 LFL data #2854

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.11%. Comparing base (01237e2) to head (f06d07e).
Report is 53 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2998      +/-   ##
==========================================
- Coverage   96.11%   96.11%   -0.01%     
==========================================
  Files         383      383              
  Lines       55673    55679       +6     
==========================================
+ Hits        53511    53514       +3     
- Misses       2162     2165       +3     

Flag	Coverage Δ
behaviourtests	3.89% <0.00%> (-0.01%)	⬇️
unittests	96.20% <100.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[coveralls]

Pull Request Test Coverage Report for Build 12991062112

Details

• 22 of 22 (100.0%) changed or added relevant lines in 4 files are covered.
• 3 unchanged lines in 3 files lost coverage.
• Overall coverage decreased (-0.005%) to 96.217%

---

Files with Coverage Reduction	New Missed Lines	%
satpy/tests/reader_tests/_li_test_utils.py	1	98.55%
satpy/readers/li_base_nc.py	1	99.62%
satpy/tests/reader_tests/test_li_l2_nc.py	1	99.53%

Totals
Change from base Build 12905542942:	-0.005%
Covered Lines:	53761
Relevant Lines:	55875

---

💛 - Coveralls

[ameraner]

Thanks for the investigations! I left some comments in the code.

I am also wondering if we still also need the fix for the float64 upcasting when multiplying with the scale_factor, as discussed in the issue...

[ameraner]

Thanks for the fix and the extra more comprehensive tests! Just two things inline.

[ameraner]

LGTM, thanks!

[pnuu]

One suggestion inline.

Maybe we should start splitting the long test helpers to smaller chunks at some point?

I also restarted the one windows CI run.

[pnuu]

This would prevent one unnecessary upcasting:

Suggested change

	data_2d = da.where(data_2d > 0, data_2d, np.nan).astype(np.float32)
	data_2d = da.where(data_2d > 0, data_2d, np.float32(np.nan))

[ClementLaplace]

@pnuu casting a nan who is a float32 is not enough to convert all the arrays into a float32 . For example if data_2d is an int32 the method where will convert it to a float64. To prevent it I have used the method astype(np.float32).

[pnuu]

Ok, that makes sense I guess.

But for integer data the data would still be converted to floats, which seemed to be the original problem reported #2854 , right? So should there be separate handling for the integer data?

[ameraner]

For this specific case, handling the accumulated LI 2-d data arrays, we need floats since we need to support NaN values. So I'm ok with the solution here that avoids float64.

The problem in the original issue was that some integer variables, that do not have a FillValue attribute, were still being casted unnecessarily to float; that problem is fixed by the other modification of this PR in apply_fill_value here https://github.com/pytroll/satpy/pull/2998/files#diff-3b2bff08b4001ec6f72cca67791cc4322b38e0db97e68f2109791093e56e6052R445

[pnuu]

Oh, ok.

The current version will cast the data to float64 (when setting the 64-bit nan) before recasting it to float32. So for the staying in float32 world I think the data_2d array should be converted to float32 before applying the nan:

data_2d = data_2d.astype(np.float32)
data_2d = da.where(data_2d > 0, data_2d, np.nan)

Here the np.nan respects the original dtype and does not do an intermediate up-casting for the integer data.

[ClementLaplace]

Modification applied