Reduce warnings in tests

[heikoklein]

This issue is a long term issue to address warnings appearing in the code, i.e. caused by deprecations.
Checklist:

• In pyproject.toml turn warnings into errors, i.e. uncomment filterwarnings section
  

  pyaerocom/pyproject.toml

  Lines 105 to 117 in d714b10

  	## uncomment to raise errors from warnings
  	#filterwarnings = [
  	# # all warnings are errors
  	# "error",
  	# "ignore::pytest.PytestUnraisableExceptionWarning",
  	# # except deprecation and future warnings ouside this packege
  	# 'ignore::PendingDeprecationWarning:^(?!pyaerocom|tests).*:',
  	# 'ignore::DeprecationWarning:^(?!pyaerocom|tests).*:',
  	# 'ignore::FutureWarning:^(?!pyaerocom|tests).*:',
  	# # and not on this list
  	# "ignore:.*please install Basemap:UserWarning:geonum.*:",
  	# "ignore:Using DEFAULT_SPHERICAL_EARTH_RADIUS:UserWarning:iris.*:",
  	#]

• Turn all errors into new issues to be done until next milestone
• Move this issue to a issue 2 milestones/month ahead.

[heikoklein]

Remove also warning:

  Expected `tuple[str, ...]` but got `list` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_python(

[lewisblake]

Remove also warning:

  Expected `tuple[str, ...]` but got `list` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_python(

No, do not remove this warning. It is telling the user important information that their config file is misconfigured. This is the intended functionality and should remain in place.

[heikoklein]

@lewisblake Sorry, to disagree. There are two things which make this warning wrong:

1. The warning does not say which field is a supposed to be a tuple. It doesn't even say which Setup is wrong.
2. and maybe more important. Configs are written dynamically, i regularly add/remove items to config lists before I send them to one of the Setups. The config-dictionary should be flexible. That you convert/deepcopy the config inside of collocation-setup to something immutable is fine, but before it is send, it needs to be flexible.

[heikoklein]

@lewisblake Reading the manual of pydantic: https://docs.pydantic.dev/latest/concepts/conversion_table/, pydantic automatically converts iterators to tuples as one would expect in python. Above mentioned warning does not come from a modellers cfg-file, but must be generated by an error inside of the EvalSetup/CollocationSetup (or another class using BaseModel).

The code below highlights the problem. An iterator given to a tuple field in ys is converted to a tuple.
Only when ys2 uses a list (intentional bug) in the internal default-value, the pydantic validator/converter is circumvented and the error is given when model_dump() is called.

from pydantic import BaseModel, field_validator


class Years(BaseModel):
    years: tuple[int, ...] = []


cfg = {"years": range(2022, 2024)}

ys = Years(**cfg)
ys.model_dump()
print(ys.years)
# (2022, 2023)

ys2 = Years()
# []
ys2.model_dump()
# Pydantic serializer warnings:
#  Expected `tuple[int, ...]` but got `list` - serialized value may not be as expected

[lewisblake]

Yes, we knew the issue was in model_dump() based on just the warning. But given that we had neither ruled out user error nor had a bug report it made it a little hard to justify investing too much time upfront. This is an important fix though, because we need to uphold serializability and recoverability of the config files so that experiments are reproducible based on the config json, so thanks for the PR.

[heikoklein]

I've just seen a new iris-warning:

/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-package
s/iris/common/mixin.py:195: FutureWarning: You are using legacy date precision f
or Iris units - max precision is seconds. In future, Iris will use microsecond p
recision - available since cf-units version 3.3 - which may affect core behaviou
r. To opt-in to the new behaviour, set `iris.FUTURE.date_microseconds = True`.

[lewisblake]

Realted to #1490