Replace model parameter default schema versions by value read from model_parameter.metaschema.yml

[GernotMaier]

Addresses the repetition of getting / reading the model parameters schema files and removes the necessity of detailed knowledge about paths and naming of schema files (especially for those in src/simtools/schemas/model_parameters/):

• introduces a now module simtools.data_model.schema (recommend to start reviewing that)
• add several of the main schema information (paths and files) to simtools.constants
• removes / changes wherever possible the code related to schema files to use the generalized approach
• add a function to get the most recent schema version for the model parameters (or check that a given version actually exists): schema.model_parameter_schema_version. Closes Replace model parameter default schema versions by value read from model_parameter.metaschema.yml #1327
• add a small improvement to simtools.utils.general.collect_data_from_file to get the Nth document of a multi-document yaml file

Copilot reviewed 7 out of 22 changed files in this pull request and generated no comments.

Files not reviewed (15)

• tests/conftest.py: Evaluated as low risk
• tests/unit_tests/data_model/test_metadata_model.py: Evaluated as low risk
• src/simtools/applications/validate_file_using_schema.py: Evaluated as low risk
• src/simtools/applications/convert_all_model_parameters_from_simtel.py: Evaluated as low risk
• tests/unit_tests/data_model/test_data_reader.py: Evaluated as low risk
• tests/unit_tests/data_model/test_metadata_collector.py: Evaluated as low risk
• tests/unit_tests/data_model/test_validate_data.py: Evaluated as low risk
• src/simtools/model/array_model.py: Evaluated as low risk
• src/simtools/data_model/model_data_writer.py: Evaluated as low risk
• src/simtools/data_model/metadata_collector.py: Evaluated as low risk
• src/simtools/data_model/validate_data.py: Evaluated as low risk
• src/simtools/layout/array_layout.py: Evaluated as low risk
• src/simtools/data_model/metadata_model.py: Evaluated as low risk
• tests/unit_tests/data_model/test_model_data_writer.py: Evaluated as low risk
• src/simtools/utils/names.py: Evaluated as low risk

[GernotMaier]

Just did a merge from the branch db-simulation-model-refactoring - this should fix the failing integration tests.

[tobiaskleiner]

Suggest to rename the functions to make their purpose clearer. Maybe get_parameters_and_schema_files, also the following get_schema_path_for_model_parameterand get_validated_schema_version.

[GernotMaier]

OK - for me it appears clear the schema.model_parameter_schema_files gives me the schema files. But your comment shows that this is not the case for everyone, so I changed it and added everywhere the get_...

[orelgueta]

Note: merges into db-simulation-model-refactoring branch - the 3 integration tests fails until PR #1319 is reviewed an merged

Is this still correct?

[GernotMaier]

Note: merges into db-simulation-model-refactoring branch - the 3 integration tests fails until PR #1319 is reviewed an merged

Is this still correct?

No. Removed it.

[tobiaskleiner]

Thanks @GernotMaier, looks good overall. Added a minor comment.

[GernotMaier]

@tobiaskleiner - thanks for the review. I've addressed the comment and changed the function names in simtools.data_model.schema. let me know if there is anything else.

[ctao-dpps-sonarqube]

Analysis Details

0 Issues

• 0 Bugs
• 0 Vulnerabilities
• 0 Code Smells

Coverage and Duplications

• 100.00% Coverage (93.80% Estimated after merge)
• 0.00% Duplicated Code (0.00% Estimated after merge)

Project ID: gammasim_simtools_AY_ssha9WiFxsX-2oy_w

View in SonarQube