Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve population error #453

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Improve population error #453

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
084ae69
Update population warning on model fitting
rtesra Nov 19, 2024
d04059c
Merge branch 'master' into improve-warnings
rtesra Nov 19, 2024
c1ce23a
Merge branch 'master' into improve-warnings
r-ash Nov 19, 2024
879bbaa
Update error message to make it more understandable
r-ash Nov 20, 2024
cd1b566
Translate the error messages
r-ash Nov 21, 2024
d11c86e
make error more infromative
rtesra Nov 29, 2024
b17b74d
Merge branch 'master' into improve-warnings
rtesra Nov 29, 2024
82fbc90
fix failing test
rtesra Nov 29, 2024
e1df8fd
Merge branch 'improve-warnings' of https://github.com/mrc-ide/naomi i…
rtesra Nov 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion NEWS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# naomi 2.10.2
naomi 2.10.3

* Improve error generated when user fits model at admin level where no population data exists.

naomi 2.10.2
* Add ANC testing outputs to T4 projection period for including in PEPFAR datapack output.
* Rename Datapack input CSV in the output ZIP folder for 2025 to `"pepfar_datapack_indicators_2025.csv"`.
* Add ANC testing indicators to T4 projection reprsenting the end of one year COP planning
Expand Down
18 changes: 18 additions & 0 deletions R/model.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,24 @@ naomi_model_frame <- function(area_merged,
## of a Spectrum file and then calibrated. Currently no way to know if areas
## comparise only part of a Spectrum file, so can't address.

if (!all(mf_areas[["area_id"]] %in% unique(population_agesex$area_id))) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the error message is about the area level selected, why is the conditional here referring to the area IDs (not the area level)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't area level available in the population data to easier to check that these intersect and then pull that in if they don't for the error message if needed


area_label <- area_merged |> sf::st_drop_geometry() |>
dplyr::select(area_level_label, area_level, area_id)

# Get level label for pop data
pop_label <- population_agesex |>
dplyr::left_join(area_label, by = dplyr::join_by(area_id))
pop_level <- unique(pop_label$area_level_label)

# Get area level label for model estimates
model_level <- unique(area_label[area_label$area_level== level,]$area_level_label)

stop(t_("MISSING_POP_LEVEL",
list(pop_level = paste(pop_level, collapse = ", "),
model_level = model_level)))
}

pop_subset <- dplyr::filter(population_agesex, area_id %in% mf_areas[["area_id"]])
pop_t1 <- interpolate_population_agesex(pop_subset, calendar_quarter1)
pop_t2 <- interpolate_population_agesex(pop_subset, calendar_quarter2)
Expand Down
3 changes: 2 additions & 1 deletion inst/traduire/en-translation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
"PROGRESS_CALIBRATE": "Calibrating outputs - {{elapsed}} elapsed",
"PROGRESS_CALIBRATE_SAVE_OUTPUT": "Saving outputs - {{elapsed}} elapsed",
"PROGRESS_CALIBRATE_GENERATE_REPORT": "Generating report - {{elapsed}} elapsed",
"NO_ART_DATA_FOR_QUARTER": "No ART data found for quarter {{calendar_quarter}}.\nSet 'Include ART data' to 'No' if you intend to include no ART data.",
"NO_ART_DATA_FOR_QUARTER": "No ART data found for quarter {{calendar_quarter}}.\nIf you do not intend to include ART data set 'Include ART data' to 'No'.",
"ANC_ON_ART_GREATER_THAN_TOTAL_POSITIVE": "ANC testing on ART greater than ANC testing total positive.",
"ANC_DATA_MISSING_FOR_YEAR": "ANC testing data not found for year {{missing_year}}.",
"ANC_DATA_MISSING_FOR_YEAR_PLURAL": "ANC testing data not found for years {{missing_year}}.",
Expand Down Expand Up @@ -276,5 +276,6 @@
"DOWNLOAD_AGYW_DESCRIPTION": "Naomi AGYW tool uploaded from Naomi web app",
"NUMBER_ON_ART": "Number on ART",
"NUMBER_ON_ART_DESC": "Number on ART description",
"MISSING_POP_LEVEL": "Unable to generate model estimates at the {{model_level}} level because population data only available at the {{pop_level}} level/s. Please review model options or population data inputs.",
"POPULATION_PROPORTION": "Population proportion"
}
2 changes: 2 additions & 0 deletions inst/traduire/fr-translation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,5 +273,7 @@
"DOWNLOAD_COMPARISON_DESCRIPTION": "Rapport de comparaison Naomi téléchargé à partir de l'application web Naomi",
"NUMBER_ON_ART": "Nombre de personnes sous TARV",
"NUMBER_ON_ART_DESC": "Number on ART description",
"MISSING_POP_LEVEL": "Impossible de générer des estimations de modèle au niveau {{model_level}} car les données de population ne sont disponibles qu'au niveau {{pop_level}}. Veuillez revoir les options du modèle ou les données démographiques.",
"POPULATION_PROPORTION": "Proportion de la population"

}
1 change: 1 addition & 0 deletions inst/traduire/pt-translation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,5 +273,6 @@
"DOWNLOAD_COMPARISON_DESCRIPTION": "Relatório de comparação Naomi carregado a partir da aplicação web Naomi",
"NUMBER_ON_ART": "Nombre de personnes sous TARV",
"NUMBER_ON_ART_DESC": "Number on ART description",
"MISSING_POP_LEVEL": "Não é possível gerar estimativas do modelo ao nível {{model_level}} porque os dados da população só estão disponíveis ao nível {{pop_level}}. Reveja as opções do modelo ou as entradas de dados da população.",
"POPULATION_PROPORTION": "Proporção da população"
}
23 changes: 23 additions & 0 deletions tests/testthat/test-02-model-options.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,3 +298,26 @@ test_that("Handle backwards regression when T4 and T5 options are missing", {
expect_equal(t5 - t3, 9)

})

test_that("Population data available for area level set in model options", {

expect_error(naomi_model_frame(a_area_merged,
demo_population_agesex,
a_spec,
scope = "MWI_1_1_demo",
level = 3,
calendar_quarter1 = "CY2016Q1",
calendar_quarter2 = "CY2018Q4",
calendar_quarter3 = "CY2019Q2",
calendar_quarter4 = "CY2022Q3",
calendar_quarter5 = "CY2023Q3",
artattend = FALSE,
spectrum_population_calibration = "none",
psnu_level = NULL),
paste("Unable to generate model estimates at the District level because",
"population data only available at the District + Metro level/s.",
"Please review model options or population data inputs."),
fixed = TRUE)


})
31 changes: 25 additions & 6 deletions tests/testthat/test-04-model-frame.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ test_that("artnum_mf() returns expected number of records", {

test_that("artnum_mf() throws errors for invalid inputs", {
expect_error(artnum_mf("CY1924Q4", demo_art_number, a_naomi_mf),
"No ART data found for quarter CY1924Q4.\nSet 'Include ART data' to 'No' if you intend to include no ART data.")
"No ART data found for quarter CY1924Q4.\nIf you do not intend to include ART data set 'Include ART data' to 'No'.")
expect_error(artnum_mf("CY2016Q1", demo_art_number, "jibberish"))
expect_error(artnum_mf(c("CY2016Q1", "CY2016Q2"), demo_art_number, "jibberish"))
})
Expand All @@ -53,7 +53,26 @@ test_that("artnum_mf() works with single quarter ART data", {
})


test_that("Informative error displayed when model run to admin level higher/lower than population data supplied", {
x <- expect_error(
naomi_model_frame(a_area_merged,
demo_population_agesex,
a_spec,
scope = "MWI",
level = 3,
calendar_quarter1 = "CY2016Q1",
calendar_quarter2 = "CY2018Q4",
calendar_quarter3 = "CY2019Q2",
calendar_quarter4 = "CY2022Q3",
calendar_quarter5 = "CY2023Q3"))

expect_equal(
x$message,
paste("Population data not available for admin level selected",
"for model projections. Please review model options",
"selection to ensure that area level selection is correct.")
)
})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the error message here quote the area level and area level label selected?

"Population data not available for selected area level 3 (District). Pleas review model options"

Also I'm think I would remove the "for model projections". We usually use "projections" to refer to the T3/T4/T5 projection, but here this area level selection pertains to everything.


test_that("population calibration options", {

Expand Down Expand Up @@ -85,7 +104,7 @@ test_that("population calibration options", {
mf_none$mf_model$population_t2 +
mf_none$mf_model$population_t3 +
mf_none$mf_model$population_t4 +
mf_none$mf_model$population_t5
mf_none$mf_model$population_t5
),
sum(mf_none$spectrum_calibration$population_raw))

Expand All @@ -102,7 +121,7 @@ test_that("population calibration options", {
calendar_quarter2 = "CY2018Q4",
calendar_quarter3 = "CY2019Q2",
calendar_quarter4 = "CY2022Q3",
calendar_quarter5 = "CY2023Q3",
calendar_quarter5 = "CY2023Q3",
spectrum_population_calibration = "national")

expect_false(sum(mf_nat$spectrum_calibration$population_raw) ==
Expand Down Expand Up @@ -132,7 +151,7 @@ test_that("population calibration options", {
calendar_quarter2 = "CY2018Q4",
calendar_quarter3 = "CY2019Q2",
calendar_quarter4 = "CY2022Q3",
calendar_quarter5 = "CY2023Q3",
calendar_quarter5 = "CY2023Q3",
spectrum_population_calibration = "subnational")

expect_false(sum(mf_subnat$spectrum_calibration$population_raw) ==
Expand Down Expand Up @@ -162,7 +181,7 @@ test_that("population calibration options", {
calendar_quarter2 = "CY2018Q4",
calendar_quarter3 = "CY2019Q2",
calendar_quarter4 = "CY2022Q3",
calendar_quarter5 = "CY2023Q3",
calendar_quarter5 = "CY2023Q3",
spectrum_population_calibration = "jibberish"),
"spectrum_calibration_option \"jibberish\" not found."
)
Expand Down Expand Up @@ -319,7 +338,7 @@ test_that("naomi_model_frame() interpolated population depends on quarter specif
calendar_quarter2 = "CY2018Q4",
calendar_quarter3 = "CY2019Q2",
calendar_quarter4 = "CY2022Q3",
calendar_quarter5 = "CY2023Q3",
calendar_quarter5 = "CY2023Q3",
spectrum_population_calibration = "subnational")


Expand Down
Loading