User Upload of Case Data

[anagperal]

📌 References

• Issue: Closes #8695m9rdb User Upload of Case Data

⚠️ METADATA CHANGES:

• New map created with id ekmIpnv7d1t
• Datastore new keys added and others modified
• Template added to Reports --> Resources
• AFTER MERGING THIS TASK: Add as mandatory field Case Data Source (WZqZdMkPLKv) attribute in ZEBRA program in DHIS2 metadata
• AFTER MERGING THIS TASK: Delete datastore data not needed

📝 Implementation

• Dropdown to use RTSL_ZEB_OS_CASE_DATA_SOURCE_eIDSR or RTSL_ZEB_OS_CASE_DATA_SOURCE_USER_DEF as data source for case data
• When RTSL_ZEB_OS_CASE_DATA_SOURCE_USER_DEF selected then show new field as Dropzone to be able to download a template and add csv file (Template saved in resources + datastore app-config)
• Being able to download current uploaded file
• Every time a file data is uploaded to RTSL_ZEBRA_CASE program, the data related to that national event ID has to be replaced by all data in the current uploaded file → DELETE existing data and CREATE all data in file uploaded. Also delete file and fileID in datastore and replace with new file and fileID
• store EBS id or IBS id according to Event Source selected and national ID in RTSL_ZEBRA_CASE program when upload file (RTSL_ZEB_OS_CASE_DATA_SOURCE_USER_DEF selected) and save event
• VALIDATION: not allow multiple reported dates equal, only one row per same date → in a pop-up show not only errors but also correct format, and validate file format.
• If edit event: do not allow to change disease or hazard type (IBS or EBS data source)
• If a national event is marked as COMPLETED: then mark all its events in RTSL_ZEBRA_CASE (A0fHWmkFPzX) as COMPLETED too and delete entries in the datastore related to this file
• Add needed config in datastore (explained here https://docs.google.com/spreadsheets/d/11dSefsH2-yquVfl-zZmEXasIzbZ9siu8X_Fsko020OE/edit?gid=2116946059#gid=2116946059) and add hardcoded prigram indicators config from code to datastore
• Adapt visualizations impacted depending on CAse data source configured
• when sync alerts to add national ID do not remove case data and only update alerts key in the object (and lastSyncTime and lastUpdated)
• Add warning message when user changes cases data file
• Add upload case data button in event summary section

📹 Screenshots/Screen capture

CASE_DATA_TEMPLATE (1).xlsx

Screencast.from.2024-12-16.16-10-55.webm

Screencast.from.2024-12-16.16-05-52.webm

Screencast.from.2024-12-16.16-15-41.webm

Screencast.from.2024-12-23.13-12-56.webm

🔥 Notes to the tester

[9sneha-n]

Code review - wow @anagperal , thats a lot of logic! looks great, minor non-blocking changes requested.
Functional testing -

1. I feel like there is a performance degrade in both dashboard and event tracker page, it could be a local issue. We can test from a performance aspect after uploading to instance.
2. I dont know if its related to this PR, but if " Establish a coordination mechanism" date is filled, the save gives an error. If N/A is selected, it works fine.
3. We need to review the access settings of the file created in DHIS, from the code we need to set the access setting to a user group. It should not be publicly accessible. Lets ask @bhavananarayanan which would eb the correct user group.

4. Also could be a good idea if can generate a unique name for each historical data file, so the file names will be more meaningful. maybe we can append the disease outbreak id to the file name. 5. Also, as discussed seems to be some issues with program indicators in event list table in dashboard, 717 metrics and case cards in event tracker page. refer to Sneha Cases event in dev instance.

[9sneha-n]

almost same code in both getOutbreakKeyForCases and getOutbreakKey, can we abstract and reuse?

[anagperal]

In fact, I should have removed the getOutbreakKeyForCases function since only getOutbreakKey is used. Done!!

[9sneha-n]

Is casesFileTemplate is required only if hasCasesDataFile? If yes, then should this getTempate fetch be moved inside the diseaseOutbreakEvent.casesDataSource === CasesDataSource.RTSL_ZEB_OS_CASE_DATA_SOURCE_USER_DEF check?

Also then, should we have two separate form data for DiseaseOutbreakEventFormData. One of type "disease-outbreak-event" which does not have any of the case related fields and another of type "disease-outbreak-event-case-data" which has the case related fields : caseDataFileTemplete, uploadedCasesDataFile,uploadedCasesDataFileId, hasInitiallyCasesDataFile

[anagperal]

Good idea! I have moved getTemplate() only when create a new event and on edit only when diseaseOutbreakEvent.casesDataSource === CasesDataSource.RTSL_ZEB_OS_CASE_DATA_SOURCE_USER_DEF. It cannot be separated because the data is needed since the case data source is not yet known when a new event occurs. If in the future we can create a new event selecting the case data source from the outside, we should separate them into 2 forms

[9sneha-n]

should we move onSaveDiseaseOutbreakEvent and all related to logic to a seperate hook? useForm can have all the generic logic across all forms and the new hook can be specific to DiseaseOutbreakEvent

[anagperal]

Done!!

[anagperal]

Code review - wow @anagperal , thats a lot of logic! looks great, minor non-blocking changes requested. Functional testing -

1. I feel like there is a performance degrade in both dashboard and event tracker page, it could be a local issue. We can test from a performance aspect after uploading to instance.
2. I dont know if its related to this PR, but if " Establish a coordination mechanism" date is filled, the save gives an error. If N/A is selected, it works fine.
3. We need to review the access settings of the file created in DHIS, from the code we need to set the access setting to a user group. It should not be publicly accessible. Lets ask @bhavananarayanan which would eb the correct user group.

4. Also could be a good idea if can generate a unique name for each historical data file, so the file names will be more meaningful. maybe we can append the disease outbreak id to the file name. 5. Also, as discussed seems to be some issues with program indicators in event list table in dashboard, 717 metrics and case cards in event tracker page. refer to Sneha Cases event in dev instance.

Thanks @9sneha-n !!

1. Totally right, we need to test performance to try to improve it once this is on the instance
2. I have checked that the attribute is mandatory in the program, but I think it has not sense. I going to change it to not mandatory and ask @bhavananarayanan just in case.

3. and 4. Done!!

5. It seems that the new program indicators from RTSL_ZEBRA_CASE are not filtering by only events with status active, so the completed data were appearing too. I have removed completed data and I would talk with metadata team to add this filter.