Question level quality check for Question Reviewers

[tarang27]

Hi All,

Sharing a brief note on the new feature that we have planned as part of Prashnavali solution on VDN for the 5.0 release.

Functionality:
A question reviewer is reviewing each question across some defined parameters before submitting for approval

Challenges in the current solution flow:

Ensuring a certain level of quality of questions is paramount for any education body creating questions for any purpose, and especially for assessment purposes. Hence to ensure good quality standards are maintained, there are some steps that each reviewer is asked to go through before approving a question. In Haryana the review process is divided into 4 steps, which involves:-

1. The reviewer opens Prashnavali and opens a question submitted by the contributor

In this step the reviewer goes through the question statement, the options of the question (only in case of MCQ questions), the correct answer of the question & the tags added to the question (topic, competency, marks & skill)

2. The reviewer opens a Google sheet (sample) shared with them and marks the questions against the specified parameters

In this step, the reviewer pastes the question statement from Prashnavali to the given google sheet, and gives a score (out of 5) for all the parameters specified in the sheet. Based on a pre-defined logic a total score is generated for the question and ‘Approved’ or ‘Rejected’ is displayed on the google sheet

3. The reviewer goes back to Prashnavali and marks the question as ‘Submit for approval’ or ‘Request changes’ based on the result from step 2.

4. The admin goes through the filled sheets in order to ensure that each question undergoes a quality check before approval

In the above mentioned steps, there are 4 major pain points:-

1. Multiple google sheets (25+) have to created for every assessment (Bi-monthly)
2. Reviewers have to navigate between Prashnavali & google sheets to review a single question and this exercise is repeated for all questions (~30 questions are added per subject-grade combination)
3. Navigating on different platforms is not ideal for reviewers as the cadre of reviewers are not very tech savvy and are mostly using mobile phones to carry out the entire process
4. The admin has to go through all the google sheets to maintain high quality standards of assessments

Functional Solution:-
Allow users to get a form/pop-up window where they can mark (yes/no or a checkbox) against all parameters to denote if they have gone through all the parameters before reviewing the question & also allow them to add a score against each parameter which with a pre-defined logic enables/grey-out the submit for review or request changes button

Detailed PRD for this solution has been created :- https://project-sunbird.atlassian.net/wiki/spaces/PRD/pages/3197304834/Question+level+quality+check+for+Question+Reviewers

Would request you to go through it and share your comments
@pallakartheekreddy @surendrasinghs

[Prerna789]

Would request your inputs on the tech design approach
@vrayulu
@maheshkumargangula

[tarang27]

Hi @surendrasinghs

As discussed on yesterday's call, sharing the screenshot of the pop-up where score for all the quality parameters needs to be entered.
Please share feedback & inputs on the same.