From fc64d7a8616cc932c0a022dc8e4ea0e53dc80be9 Mon Sep 17 00:00:00 2001 From: Jay Baxter Date: Mon, 13 Nov 2023 17:21:09 -0800 Subject: [PATCH] Update contributor-scores.md --- documentation/under-the-hood/contributor-scores.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/documentation/under-the-hood/contributor-scores.md b/documentation/under-the-hood/contributor-scores.md index d73d23a3..69a2cab1 100644 --- a/documentation/under-the-hood/contributor-scores.md +++ b/documentation/under-the-hood/contributor-scores.md @@ -26,9 +26,15 @@ This score is the average score of notes you’ve written (that have gotten at l The Rater Helpfulness Score reflects how similar a contributor’s ratings are to the ratings on notes that eventually reached the status of “Helpful” or "Not Helpful” (indicating clear widespread consensus among raters, and not labeled “Needs More Ratings”). -Only [Valid Ratings](#valid-ratings) are used in computing rater helpfulness scores. This is done to both reward quick rating, and to prevent one form of artificially gaming this score (retroactively rating old notes with clear labels). +First, we compute each raters' [Valid Ratings](#valid-ratings) are used in computing rater helpfulness scores. This is done to both reward quick rating, and to prevent one form of artificially gaming this score (retroactively rating old notes with clear labels). -Rater Helpfulness is not defined until the contributor has made at least one valid rating (defined below). Then the Rater Helpfulness Score is the fraction of their valid ratings that match the final note status label of whether the note was rated helpful or not rated helpful. +Rater Helpfulness is not defined until the contributor has made at least one valid rating (defined below). Then the Rater Helpfulness Score is equal to + +$$\frac{s - (10 * h)}{t}$$ +With the terms defined as: +* (s): the number of successful valid ratings they've made (valid ratings that matched the final note status label) +* (-10 * h): minus a penalty of 10 for note they rated helpful (valid or not) with an extremely [Tag-Consensus Harassment-Abuse Note Score](./ranking-notes.md) +* (t): divided by the total number of valid ratings they've made ## Valid Ratings