Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
add balanced accuracy for SelfCheck-BERTScore
  • Loading branch information
potsawee authored Feb 9, 2024
1 parent 41290a9 commit f678a98
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions demo/experiments/balanced_accuracy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@ Balanced Accuracy

- Sentence-level balanced accuracy for the Wikibio dataset
- For these results, the threshold is set to be 0.5
- For SelfCheckGPT with BERTScore, set `rescale_with_baseline=True` to ensure that the scores are calibrated in [0.0, 1.0]
- As the metric is balanced accuracy, NonFact and Factual scenarios yield the same results

| Method | NonFact | NonFact* |
|----------------------|:------------------:|:------------------:|
| Random Guessing | 50.00 | 50.00 |
| SelfCheck-BERTScore | 59.31 | 63.42 |
| SelfCheck-QA | 62.87 | 60.08 |
| SelfCheck-NLI | 70.55 | 62.15 |
| SelfCheck-Prompt (gpt-3.5-turbo) | 76.69 | 65.93 |
Expand Down

0 comments on commit f678a98

Please sign in to comment.