MLVU aggregate function is incorrect

[SCZwangxiao]

According to results in their leaderboard, the overall score is calculated by the average of each subtype accuracy, not the sample accuracy.

lmms-eval/lmms_eval/tasks/mlvu/utils.py

Lines 117 to 122 in d294646

	total_correct = 0
	total_answered = 0
	for k, v in category2score.items():
	total_correct += v["correct"]
	total_answered += v["answered"]
	eval_logger.info(f"Overall Performance: {100 * total_correct / total_answered if total_answered > 0 else 0 : .1f}%")

[kcz358]

Hi @shuyansy , do you mind take a look whether this logic is correct? Thanks!

[kcz358]

Hi @SCZwangxiao , this issue should have been fixed in #555