Paper, Tags: #physics
Belonging to a prestigious group affects the way you're viewed by others. In this paper I use a simple mathematical model to explore the implications of this 'prestige bias' when candidates undergo repeated rounds of evaluation. Candidates who are evaluated most highly are admitted to a 'prestige class', and their membership biases future rounds of evaluation in their favour.
I use the language of Bayesian inference to describe this bias, and show that the weight given to the prior expectation associated with a candidate's class becomes stronger with each round. The strength of the prestige bias after many rounds undergoes a first-order transition as a function of the precision of the examination on which the evaluation is based.
The primary viewpoint is that a major function of prestige is to affect the way a person is evaluated by others. It has been long recognized that academic science exhibits a 'rich get richer' effect, wherein prestige enables opportunity and opportunity enables greater prestige.
In this paper, the major goal is to suggest a model for thinking about prestige bias using the language of Bayesian inference across iterated rounds of evaluation.
In terms of the evaluation of candidates, Bayesian inference provides a way to incorporate information from an imprecise examination of the candidate with prior knowledge about the class from which the candidate comes. This process, when implemented optimally, maximizes the probability for the evaluator to select the best candidate. But it also provides a strong advantage for candidates from the prestige class, and can lead to persistent biases.
In this model, a number N of candidates undergo a process of repeated evaluation. For simplicity, candidates are only characterized by x, 'ability', constant and unchanging in time. Prob(x) : normal distribution. The goal of the evaluation process is to identify candidates with the highest ability. Each round begins with candidates taking an exam. This exam is an unbiased estimation of the ability x of the candidate, with finite precision p.
Prob(s|x) = N(s: u=x, o2=p2)
After each round, the candidates are given a ranking depending on their exam score. In the first round the evaluators have no prior expectation about each candidate. The top scoring fraction f are admitted into the 'prestige class'. I fix f = 0.05.
In the next rounds, the evaluator knows both the candidate's exam score s and whether the candidate belongs to the prestige class.
Prob(x|s) = prob(s|x) * prob_prior(x) / prob(s)
prob_prior is the prior probability obtained from knowing if the candidate belongs to the prestige class or not. From this equation we can calculate the expectation value of the candidate's ability: estimated ability.
But how is the prior distribution formed in the mind of the evaluator? In general, the prior for a given round of evaluation is based on the abilities, or perceived abilities, of the class to which the candidate belongs. One can consider two extreme cases for how an evaluator might form their perception:
- Case 1: prior based on perfect knowledge. The evaluator knows with perfect accuracy the distribution of abilities within each class. After each round of evaluation, the evaluator becomes well acquainted with the members of each class and comes to understand their true abilities. The evaluator has a perfect accurate estimation of the mean and variance in ability for the candidate's class, which can be used in the next roudn
- Case 2: prior based on past evaluation. more realistic approach, evaluators acquire no new knowledge about the candidates after the evaluation so they don't have enough information to form an accurate prior, but they might conclude they do so, since they explicitly estimated their abilities during the last round of evaluation.
The major simplification until now is that the distributions are normal, which isn't the case in reality.
Formulas in paper page 2.
The rate at which candidates are \promoted" to the prestige class, or \demoted" from it, depends on the overlap of the distributions of estimated ability " for the two classes.
The paper explores how the distributions of actual ability and estimated ability evolve over successive rounds of evaluation, as well as the rate of exchange between the 2 classes.
Distributions of actual and estimated ability quickly converge to a steady state under increased iteration.
Unlike in case 1, the recursive use of past evaluations can lead to a transition in the weight given to prior evaluations.
There's a first-order transition in the way the evaluation operates after many rounds of evaluation. If the exam used for evaluation is less precise than a certain critical value, there's a 'freezing out' of the prestige class, which blocks the turnover of candidates between classes regardless of thir ability.
The biggest message in the paper is that evaluators must be conscientious about the basis for their prior assumptions about a candidate's class.
A candidate's ability changing over time, demogaphic bias, etc. aren't considered in this model.