Can a Dose of Randomness Help Fairness?

As the philosophers teach, there are many ways to think about “fairness.”

One common approach is to identify a source of what is deemed to be unfairness, or a practice that leads to outcomes deemed to be unfair, and then to address this specific practice or outcome in a direct way. However, there is also a line of philosophy which suggests that thinking about “fairness” might usefully include an element of randomness. For example, the outcome of a lottery is “fair,” not in the sense that it corrects or addresses other aspects of unfairness, but in the sense that every ticket has an equal chance of winning. So, can using a random country generator or implementing a random selection process help add fairness to a decision-making process? And how might that be beneficial? The answer to these questions will depend on the context and the specific situation. But, if fairness is an important consideration in making decisions, it may be worth considering how randomness can help achieve a more equitable outcome.

Some recent research takes this connection between randomness and fairness and puts it to work. For example, in the July 2021 issue of the American Economic Review, Rustamdjan Hakimov, C.-Philipp Heller, Dorothea Kübler, and Morimitsu Kurino discuss “How to Avoid Black Markets for Appointments with Online Booking Systems” (111:7, pp. 2127-51).

They point out a number of examples of online booking systems where entrepreneurially-minded scalpers snap up many or all of the reservations, and then resell them in a secondary black market. This happened in California with prime-time appointments at the Department of Motor Vehicles, in Ireland and with the offices that immigrants need to get their residential permits and visas, in China with appointments at state-run hospitals, and so on. These were all first-come, first-served settings, which is of course a different working definition of “fairness.” However, the authors suggest an alternative mechanism. They write:

We propose an alternative system that collects applications in real time, and randomly allocates the slots among applicants (“batch” system). The system works as follows: a set of slots (batch) is offered, and applications are collected over a certain time period, e.g., for one day. At the end of the day, all slots in the batch are allocated to the appointment seekers. Thus, the allocation is in batches, not immediate as in the first-come-first-served system. In the case of excess demand, a lottery decides who gets a slot. If a slot is canceled, this slot is added to the batch in the next allocation period, e.g., the following day. Thus, the scalper cannot transfer the slot from the fake name to the customer by way of cancellations and rebookings. We show that under reasonable parameter restrictions, the scalper not entering the market is the unique equilibrium outcome of the batch system. The intuition for this result is that, keeping the booking behavior of the scalper fixed, a seeker has the same probability of getting a slot when buying from the scalper as when applying directly. Flooding the market with fake applications increases the probability that the scalper will receive many slots, but he cannot make sure that he gets slots for his clients, and he cannot transfer slots to the names of the clients.

Notice how the built-in lottery is a key part of improving the fairness of the outcome in this outcome. When you think about it, other examples of randomness as a form of fairness come to mind. For example, when public charter schools are oversubscribed, the usual practice is they are required to select students through a lottery–and in turn, this randomness gives researchers a tool for evaluating the effectiveness of charter schools by comparing those randomly admitted with those randomly not admitted. In 2008, Oregon opened up its Medicaid program to additional enrollees, but had 90,000 applicants for only 30,000 slots–so it used a lottery to choose who would get the expanded coverage.

A dollop of randomness might have other applications as well. For example, imagine its application in hiring decisions. Say that the company has five qualified internal candidates. If they are all deemed qualified, perhaps the job should be handed out by random chance. There are several possible benefits of such an approach.

One is that when there is a group of candidates who are all deemed to be qualified, it’s a time when biases can begin to creep in. After all, if everyone is qualified, then maybe it feels safer or more comfortable to go with a familiar choice, or a politically connected choice. Joël Berger, Margit Osterloh, Katja Rosta, and Thomas Ehrmann explore this possibility in “How to prevent leadership hubris? Comparing competitive selections, lotteries, and their combination” (Leadership Quarterly, October 2020).

As an historical example, they point to a problem that arose at the University of Basel when professors were often being appointed based on political connections, and adding a dose of randomness was part of the answer. They write:

This problem led to changes that took place at the University of Basel in the 18th century. Until the end of the 17th century, the appointment of professors at this university was seriously compromised by the interventions of politically influential family dynasties and by corruption. To combat this problem, a law was passed requiring the appointment of new professors through a procedure that combined competitive and random selection (Burckhardt, 1916), termed the Wahl zu Dreyen or selection from three. The law, which was introduced in 1718, required candidates to submit proof of their qualifications to the governing body of the university, which then decided whether a can-
didate was eligible or not (Burckhardt, 1916). Subsequently, all the professors of the University of Basel came together to act as the electoral authority. If two or three candidates were eligible, the candidate to be appointed was chosen by lottery. If more than three candidates came into question, the electoral authority was divided by lottery into three electoral colleges. Each college had to propose one candidate by secret voting. Finally, the candidate to be appointed was decided by
lottery.

The authors argue further that applying a degree of randomness in this way might be useful in choosing top corporate executives: that is, decide on a group of qualified candidates, and then choose by lottery. They argue that top executives may be prone to excessive hubris (and one might add, excessive pay) because they view themselves as selected. If they viewed themselves as literally lucky to be chosen from among equally capable potential replacements, they might act differently. The authors offer some experimental evidence that those who view themselves as selected to be group leaders may be more prone to hubris than those chosen from a mixture of selection and randomness.

Another possible application of randomness in a hiring context is discussed by an overlapping group of authors, Joël Berger, Margit Osterloh, and Katja Rost in “Local random selection closes the gender gap in competitiveness” (Science Advances, November 20, 2020). The authors point to a body of evidence suggesting that women in job settings are more likely to opt out of competitive settings than men, and in general to act in more risk-averse ways than men. Thus, if promotions are set up as a competitive tournament, women may be systematically disadvantaged. They argue, based on some experimental evidence, that a process in which people enter a process to be designated as qualified for a certain job, knowing in advance that the actual promotion will be determined by lottery, will attract more women without diminishing the quality of the applicant pool.

One can think of various ways of applying a dose of randomness in various academic settings, too. Imagine, for example, that an organization is evaluating grant proposals. For some of the proposals, everyone is enthusiastic. But for encouraging but marginal grants, there is disagreement over quality. When choosing among these marginal proposals, it’s easy to imagine various biases creeping in: for example, the evaluators–other things equal–might tend to favor the proposals from better-known institutions, or better-known people, or perhaps they will tend to favor more conventional ideas as opposed to higher-risk proposals that might look foolish in retrospect.

To avoid such biases, the Swiss National Science Foundation decided to use a degree of randomness when evaluating grant applications. Some grant applications evoke virtual unanimity, either about whether to fund them or whether not to do so. But what about the in-between cases? Rather than pretending that it’s possible to make fine distinctions between these applications–which is a situation where bias can easily creep in–instead a proportion of these grants are given by lottery.

Margit Osterloh and Bruno S. Frey discuss the rationale for applying a similar process to academic journals in “How to avoid borrowed plumes in academia” (Research Policy, February 2020). They note that of the economics articles published in top journals, some turn out to be highly influential and some do not. They also point to a fear that, at least in economics, being published in a top journal is viewed as proof of research of the highest quality, which (at least as judged by how often articles are later cited) is not necessarily true. Given the uncertainties involved, they propose experimentation with a partly random method of selection. If all the referees love a paper, or hate it, then the choice is simple. But for the in-between papers, they suggest random selection. They write:

Our own proposal is the most radical. It is based on the insight that fundamental uncertainty is symptomatic for scholarly work. This is indicated by the low prognostic quality of reviews and the low inter-rater reliability revealed by many empirical analyses. Our suggestion takes this evidence into account. It suggests the introduction of a partly random mechanism. Focal randomisation takes place after a thorough preselection of articles by peer reviews. Such a rationally founded and well-orchestrated procedure promises to downplay the importance (or even “tyranny”) of top journals and to encourage more unorthodox research than today.

Of course, one also sometimes hears proposals for the use of lotteries in admission to selective colleges. It’s not uncommon to hear presidents or admissions officers from such places say “we have many more qualified candidates than we can possibly admit.” So rather than turning of the decision to admissions offices, which will inevitably have shifting agendas and biases of their own about what makes a student “authentic” and full of potential, perhaps instead the admissions office should just decide who is super-qualified for the school, and who is not qualified for he school, and then admit the rest by lottery. It would be problematic for qualified applicants turned down by such a process to claim that it was unfair. But perhaps more interesting, it would be openly acknowledged that there was an element of luck in who is admitted–and the hubris that stems from being selected to a selective college might be reduced as a result.

Here’s one more example. In Nigeria, a competition called YouWiN! was launched in 2011 to provide funding to small businesses and start-ups. David McKenzie describes the process in “Identifying and Spurring High-Growth Entrepreneurship: Experimental Evidence from a Business Plan Competition” (American Economic Review, 107 (8): 2278-2307):

The YouWiN! competition was launched in late 2011 by the president of Nigeria, and in its first year attracted almost 24,000 applications aiming to start a new business or expand an existing one. The top 6,000 applications were selected for a 4-day business plan training course, and then winners were chosen to receive awards averaging US$50,000 each, paid out in four tranche payments conditional on achieving basic milestones. The top-scoring plans overall and within the region were chosen as winners automatically, and then 729 additional winners were randomly selected from a group of 1,841 semifinalists, providing experimental variation from US$34 million in grants that enables causal estimation of the program’s impact.

Again, choosing most of the grant winners at random, out of those deemed qualified, seems fair in at least one sense of the word.

The idea of randomness as a form of fairness may seem counterintuitive. Much of the time, we think of fairness as the outcome of a process of selection. But selection inevitably has biases of its own, which will be tied up in issues like what information is used in the selection process, who is more or less comfortable being part of the selection, and biases of who does the selecting. None of the examples here suggest that a desired outcome should be totally allocated at random. There’s always a screening process first. But when a high level of uncertainty exists between candidates or projects that seem very similarly qualified, there is a case for using randomness as a way to reduce some of the biases that are always likely to exist in any selection process.