Real People, Statistical People, and the Probability Grid

Real People, Statistical People, and the Probability Grid

Real People, Statistical People, and the Probability Grid

In the perspective of economists versus regular individuals, a distinct contrast emerges.

Economists focus on statistical patterns rather than individual anecdotes. While non-economists tend to reason from specific instances, economists seek to discern causal connections and policy implications from broader statistical data.

I sometimes say that one difference between those who have been trained as economists and normal human beings is that economists don’t believe in real people, but instead believe in statistical people. My point is that normal humans tend to reason from examples of particular people: perhaps a person who lost their job, or a stock-picker who recommended buying Amazon back in the 2000s, or someone who received the COVID vaccine but became sick anyway. Let us stipulate that these individual people are real; indeed, they can be interviewed on camera.

But any economist will be reluctant to draw conclusions about causal connections or policy choices related to unemployment or investment strategies or vaccines from individual stories of real people. An economist will want to know about the statistics of all the people who were working, and which ones became unemployed; or the statistics that capture all the investment predictions made by a stock-picker, not just the ones that turned out well; or all the people who were vaccinated and what happened. Any single person being interviewed on camera may or may not represent the broader statistical reality. As the old saying goes, “The plural of `anecdote’ is not `data.'”

Daniel Simons and Christopher Chabris have developed a more elegant way of making this point clearly, and they offer an overview in “How the Possibility Grid Can Help You Evaluate Evidence Better” (Behavioral Scientist, July 17, 2023)

Consider the example of Mr. Pink Shirt, who recommended buying stock in Amazon and in Tesla years before the prices soared. Let us stipulate that Mr. Pink Shirt did indeed offer this advice. Should you take stock-picking advice from this person? Simons and Chabris offer a possibility grid to evaluate Mr. Pink Shirt.

The upper left-hand corner is the information presented to you: that is, Mr. Pink Shirt picked some stock market winners. The gray squares are the information not presented to you: that is, you don’t know what duds he picked, nor what winners he did not pick, nor what duds he did not pick. They write:

To avoid being deceived, we don’t need to know exactly how many stocks are in each box—just thinking about the possible contents of the full grid tells us there is no reason to believe that Mr. Pink Shirt, a guy who made two good picks in fourteen years, is worth paying attention to now. The possibility grid is a universal tool to draw attention to what is absent. It alerts you to think about rates of success rather than stories of successes. Applied to scientific research, the possibility grid reminds us that we can’t evaluate the state of the literature by tallying up only the significant results—we also have to think about the studies that failed or never got published. And it tells us to be wary when someone claims that their intervention will improve your performance or your health if they don’t show that the gains they promise are more likely to occur with than without their product’s help.

As the authors point out, when there is a great success story about someone who “just went with their gut” or “just knew what to do” or “just followed their bliss,” you can’t evaluate whether that course of action is useful to follow unless you have information about the rest of the probability grid. Sometimes people are lucky or unlucky. Sometime unlikely things do happen: an event that has only a 0.01% chance of happening will in fact actually happen one out of every 10,000 times–but you might not want to rely on it happening to you

Share this article

Leave your comments

Post comment as a guest

terms and condition.
  • No comments found

Share this article

Timothy Taylor

Global Economy Expert

Timothy Taylor is an American economist. He is managing editor of the Journal of Economic Perspectives, a quarterly academic journal produced at Macalester College and published by the American Economic Association. Taylor received his Bachelor of Arts degree from Haverford College and a master's degree in economics from Stanford University. At Stanford, he was winner of the award for excellent teaching in a large class (more than 30 students) given by the Associated Students of Stanford University. At Minnesota, he was named a Distinguished Lecturer by the Department of Economics and voted Teacher of the Year by the master's degree students at the Hubert H. Humphrey Institute of Public Affairs. Taylor has been a guest speaker for groups of teachers of high school economics, visiting diplomats from eastern Europe, talk-radio shows, and community groups. From 1989 to 1997, Professor Taylor wrote an economics opinion column for the San Jose Mercury-News. He has published multiple lectures on economics through The Teaching Company. With Rudolph Penner and Isabel Sawhill, he is co-author of Updating America's Social Contract (2000), whose first chapter provided an early radical centrist perspective, "An Agenda for the Radical Middle". Taylor is also the author of The Instant Economist: Everything You Need to Know About How the Economy Works, published by the Penguin Group in 2012. The fourth edition of Taylor's Principles of Economics textbook was published by Textbook Media in 2017.

Cookies user prefences
We use cookies to ensure you to get the best experience on our website. If you decline the use of cookies, this website may not function as expected.
Accept all
Decline all
Read more
Tools used to analyze the data to measure the effectiveness of a website and to understand how it works.
Google Analytics