When Fewer People Answer Surveys, What Should Government Statisticians Do?

Back in 1790, when Congress was arguing about the process for the first Census, one argument was that the Census should limit itself to counting heads, for purposes of determining how many representatives each state should receive.

But James Madison argued that it was important to seize the opportunity of the Census to gather additional information, as he put it, “in order that they might rest their arguments on facts, instead of assertions and conjectures.”

I suppose there was a time, pre-internet, when being surveyed by a government agency or a political campaign might have felt like the exciting part of your day. But those days are long behind us. But as people become less willing to answer survey questions, the database for understanding our society and policy choices diminishes.

Here’s a figure from the US Census Bureau about the decline in response rates to surveys over just the last decade. For the uninitiated, ATUS is the American Time Use Survey; CE is the Consumer Expenditures Survey, which can be carried out either by people keeping a diary or by an interviewer; CPI-Housing is a survey for measuring what people spend on housing for inclusion in the Consumer Price Index; CPS is the Current Population Survey uses for many purposes, including measurement of employment; and TPOPS is the Telephone Point of Purchase Survey used to gather consumption data.

The obvious problem is that lower response rates are probably not randomly distributed across the population: as a result, lower responses mean that the numbers will include a potentially changing degree of bias over time.

What might be done? As one step, the Census Bureau is experimenting with survey methods that can be conducted via the internet, rather than via interviewers. But an array of other approaches are under consideration, including use of “administrative” data that the government collects for an array of other purposes: Social Security, K-12 attendance and grades, unemployment insurance data, Medicare and Medicaid data, and others. There are also efforts to see if private-sector sources of data should be incorporated. Of course, new data sources may not be directly comparable to earlier data, which raises additional issues. But government spending on data collection is about 0.18% of the federal budget: to be clear, that’s not 18%, but less than one-fifth of 1 percent. We could jack that spending all the way up to a hefty, say. 0.25% of total spending, and it would still be indistinguishable from rounding error in the federal budget.