Is Wikipedia Politically Biased?

Wikipedia aspires to a neutral point of view. How well does it succeed?

Shane Greenstein and Feng Zhu tackle this question in the May 2012 issue of the American Economic Review.

They conclude that many contributions are needed to reduce considerable bias and slant to something close to neutral:

”To summarize, the average old political article in Wikipedia leans Democratic. Gradually, Wikipedia’s articles have lost that disproportionate use of Democratic phrases, moving to nearly equivalent use of words from both parties, akin to an NPOV [neutral point of view] on average. The number of recent articles far outweighs the number of older articles, so, by the last date, Wikipedia’s articles appear to be centered close to a middle point on average. Though the evidence is not definitive about the causes of change, the extant patterns suggest that the general tendency toward more neutrality in Wikipedia’s political articles largely does not arise from revision. There is a weak tendency for articles to become less biased over time. Instead, the overall change arises from the entry of later vintages of articles with an opposite point of view from earlier articles.”

How do they reach this conclusion? Greenstein and Zhu focus on entries that bear on topics of importance in U.S. politics; in particular, they begin by selecting all articles in January 2011 that include ”republican” or ”democrat” as keywords. This procedure generates about 111,000 articles, and when they have dropped the articles that aren’t about U.S. politics, they have about 70,000 articles remaining.

They then rely on a process from earlier research, which selects ”1,000 phrases based on the number of times these phrases appear in the text of the 2005 Congressional Record, applying statistical methods to identify phrases that separate Democratic representatives from Republican representatives, under the model that each group speaks to its respective constituents with a distinct set of coded language. In brief, we ask whether a given Wikipedia article uses phrases favored more by Republican members or by Democratic members of Congress.”

Some of their 70,000 articles don’t include any of these phrases, and so can\’t be evaluated by this method. For the 28,000 articles they can evaluate, they find on average a Democratic slant.
”[W]hen they have a measured slant, articles about civil rights tend to have a Democrat slant (-0.16), while the topic of trade tends to have a Republican slant (0.06). At the same time, many seemingly controversial topics such as foreign policy, war and peace, and abortion are centered at zero [that is, no slant].”

They then look back at the earlier revisions of their 70,000 articles, and to keep the numbers manageable, when an article has more than 10 revisions they look only at 10. This gives them 647,000 entries, but again many of them don’t use any of the key phrases, leaving 237,000 that do include some of those phrases. They find that older revisions tend to lean more Democratic, while newer revisions and newer entries are more balanced.

Wikipedia is in many ways an extraordinary success. Greenstein and Zhu write:

”As the largest wiki ever and one of the most popular websites in the world, Wikipedia accommodates a skyrocketing number of contributors and readers. At the end of 2011, after approximately a decade of production, Wikipedia supports 3.8 million articles in English and well over twenty million articles in all languages, and it produces and hosts content that four hundreds of millions of readers view each month. Every ranking places Wikipedia as the fifth or sixth most visited website in the United States, behind Google, Facebook, Yahoo!, YouTube, and, perhaps, eBay. In most countries with unrestricted and developed Internet sectors, Wikipedia ranks among the top ten websites visited by households.” 

Any semi-serious researcher (and here I include junior-high-school students) knows that while Wikipedia can be a useful starting point, it should never be an endpoint. Instead, it can serve as a useful shortcut to finding links to other sources. But the Greenstein and Zhu evidence suggests that Wikipedia on average has found a reasonable level of political balance–although you may need to read a few related entries on the same broad topic to achieve it.