Columbia International Affairs Online: Working Papers

CIAO DATE: 09/2010

Evaluating the Evaluators: Media Freedom Indexes and What They Measure

John Burgess

August 2010

National Endowment for Democracy

Abstract

All over the world, studies that rank countries by media freedom figure prominently in civil liberties debates, aid programming, foreign policy decisions, and academic research. The three most widely cited indexes—compiled by Freedom House, the International Research & Exchanges Board (IREX), and Reporters Without Borders (RSF in its French initials)— often become media events in their own right on release day, written about by local newspapers and Web sites and analyzed on television and radio. In view of the breadth and depth of their impact, academics have been studying the quality of the social science that underlies these and other studies. Some academics claim deficiencies in such issues as methodology, cultural neutrality, and focus on “old media.” Yet many go on to conclude that whatever the shortcomings, the studies produce basically consistent findings over time and are credible, useful tools for tracking the evolution of media freedom in the countries of the world. They serve to highlight the crucial role of a free press in democracy and good governance.