🦋 How to measure democracy: A practitioner’s view

How we measure democracy matters, writes Alexander Hudson. For practitioners, measuring the component parts of democracy is just as important as using an integral concept of democracy

Jean-Paul Gagnon’s search for the 'total texture of democracy' is a laudable enterprise. We cannot hope to fully identify the meaning of democracy without a reasonably comprehensive understanding of its conceptual space and the areas in which meanings cluster. The potential contributions of this endeavour to our collective knowledge of what democracy is, and how it has been (and will be) realised across time and cultures are immense. Particularly for practitioners and international donors, the liberal democratic paradigm is dominant. Our imagination of how we can realise democracy is thus quite limited. This more wide-ranging understanding of democracy’s many realisations could therefore help us identify new ways to support democratic growth in more context-sensitive ways.

a more wide-ranging understanding of democracy’s many realisations could help us identify new ways to support democratic growth

The challenge of measuring democracy

Gagnon’s project poses a particular challenge to those organisations that attempt to measure democracy. At International IDEA, we subscribe to a rather broad definition of democracy. Our organisation understands democracy to require popular control of decision making and decision makers, and equality in the exercise of that control. We measure democracy in a disaggregated way that can accommodate different institutional forms. This means that we do not come to a single value for democracy. Instead, we measure the core attributes of democracy as stand-alone values.

The Global State of Democracy Indices (GSoDI), combine 13 extant data sources (most notably the Varieties of Democracy dataset), and 116 individual indicators. We use these sources to estimate the values of five attributes and sixteen sub-attributes of democracy. The five principal attributes are representative government, fundamental rights, checks on government, impartial administration, and participatory engagement.

Like Gagnon, International IDEA does not have a unidimensional concept of democracy. Rather, our analyses focus on how countries perform in areas we consider to be fundamental aspects of democracy

The different attributes are more or less important to different conceptions of democracy (liberal, participatory, deliberative, etc). Yet we do not combine the attributes to provide a single score for democracy. Nor do we rank countries according to how democratic they are. In this sense, International IDEA’s applied approach to measuring democracy sympathises with Gagnon. Like Gagnon, we do not focus on a unidimensional concept of democracy. Rather, our analyses focus on how countries perform in areas we consider to be fundamental aspects of democracy. We have built our own 'data mountain' – or at least a hillock. We look at this hillock from different angles, depending on the questions we wish to answer.

Measuring the component parts of democracy

Yet whenever one begins to measure something, one has to make choices about what is and what is not X. Contributions to this series by, inter alia, Maia Setälä and Tom Theuns have already noted this problem. One solution is to measure the component parts of the larger concept rather than the concept itself.

In the GSoDI dataset we do just that as we estimate the values of individual components of democracy. These components include relatively concrete things like freedom of expression and the effectiveness of parliaments. Yet, when estimating these values, we are conscious that we rely on judgments made by contributors to our source datasets. These contributors have decided what counts as expression that should be free (including or excluding profane, lewd, or sacrilegious content) and what counts as a legislature (including or excluding appointed or lottocratic bodies). So even at this first step of measurement we lose some of the variety of democratic forms that have been or will be possible.

However, even as information is lost in the act of measuring, measurement of democracy is desirable for many reasons. Measures of democracy can be very useful in quantitative political science studies. They are also increasingly useful to governments, international organisations, and journalists, for tracking programme effectiveness and reporting on national trends.

To take an example: In 2015, UN member states adopted the 2030 Agenda for Sustainable Development. Since then, need has grown for data on governments' progress toward Sustainable Development Goals (SDGs). Measures of democracy are especially relevant for measuring gender equality in government (SDG 5.5) and peaceful, inclusive societies (SDG 16). Of course, any measure of democracy is flawed. But it is useful to be able to say something concrete about the extent to which each country has made progress toward ensuring ‘responsive, inclusive, participatory and representative decision-making at all levels’ (SDG 16.7).

Measuring democracy can help us face democratic threats

A further interesting problem for organisations that measure democracy is how countries should be clustered together. In some applications (analyses of democratic backsliding, for example), it seems useful to use some of the more common adjectives of democracy that Gagnon has collected to distinguish between, say, liberal and delegative democracies. But there are clear limits to the number of adjectivally designated forms of democracy useful in applied work and analysis. Policy makers and journalists are not especially interested in reading a report documenting more than a few types of democracy. In this sense, while there may be thousands of particular forms of democracy, to make our work sensible to a non-specialist audience, it is helpful to limit ourselves to the discussion of just a few.

There may be thousands of forms of democracy. But to make our work sensible to a non-specialist audience, we should limit ourselves to discussion of just a few

Gagnon’s project has the potential to be helpful for political theorists and those working in applied research on democracy. We might approach the 'data mountain', however, with different goals in mind. Let's return to the lepidopteran metaphor. Some of us, for example, might be most engaged in finding new butterflies and describing their characteristics. Others might want to determine how many butterflies exist in the world, and their state of health.

Much like the ongoing dangers to literal butterflies, the figurative ones are under threat, too. Democracies of all descriptions are facing increasing threats. The work to understand and describe democracy could play an important role in finding the best ways to organise people to solve the most pressing challenges of the 21st century.

Number 30 in a Loop thread on the science of democracy. Look out for the ðŸ¦‹ to read more in our series

This article presents the views of the author(s) and not necessarily those of the ECPR or the Editors of The Loop.

Author

photograph of Alexander Hudson
Alexander Hudson
Democracy Assessment Specialist, Democracy Assessment Unit, International IDEA, Stockholm

As part of the team that produces the Global State of Democracy Indices and Global State of Democracy Report, Alexander contributes to data collection, analysis and visualisation work.

Prior to joining International IDEA, he was a postdoctoral research fellow at the Max Planck Institute for the Study of Religious and Ethnic Diversity.

Alexander has also taught courses at the University of Texas at Austin, Fundação Getúlio Vargas Rio de Janeiro Law School, and Georg-August-Universität Göttingen.

His research has been published by Cambridge University Press, Political Research Quarterly, Comparative Politics, Electoral Studies, and Democratization, among other outlets.

He tweets @a2hudson

Read more articles by this author

Share Article

Republish Article

We believe in the free flow of information Republish our articles for free, online or in print, under a Creative Commons license.

Creative Commons License

Comments

Leave a Reply

Your email address will not be published.

The Loop

Cutting-edge analysis showcasing the work of the political science discipline at its best.
Read more
THE EUROPEAN CONSORTIUM FOR POLITICAL RESEARCH
Advancing Political Science
© 2020 European Consortium for Political Research. The ECPR is a charitable incorporated organisation (CIO) number 1167403 ECPR, Harbour House, 6-8 Hythe Quay, Colchester, CO2 8JF, United Kingdom.
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram