How we measure democracy matters, writes Alexander Hudson. For practitioners, measuring the component parts of democracy is just as important as using an integral concept of democracy
Jean-Paul Gagnon’s search for the 'total texture of democracy' is a laudable enterprise. We cannot hope to fully identify the meaning of democracy without a reasonably comprehensive understanding of its conceptual space and the areas in which meanings cluster. The potential contributions of this endeavour to our collective knowledge of what democracy is, and how it has been (and will be) realised across time and cultures are immense. Particularly for practitioners and international donors, the liberal democratic paradigm is dominant. Our imagination of how we can realise democracy is thus quite limited. This more wide-ranging understanding of democracy’s many realisations could therefore help us identify new ways to support democratic growth in more context-sensitive ways.
a more wide-ranging understanding of democracy’s many realisations could help us identify new ways to support democratic growth
Gagnon’s project poses a particular challenge to those organisations that attempt to measure democracy. At International IDEA, we subscribe to a rather broad definition of democracy. Our organisation understands democracy to require popular control of decision making and decision makers, and equality in the exercise of that control. We measure democracy in a disaggregated way that can accommodate different institutional forms. This means that we do not come to a single value for democracy. Instead, we measure the core attributes of democracy as stand-alone values.
The Global State of Democracy Indices (GSoDI), combine 13 extant data sources (most notably the Varieties of Democracy dataset), and 116 individual indicators. We use these sources to estimate the values of five attributes and sixteen sub-attributes of democracy. The five principal attributes are representative government, fundamental rights, checks on government, impartial administration, and participatory engagement.
Like Gagnon, International IDEA does not have a unidimensional concept of democracy. Rather, our analyses focus on how countries perform in areas we consider to be fundamental aspects of democracy
The different attributes are more or less important to different conceptions of democracy (liberal, participatory, deliberative, etc). Yet we do not combine the attributes to provide a single score for democracy. Nor do we rank countries according to how democratic they are. In this sense, International IDEA’s applied approach to measuring democracy sympathises with Gagnon. Like Gagnon, we do not focus on a unidimensional concept of democracy. Rather, our analyses focus on how countries perform in areas we consider to be fundamental aspects of democracy. We have built our own 'data mountain' – or at least a hillock. We look at this hillock from different angles, depending on the questions we wish to answer.
Yet whenever one begins to measure something, one has to make choices about what is and what is not X. Contributions to this series by, inter alia, Maia Setälä and Tom Theuns have already noted this problem. One solution is to measure the component parts of the larger concept rather than the concept itself.
In the GSoDI dataset we do just that as we estimate the values of individual components of democracy. These components include relatively concrete things like freedom of expression and the effectiveness of parliaments. Yet, when estimating these values, we are conscious that we rely on judgments made by contributors to our source datasets. These contributors have decided what counts as expression that should be free (including or excluding profane, lewd, or sacrilegious content) and what counts as a legislature (including or excluding appointed or lottocratic bodies). So even at this first step of measurement we lose some of the variety of democratic forms that have been or will be possible.
However, even as information is lost in the act of measuring, measurement of democracy is desirable for many reasons. Measures of democracy can be very useful in quantitative political science studies. They are also increasingly useful to governments, international organisations, and journalists, for tracking programme effectiveness and reporting on national trends.
To take an example: In 2015, UN member states adopted the 2030 Agenda for Sustainable Development. Since then, need has grown for data on governments' progress toward Sustainable Development Goals (SDGs). Measures of democracy are especially relevant for measuring gender equality in government (SDG 5.5) and peaceful, inclusive societies (SDG 16). Of course, any measure of democracy is flawed. But it is useful to be able to say something concrete about the extent to which each country has made progress toward ensuring ‘responsive, inclusive, participatory and representative decision-making at all levels’ (SDG 16.7).
A further interesting problem for organisations that measure democracy is how countries should be clustered together. In some applications (analyses of democratic backsliding, for example), it seems useful to use some of the more common adjectives of democracy that Gagnon has collected to distinguish between, say, liberal and delegative democracies. But there are clear limits to the number of adjectivally designated forms of democracy useful in applied work and analysis. Policy makers and journalists are not especially interested in reading a report documenting more than a few types of democracy. In this sense, while there may be thousands of particular forms of democracy, to make our work sensible to a non-specialist audience, it is helpful to limit ourselves to the discussion of just a few.
There may be thousands of forms of democracy. But to make our work sensible to a non-specialist audience, we should limit ourselves to discussion of just a few
Gagnon’s project has the potential to be helpful for political theorists and those working in applied research on democracy. We might approach the 'data mountain', however, with different goals in mind. Let's return to the lepidopteran metaphor. Some of us, for example, might be most engaged in finding new butterflies and describing their characteristics. Others might want to determine how many butterflies exist in the world, and their state of health.
Much like the ongoing dangers to literal butterflies, the figurative ones are under threat, too. Democracies of all descriptions are facing increasing threats. The work to understand and describe democracy could play an important role in finding the best ways to organise people to solve the most pressing challenges of the 21st century.
Number 30 in a Loop thread on the science of democracy. Look out for the 🦋 to read more in our series
