Frequently asked questions about the Perceptions of Electoral Integrity (PEI) Index

Figure 1: Electoral Integrity around the world (2012-2014)
Source: Electoral Integrity Project. 2016. The expert survey of Perceptions of Electoral Integrity, Release 4.5 (PEI 4.5)

Part of the Electoral Integrity Project's mission is to to raise awareness about the important issues pertaining to electoral integrity globally. With this goal in mind we engage academics, practitioners, and the wider public through outreach events. Examples are the Lisbon Forum, the deliberations surrounding the Sustainable Development Goals (SDGs) at the UN, the ECPR Joint Sessions in Warsaw, or the second Asian Electoral Stakeholder Forum in Dili. More information about previous and upcoming outreach here.

One task at such events is to present the EIP's annual report and explain the Perceptions of Electoral Integrity (PEI) Index as a novel tool for measuring electoral integrity. One tool among many others, to be sure. 

A number of questions about the PEI and its methodology are asked repeatedly at such events. Enough to prompt us to collate the answers to some of the most common questions below. We keep a record of these FAQs here and will update these if and when new questions emerge. Please do not hesitate to contact us directly via electoral.integrity[at]sydney.edu.au with further questions.

Q: Who are the experts?

The identity of the country experts must be treated with confidentiality due to privacy issues. But in more general terms, an expert is defined as a political scientist (or social scientist in a related discipline such as sociology, economics, law…) who has published or who has other demonstrated knowledge of the political processes in a particular country. Specifically, we define demonstrated knowledge by the following criteria: (1) membership of a relevant research group, professional network, or organized section of such a group; (2) existing publications on electoral or other country-specific topics in books, academic journals, or conference papers; (3) employment at a university or college as a teacher. The selection sought to include a roughly equal balance between international and domestic experts. 40 persons per country were invited to participate in the survey. 

For more information about expert selection please also refer to:

Norris, Pippa, Ferran Martínez i Coma, and Richard W. Frank. 2013. ‘Assessing the quality of elections.’ Journal of Democracy. 24(4): 124-135. 

For a more detailed account of how the validity of the experts’ judgments was tested see:

Martínez i Coma, Ferran and Carolien Van Ham. 2015. ‘Can experts judge elections? Testing the validity of expert judgments for measuring election integrity’. European Journal of Political Research, 54(2):305–325.

Q: Why does the EIP not seek the opinion/assessment of domestic or international election observers?

There is no doubt that the reports of domestic election monitoring organizations are a crucial source of information about the quality of elections. Very important work is done by professional associations such as GNDEM or regional networks such as ANFREL to develop systematic indicators and standards. 

Nevertheless, there are reasons why one might want to supplement and triangulate several sources of information to make reliable and valid claims about the quality of an election.. Some of the shortcomings of election observer assessments are as follows: 

1) There are sometimes diverging assessments of election quality by different observer groups. Three examples can exemplify this:

• One example is the Cambodian 2013 election: the invited group of observers of the International Conference of Asian Political Parties (ICAPP) and the Centrist Asia Pacific Democrats International (CAPDI) claimed that the process had been ‘free, fair and transparent.’ Yet, other groups such as the Committee for Free and Fair Elections in Cambodia (COMFREL) or the Neutral and Impartial Committee for Free and Fair Elections in Cambodia (NICFEC) under the umbrella of the Electoral Reform Alliance (ERA) asserted significant problems. 
• In another example in the 2013 election in Azerbaijan observers of the Parliamentary Assembly of the Council of Europe (PACE) concluded that there was “a free, fair and transparent electoral process.” By contrast, the OSCE/ODIHR observation mission, which consisted of 13 Baku-based international experts and 30 long-term observers, criticized the electoral process by concluding that it "was undermined by limitations on the freedoms of expression, assembly, and association that did not guarantee a level playing field for candidates." 

It seems fair to say that there are observer groups of different standard, some of which might have partisan leanings, while others conduct their assessments with the utmost professionalism. Initiatives such as the Declaration of Global Principles for Nonpartisan Election Observation and Monitoring are crucial in generating common standards in this regard. In practice, however, given the lack of impartial accreditation of observer missions, the dividing line is somewhat slippery. Given this heterogeneity, one cannot rely solely on domestic observer organizations’ reports to assess the quality of elections.

2) In addition, it has not been clearly established whether observer reports are based on consistent standards over time, or whether expectations have altered with the expansion of international missions and developments in human rights thinking since the use of observers became more common in the mid‐1980s. Comparative evidence needs to be reliable across countries and across time.

3) The most severe restriction in relying on domestic election observer assessments is the fact that not all countries have such groups. When it comes to international observers, the pool of countries that are being observed is even more limited. What is worse, observers are more likely to go into countries that already have problems with elections, whereas in a large group of countries in the OECD world elections are not monitored by civil society group or foreign observers at all. For a comparative view on the quality of elections it is thus imperative to rely on evidence other than that of domestic observers, simply for the fact that otherwise one could not cover all countries in the world. It is for instance impossible to find civil society observers of elections in Norway, Germany, or France. But we would still like to get an idea about the quality of elections there.

Q: Why are one-party states such as Cuba or Vietnam included in the survey?

Some researchers might automatically exclude one-party states like Vietnam from the comparison, on the grounds that human rights in these countries are so deeply flawed as to make the elections just a façade disguising autocratic rule. We feel, however, that there are several reasons for documenting levels of integrity in all these diverse cases. 

1) One is that the degree of party competition varies substantially worldwide, as illustrated in Figure 2 below. Legal bans, while a major violation of human rights, are only one mechanism to restrict opposition. It is an empirical matter to measure the degree of party competition, such as by monitoring the seat or vote share won by the leading party in parliamentary contests, or the vote share of the winning presidential candidate. The PEI Index is designed to measure all the ways that party and candidate competition can be limited, for example through lack of a level playing field in access to party finance or state resources, partisan manipulation of district boundaries (gerrymandering), excessive legal requirements for ballot access, and high de jure or de facto vote-seat electoral thresholds. In several micro-states, small legislatures with majoritarian electoral systems also allow a clean sweep in a landslide victory for one party.

2) In addition, it is also important to monitor the contemporary quality of all elections worldwide to create benchmarks for future change, should states loosen legal restrictions on party and candidate competition in subsequent elections. 

3) Several aspects of electoral governance may still function relatively cleanly and efficiently even in states with restricted party competition and human rights. Indeed the quality of electoral governance may be higher in these cases than in several fledgling democracies with weak state capacity and insufficient resources to stamp-out malpractices and irregularities such as vote-buying, ballot-stuffing, or security threats. In Cuba, for example, during the nomination process some genuine competition is reported among rival candidates. Moreover, in the case of Vietnam, mustering the nation is a chance for the authorities to hone their mobilisation skills, check the efficiency of local leaders and get a snapshot of internal movements, and in all the logistical aspects may be well run.

4) The mean results also need to be read along with the confidence intervals which we publish, as well as the number of responses and response rates per country. Anyone is also free to exclude states with few expert responses to the evaluations. 

5) Finally this is a Perceptions of Electoral Integrity survey and obviously perceptions differ, and they may be wrong, for example concerning Republicans believing in fraud and Democrats believing in suppression. If we second guess the expert responses, for example if we personally disagree with the face validity of the assessments in cases such as Cuba, Iran and Laos so that we drop these cases, then this invalidates the process of gathering expert views.

Figure 2: PEI by levels of party competition
Note: The seat share is calculated by the proportion of seats in the lower house of the national parliament held by the largest party following the election.
Source: Electoral Integrity Project. 2014. The expert survey of Perceptions of Electoral Integrity, Release 3 (PEI_3) 

Q. Why is my country not included in the survey?

The PEI survey of electoral integrity focuses upon independent nation-states around the world which have held direct (popular) elections for the national parliament or presidential elections. The criteria for inclusion are listed below. Excluded from the universe of cases are micro-states, since it turned out to be excessively difficult to locate 40 experts for such small countries. Furthermore excluded are countries that do not hold direct national elections either de jure or de facto (see below). The elections analyzed thus far cover the period from 1 July 2012 to 30 June 2016. In total, PEI 4.5 covers 213 elections in 153 nations.

Criteria for inclusion in the survey	#	Definition and source
Total number of independent nation-states	194	Membership of the United Nations (plus Taiwan)
Excluded categories
Micro-states	12	Population less than 100,000 in 2013, including Andorra, Antigua and Barbuda, Dominica, Liechtenstein, Marshall Islands, Monaco, Nauru, Palau, Saint Kitts and Nevis, San Marino, Seychelles, and Tuvalu.
Without de jure direct (popular) elections for the lower house of the national legislature	5	Brunei Darussalam, China, Qatar,  UAE, and Saudi Arabia
State has constitutional provisions for direct (popular) elections for the lower house of the national legislature, but none have been held since independence or within the last 30 years (de facto)	3	Eritrea, Somalia, and South Sudan
Sub-total of nation-states included in the survey	174
Covered to date in the PEI 4.5  dataset (from mid-2012 to mid-2016)	153	87% of all the subtotal of nation-states containing 4.2bn people.

Table 1: Countries covered in PEI release 4.5

We will be covering more national parliamentary and presidential elections in independent nation-states around the world once these are held. For example, elections to be covered in 2015 include the UK, Canada, Nigeria, and Myanmar. The results of the survey will be published in subsequent reports in a cumulative fashion so that eventually the project will achieve global coverage.

Max Grömping

Sydney, 18 May 2015
(updated with reference to the PEI 4.5 data release on 26 September 2016)

Max Grömping is a researcher for the Electoral Integrity Project at the University of Sydney. His current research focuses on the impact of social media and domestic observers on electoral integrity. Prior to this, he lectured at Thammasat University, Thailand. 

Contact: max.groemping[at]sydney.edu.au