+ All Categories
Home > Documents > OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic...

OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic...

Date post: 27-Jun-2020
Category:
Upload: others
View: 5 times
Download: 0 times
Share this document with a friend
27
NORDEM Report 15/2005 OBSERVING ELECTRONIC VOTING Report by Kåre Vollan
Transcript
Page 1: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

NORDEM Report 15/2005

OBSERVING ELECTRONIC VOTING

Report by

Kåre Vollan

Page 2: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

Copyright: the Norwegian Centre for Human Rights/NORDEM and Kåre Vollan.

NORDEM, the Norwegian Resource Bank for Democracy and Human Rights, is a programme of the Norwegian Centre for Human Rights (NCHR), and has as its main objective to actively promote international human rights. NORDEM is jointly administered by NCHR and the Norwegian Refugee Council. NORDEM works mainly in relation to multilateral institutions. The operative mandate of the programme is realised primarily through the recruitment and deployment of qualified Norwegian personnel to international assignments, which promote democratisation and respect for human rights. The programme is responsible for the training of personnel before deployment, reporting on completed assignments, and plays a role in research related to areas of active involvement. The vast majority of assignments are channelled through the Norwegian Ministry of Foreign Affairs.

NORDEM Report is a series of reports documenting NORDEM activities and is published jointly by NORDEM and the Norwegian Centre for Human Rights.

Series editor: Siri Skåre

Series consultants: Hege Mørk, Christian Boe Astrup

The opinions expressed in this report are those of the author(s) and do not necessarily reflect those of the publisher(s).

ISSN: 1503–1330

ISBN 10: 82–8158–010–0

ISBN 13: 978-82-8158-010-7

NORDEM Report is available online at: http://www.humanrights.uio.no/forskning/publ/publikasjonsliste.html

Page 3: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

1 Observing Electronic Voting

Preface The author, Kåre Vollan, has headed thirteen international observation missions on behalf of OSCE/ODIHR and NORDEM, and he has done extensive consultancy on elections in post conflict areas and in new democracies. He is also a registered Lead Quality Auditor with the International Register for Certificated Auditors (IRCA) within the recognised UK based TickIT scheme for quality certification of IT organisations.

The Norwegian Centre for Human Rights / NORDEM University of Oslo December 2005

Page 4: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

2 Observing Electronic Voting

Contents Preface

Contents

Introduction ........................................................................................................................... 3 The Challenges....................................................................................................................... 4 Types of Voting ...................................................................................................................... 5 Benefits and Risks of Electronic Voting..................................................................................... 7 International Standards .......................................................................................................... 9 Assessing the Technology ..................................................................................................... 11 Product Validation ................................................................................................................ 12 Process Verification............................................................................................................... 13 Audit Trails........................................................................................................................... 14 Observing Electronic Voting................................................................................................... 16 Conclusion ........................................................................................................................... 19 Appendices .......................................................................................................................... 20

Page 5: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

3 Observing Electronic Voting

Introduction

Since the early 1990s methodologies for observing elections have been developed, in particular by regional organisations like the OSCE, AU and EU, and by the UN. Observation has become an important instrument for assisting transfer democracies in developing multi-party elections with the general credibility needed in the respective country and internationally. The impact of observation depends on the willingness of a country to carry out democratic, multi-party elections. In countries where all parties involved have a clear intention of carrying out the elections properly, there may be a positive will to implement proposals from observation missions. In other countries where the authorities may not have an intention to make elections an arena for the free competition of powers, the observer reports may fill a need towards the civic society and any democratic opposition in the country rather than being useful for the election organisers. In addition, the international community will often use the reports when deciding on the level of cooperation with such countries.

With the introduction of electronic voting, the general will to get the process right becomes increasingly important. Observer missions should base their assessment on a clear methodology and at the same time realise the limitations of what can be observed and what cannot be observed. In a deeply divided society the introduction of electronic voting may even broaden the gap between the competing groups.

Observing elections is not limited to transfer democracies. It is worthwhile assessing to what extent old democracies comply with standards developed for transfer democracies, and in particular with the introduction of new technologies it is crucial to assess how international commitments can still be maintained.

The term electronic voting may include a number of ways by which voting is supported by computer systems. The purpose would be to make the voting process easier and more accessible to voters, and above all to make the count faster, less manpower intensive and more reliable. The systems supporting the actual voting may either be systems installed in the polling stations, or systems supporting remote voting, such as via the Internet.

This paper will discuss systems both installed in polling stations and used for remote voting. However, IT systems being used to maintain voter registers, mark the voter registers or count ballots will not be discussed here. Part of the methodology may still be applicable, but such systems would normally have a clear paper trail that can be assessed, and will therefore be less problematic than electronic voting systems.

Electronic voting represents new and big challenges. The ballot itself should be kept secret and there must be full confidence that the vote cast by a person is recorded accurately in the system. The electronic trace of the vote must not be linked to the voter, but there must still be full trust that each vote is correctly registered and counted.

Page 6: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

4 Observing Electronic Voting

The Challenges

Electronic Voting in Transition Democracies In transition democracies the trust in the elections process is often poor. International observation missions have often found that the process has a particular reputation. The main instrument for giving elections credibility is transparency in every step.

In countries where parts of the administration have an intention of using illegal means to bring about a certain result, the need for transparency is even greater. One should be able to implement checks and balances by giving full access to records and information throughout the electoral process.

The main challenge of electronic voting is the lack of transparency. If a vote is cast by use of a machine, either in a polling station or by the Internet, the vote is being processed and stored, and there is not a straightforward method for checking that the votes are being stored without any kind of alteration. The only way of checking that the machine does its job at all times is to produce a paper trail, which duplicates and documents the vote.

Therefore, in situations where the general trust in the electoral process is low, an introduction of electronic voting may only increase suspicions.

When assessing electronic voting in transition democracies it is therefore not sufficient to check if the system is working according to specifications, that it is reliable, has the right capacity and that it is protected from attacks and manipulation from the outside. It is also necessary that those organising the elections check whether the system is being manipulated from the inside. In a paper based system there are normally representatives of all candidates present at every step dealing with ballots, from the time when these are cast till the protocol is drawn up. With electronic voting, the basic means of verification, the paper ballot, may be missing. The records of the voting are kept in a machine without direct access to the basic data, and the integrity of the process is dependent on the parties’ trust in the machines. For observer missions it will not be possible to carry out verification of the machines and the computer programs in a manner that can establish such trust. This is the major challenge in observing electronic voting.

This conclusion is supported by Oostveen and van den Besselaar (Ref /8/) who states:

“However, technologists start to acknowledge that security mechanisms are fundamental social mechanisms. Trust is of great importance; people no longer have a blind faith in scientific objectivity and the “experts”.”

The Risks The risks with electronic voting fall into three categories:

1. The possibility that the system fails because of bad design, errors, etc.

2. The possibility of manipulation (e.g. hacking) from the outside

3. The possibility of manipulation from the inside.

Observer missions will have no way of assuring that these risks are under control by detailed studies of the hardware and the software. These risks must first of all be handled by the authorities and the suppliers of the equipment. Observer missions may only assess if such measures are taken and if sufficient audit trails are available for checking results.

Page 7: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

5 Observing Electronic Voting

In established democracies the last issue has less prominence than the first two. However, even there the question of how to check whether or not there is a bug in the software that will give one party an advantage is being discussed. However, the first issue is more important (e.g. in Florida in 2000).

Types of Voting

The Development of Voting Methods International conventions and standards have offered criteria that have become common practice for elections around the world. Such criteria include:

Voting should be done in person without pressure or intimidation;

Every voter should only vote once;

The vote should be cast in secrecy;

The vote should be universal, allowing everybody with the right to vote to participate;

The election process should be transparent, and thus secure against fraud and irregularities.

The traditional form of voting takes place at dedicated polling stations on one particular day (or more than one day) where voters turn out in person to vote. In such an environment the voting can most easily be checked against the international standard requirements.

Modifications to the simplest form of voting have been introduced in many countries:

Advance voting to allow voters to vote before Election Day in case they are absent on the day of voting;

Tendered or conditional voting, where voters are allowed to vote in another polling station than the one at which they are registered;

Mobile voting, where mobile teams travel with voting material to somebody’s home or to institutions to enable physically disabled persons to vote;

Assistance to voters who are illiterate or blind and therefore not capable of carrying out the voting alone.

All these arrangements have been made to strengthen the universal element of the election, but they challenge the transparency of the vote, and the ability to control that persons are voting only once and in full secrecy. However, there are reliable methods of supporting these modifications without jeopardising the integrity of the vote, even though there are examples where such modifications have been used to manipulate elections. The voting is still done in person and with a high level of security.

Early voting may be done in polling stations with the same level of control as on Election Day. In such a case, the challenge is in the safeguarding of the ballots up to Election Day and the subsequent count. In other countries advance voting may be organised with less control, e.g. in post offices. In such a case, trust is dependent on the general trust in the office receiving such votes. Yet another possibility is that advance voting is done by mail (e.g. in the UK and Spain). This is voting in an uncontrolled environment and involves challenges both to the security and the secrecy of the vote.

Page 8: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

6 Observing Electronic Voting

In order to allow voters abroad to vote, some countries have introduced by-mail voting. The ballot is mailed to the voter who in turn will have to return the ballot with some documents showing that the person has the right to vote and is voting in person. By-mail voting puts more emphasis on allowing a group of voters to vote than on the control of a personal and secret vote. In Bosnia and Herzegovina the votes of the refugees were crucial to the elections. At the same time, many attempts at stealing voters’ identities were revealed and even if the voter was genuine, there could be no guarantee that the vote was cast in secrecy.

Classification of Types of Voting With the diversity of voting now being implemented it is useful to classify the voting in two types: Voting in a controlled environment and voting in an uncontrolled environment.

Voting in a controlled environment is voting where the voter casts a vote in person and where election staff verifies that the vote is cast in secrecy with the prescribed identity checks, etc. The obvious example is voting in polling stations on Election Day(s). On Election Day the environment is fully controlled by election staff and observed by representatives of the candidates. Many countries allow for the presence of observers and media as well. Early voting will often also be carried out in a (more or less) controlled environment, such as at polling stations, public offices, etc. Even external voting – i.e. voting outside the country – may be organised in a controlled environment only.

An uncontrolled environment would be voting where the voter is casting the vote by himself or herself without anybody controlling that the vote is cast in secrecy or without intimidation or pressure. The most common example is postal voting, but even voting over the Internet has started in some countries. Voting in uncontrolled environments was originally an offer to those who were unable to come to the polling stations in person either because they were abroad, hospitalised or in other ways not able to cast a vote in person on Election Day. However, some countries (such as Great Britain and Spain and some states in the USA) have opened for postal votes as an alternative to voting in a polling station that is open to any voter.

The term Electronic Voting (or e-voting) is used both for voting by use of specially designed voting machines in a controlled environment and for voting via the Internet, which could be accessed in controlled or uncontrolled environments. A specially designed voting machine consists of computer hardware and software, and the voter manipulates a user interface on a screen, keyboard, buttons or other such means of indicating the choice of candidates, instead of casting a paper ballot.

Internet voting would be accessed via an Internet screen from any computer with access to the Internet. The voter needs to be identified by a user identification code, and security measures, such as PIN-codes or more advanced security keys, must be in place to ensure that the voter is genuine. The voter will then have to use the mouse or keyboard to make her or his choice of candidates or lists.

Internet technology may also be used for e-voting in controlled environments, and may in such cases not be accessed from the Internet itself, but only from a so-called Intranet.

Voting in a Controlled Environment Voting on Election Day(s) is normally done in a controlled environment, even though the various forms of early, uncontrolled voting may be extended to Election Day as well. Traditionally the ballots are cast as sheets of paper in a ballot box, and the ballots are counted at the end of the

Page 9: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

7 Observing Electronic Voting

poll. A number of machines operated by the voter in secrecy in the polling station when casting the vote have also been introduced. This would make the voting more accurate since checks on invalid ballots may be worked into the machine, and the count may be faster and more accurate.

Early voting may also be conducted as e-voting in a controlled environment. In such cases the secrecy of the vote is supposed to be assured by the election staff.

Voting in an Uncontrolled Environment Voting in an uncontrolled environment includes both postal votes and voting over the Internet. The voter can vote from home or from other places free of election staff. Such voting raises two major problems: There is a risk of impersonation and fraud where the vote is cast by other persons than the voter and without the person’s consent or knowledge, and the vote may not be cast in secrecy free from intimidation or undue influence.

Voting over the Internet has advantages over the postal vote in that the risks of impersonation may be less. Personal keys are now being developed in such a way as to allow people to perform sensitive tasks via the Internet. Tax returns and bank transactions are done via the Internet in many countries, and even more sensitive communication will follow. However, so far the keys to such transactions can be given to a third person who will make the transactions on another person’s behalf.

In the future, personal keys for access to sensitive Internet services will include biometric data. This can ensure that the person owning the key is in fact present at the computer when accessing the Internet.

Internet voting was introduced in the local elections in Estonia in 2005 and was possibly used for the first time in an election where any voter could use this option.

Benefits and Risks of Electronic Voting

Reasons for Electronic Voting Electronic voting has been introduced for a number of reasons:

The most obvious benefit is that once an electronic ballot is cast, the count can be done in an efficient and accurate manner;

Some have argued that electronic voting would increase the turnout. In particular young voters would find voting more interesting if the casting of the ballot involved computers. If Internet voting is an option, the access to the process will be much easier for a number of people;

The process will be more reliable and less prone to human errors.

In addition, there may be other advantages. In India the Election Commissioner stated that the electronic voting saved 8,000 tons of paper during the 2004 elections.

The Risks As already mentioned the risks fall in three categories:

Page 10: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

8 Observing Electronic Voting

1. The possibility that the system fails because of bad design, errors, etc. (technical breakdown);

2. The possibility of manipulation (e.g. hacking) from the outside;

3. The possibility of manipulation from inside;

4. The possibility of impersonation with Internet voting;

5. The possibility of intimidation and the lack of secrecy with Internet voting.

The two first risks can be controlled in a reasonable manner, whereas the third has more to do with the political context of the country than technical measures that can be taken. The last two points will come in addition to the three first if Internet voting is introduced.

Technical System Failures These types of events may happen in any country. The way of preventing this from happening is by professional work in designing, developing and testing the system, as with all major IT-systems.

Threats from Outside Outside threats may consist of manipulation during the design and development phase (e.g. a programmer implementing an algorithm giving extra votes to one party), attacks from the outside via a network or manipulation of the actual voting machines by, for example, exchanging firmware or other components. In addition attacks may be physical as they may be with manual systems.

Threats from Inside In transfer democracies this issue may be the most dominant. Even though the authorities may have no intention of implementing a system with a bias, the parties may have little trust in their good will. There have been examples where an algorithm giving a small percent of extra votes to one party has been found in a counting system. The chances of disclosing such a manipulation by external reviews are very low, unless massive resources are available. For counting software, checks against manual counts will always be possible, and the possibility for revealing fraud is realistic. For voting systems it is more complicated, unless there is a paper trail, which can be used to verify the ballot.

There are other systems where the organisation operating the system may have an interest in showing manipulated results. The most prominent example is accounting systems, where a company for tax or market perception reasons may want to show a result different from the legal one. Governments have therefore established checks and balances to verify accounting systems as well as actual accounts produced.

The Security of Internet Voting Some countries have introduced or plan to introduce e-voting in an uncontrolled environment. Voters would, by use of a user ID and a security key, be able to vote via the Internet. Conventional methods of identification provide no guarantee that the user ID and key have not been given to another person. For bank transactions and tax returns this may be acceptable with the person’s consent. However, with voting this is not in compliance with international standards. In addition, regular thefts of identities represent a risk.

Page 11: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

9 Observing Electronic Voting

With improved security keys the risks of impersonation, both with and without consent are reduced and may be acceptable compared to any other means of voting.

The Secrecy and Integrity of Internet Voting Even with measures taken to reduce the risk of impersonation, there is no guarantee that any vote in an uncontrolled environment (postal votes or Internet votes) is cast in secrecy free from intimidation or pressure.

Nevertheless, remote voting has been introduced in a number of countries. In the UK and in Spain all voters may request a postal ballot without having to give a reason, such as not being in the country on Election Day, and Internet voting has been introduced as an option for the general public in Estonia.

In particular in the UK the postal vote has raised an important debate on the secrecy of the vote, in addition to the possibility for impersonation and fraud. Internet voting will have the same features as postal votes when it comes to the secrecy aspect.

During the 2004 local elections significant fraud was revealed in three wards in Birmingham. The Birmingham case included members of immigrant communities where community leaders took control over families’ ballots. But within a family it is also possible that a strong head of the family would vote on behalf of the whole family as he or she would with other paperwork. This would not necessarily be seen as being wrong or questionable within a family. With Internet voting it may not be easy within every family to tell a strong family head that the vote is secret and that every family member should fill in the ballot without witnesses.

International Standards

Before discussing the ways of validating electronic voting, we will summarise the standards according to which it may be of interest to assess the systems.

Elections Related Conventions and Standards The 1966 International Covenant on Civil and Political Rights passed by the UN is binding for the states having ratified it. Article 25 states:

Every citizen shall have the right and the opportunity, without any of the distinctions mentioned in article 2 and without unreasonable restrictions:

(a) to take part in the conduct of public affairs, directly or through freely chosen representatives;

(b) to vote and to be elected at genuine periodic elections, which shall be by universal and equal suffrage and shall be held by secret ballot, guaranteeing the free expression of the will of the electors;

(c) to have access, on general terms of equality, to public service in his country.

Both the secret and the free ballot are mentioned. The right to participate is also mentioned, and obviously the strongest argument for remote electronic voting is to facilitate participation.

Page 12: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

10 Observing Electronic Voting

By introducing widespread voting outside of a controlled environment, such as the postal vote on demand in the UK, some have argued that the requirements for a secret vote are fulfilled by offering the voters a possibility for casting a vote in a controlled environment, without securing such an environment for everybody. Secrecy is an offer to the voter; it is not an obligation on the part of the country to enforce a secret vote by everybody.

The opposite view would be that the most efficient way of teaching young members of families, not least those without a strong election history, that the vote is a personal one and that the ballot is to be cast in secrecy without undue influence, would be to make everybody vote in the strictly controlled regime of a polling station.

When striking the balance between strengthening the universal aspect of the vote by making the ballot more accessible and maintaining a personal and secret vote, it seems that many countries accept that voting in an uncontrolled environment would be acceptable to groups that would otherwise be disenfranchised, such as voters who are abroad on election day. However, making voting in an uncontrolled environment available to all voters on demand is still controversial, and the debate about how such voting complies with international standards has not yet found a generally accepted conclusion.

Relevant regional conventions and documents are listed in Appendix A.

IT Related Standards A number of standards are being used to assess the quality of IT systems. Most of these are, however, related to the process of developing the software rather than the product as such. The most prominent standards would be:

ISO 90003:2004 which is an interpretation of the quality standard ISO 9001:2000 applied on software development.

ISO/IEC 27001:2005 (built on the British standard BS 7799) is a recognised security standard defining requirements both to the organisation developing as well as operating a system. It is more powerful in protection from outside threats including own staff than it is against wilful internal manipulation.

In addition to these standards, standards and methods like TickIT, COBIT and CMMI are being used in the industry.

Standards set requirements, but just because an organisation declares or commits to adhere to a standard it is not a given that it will do so. Therefore a system for certification of both organisations (their management system) and products has been established. Such certification may be done within governmental schemes or within industries without a formal government stamp.

Accredited certification would mean that the certificates issued are endorsed within a government scheme. In Europe such accreditation is developed in great detail, and accredited certificates issued are recognised across Europe. The most commonly known certification schemes are those performed by accredited certification bodies in accordance with ISO 9001:2000. The certificates are issued to an organisation based upon a quality audit and maintained through frequent visits by the certification body. Such certificates are not product certificates.

The most well-known, IT-industry recognised, assessment system is the Capability Maturity Model Integration (CMMI) assessment established by the US-based Software Engineering

Page 13: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

11 Observing Electronic Voting

Institute. This is a private scheme and there is no government accreditation and no system for maintaining a certificate over time.

In addition, test laboratories may assess products against product standards. They would assess a prototype of the product and it would be up to the producer to maintain the quality throughout their production.

A list of IT-related standards is given in Appendix B.

Assessing the Technology

When establishing an electronic voting system, the organisation in charge (normally the EMB – Election Management Body) will have to be aware of requirements for validating the product as well as for verifying the acquisition process. This section describes what the EMB, not an observer organisation, would be expected to carry out.

Terminology Before discussing what observer missions should do, we will introduce a few terms:

Verification: Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled (ISO 9000:2000).

Validation: Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled (ISO 9000:2000). The difference between verification and validation is that validation is against user requirements checking that the final purpose of the system is fulfilled. This is often done in test laboratories, and in some cases it may end up with a certificate (e.g. an e-mark).

Audit: A systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled (ISO 9000:2000). It is, in other words, a verification process intending to verify that a quality management system meets requirements (requirements as defined by for example ISO 9001:2000). It is the processes rather than the products that are verified.

Observation: A process including elements of both review and audit, with a main focus on verifying election processes based upon samples of evidence.

Certification: A process performed by an independent organisation (certification body) where a quality management system or a product is verified (or validated) to demonstrate through objective evidence that specified requirements (e.g. ISO 9001:2000 for quality management systems) are met.

Product and Process Assessment If an EMB (or a local election body) acquires a system for electronic voting, they should have a plan for validating the final system, and a managed process for acquiring the system. The validation consists of testing and reviewing the system and assuring that it meets the specified requirements. Verifying the managed process would involve assessment through a quality audit

Page 14: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

12 Observing Electronic Voting

of the acquisition, from supplier selection through to delivery and ensuring that sufficient checks are built into the process.

The basic requirement for a managed process to work is, however, that there is general trust across the political spectrum that electronic voting should be introduced and that it can be done in a transparent and safe manner. Unless this condition is fulfilled, it is extremely difficult to establish the objective evidence needed to decide if the process meets international standards for IT or for elections. The evidence being reviewed in a quality audit would have to be records kept in good faith. It is difficult to make such an assessment if the records have been wilfully tampered with.

Product Validation

An electronic voting system should be validated. This is a highly technical exercise whereby the system is proven to meet requirements. It should be carried out by the supplier of the system as well as the purchaser. However, it would only prove that the particular version of the system that has been under validation meets requirements. There is of course no guarantee that the persons having the systems (machines, boxes) under their control cannot patch up the software with a bug that will create certain results.

Product Requirements In order to validate an IT system the system must be clearly specified. This is done by a detailed, written requirements specification. In the case of an e-voting system, the requirements should be a public document open for review.

The purpose of the validation is both to make sure that the functional requirements are met, as well as the fulfilment of so-called non-functional requirements. Such requirements are:

Reliability, which means that the system should work without faults at all times.

Availability, which specifies that the system is actually working most of the time. A system of this kind should at least have an availability of 99.9%.

Failsafe operation, which means that if the system should fail, the data would not be lost, and that the system can be restarted. In the worst case, it should be possible to revert to manual procedures.

Security, which means that the system is protected from physical and logical (by electronic means) attacks from outside and inside. This includes both resistance against hacking and manipulation of the system.

User friendliness, which means that the system is easy to operate for voters as well as for election staff.

Maintainability. It should be possible to expand and maintain the system, which means that the system must have a modular, well-documented structure.

Response times, which is the time from the moment a vote is cast till the machine is ready for the next ballot. This should be short (e.g. less than two seconds).

The capacity in terms of load on the system should be sufficient to carry any realistic ø-turnout in a polling station. This is normally not a big problem with modern systems.

Page 15: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

13 Observing Electronic Voting

The ability to check the results, by reconciling reports (see the section on audit trails).

Assessing against Requirements The validation would consist of detailed acceptance testing against the requirements specification, review of documents and in some cases review and inspection of design documents and code. The latter depends on the skills at the receiving end of the delivery.

Process Verification

The process of acquiring the electronic voting system should be managed and controlled. In most cases the system is supplied by a third party, such as a company or a public sector IT-department. (In-house development by the EMB may be possible as well, but this is not covered here. In such cases the development should be under similar control as in the case of an external supplier.)

The managed process seen from the EMB’s point of view will take the following steps:

1. The establishment of the requirements specifications;

2. Selection of supplier;

3. The supply of the systems with development needed to meet the requirements;

4. The acceptance of the delivery;

5. Support and maintenance of the system.

The Requirements Specification We have already covered this point. The requirements need to cover the functionality as well as non-functional attributes to the system. An important point is the requirements for audit trails and the possibilities for reconciliation of numbers in the system.

Selection of the Supplier This is an extremely important point and may be decisive to whether the solution will incite general trust in the end. The criteria for the selection may include:

Financial solidity;

References and market reputation;

Quality System, including certification;

Security System, including certification;

Current product suite.

For a sensitive product such as a system for electronic voting, the supplier needs to demonstrate that they follow sound processes for the development and testing of software, and that security requirements are an embedded part of their internal control system. One way of demonstrating such capabilities would be to show valid certificates issued by accredited certification bodies in

Page 16: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

14 Observing Electronic Voting

accordance with relevant quality and security standards. Should that not be possible, one may choose to engage a third party to carry out a pre-contract audit of the potential supplier.

The Supply of the System Supply of the system would in most cases not just be off-the-shelf delivery but substantial parts of product development. During this period, the EMB should receive regular reports, and be prepared to verify detailed specifications and changes if needed. Technical reviews would normally be outside the competence scope of the EMB.

It would enhance the trust in the process if the EMB, during the development of the system, engaged a professional third party to carry out audits of the project. Such audits would cover the following processes on the supplier’s side:

Management control, including planning, risk management, corrective actions and measurements;

Sub-contractor control;

Design planning and verification;

Testing;

Configuration management;

Internal audits.

The EMB would have to make sure that the right to carry out such audits is granted in the contract with the supplier.

The Acceptance of the Delivery The acceptance of the delivery should always include extensive acceptance testing. The acceptance tests are developed against the requirements specifications and they include both functional and non-functional requirements. Technical expertise should be taken on to help out with this if needed.

In addition, any open issues from reviews and audits during the delivery should be assessed and resolved.

Support and Maintenance of the System It is important that the supplier has an ongoing obligation to correct errors disclosed after acceptance, and in particular during the elections the availability of experts from the supplier should be specified. A service level agreement (SLA) should specify the maximum time for repair of the various components of the system and of the system as such.

Audit Trails

After having validated a system prototype and verified the acquisition process, one may still not be satisfied that the electronic voting system always produces the correct result. The reason may be that there is a fear that the election organisers have manipulated the systems from inside, but also that the technology happens to fail for any other reason. The only way of verifying the

Page 17: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

15 Observing Electronic Voting

whole process would be to produce a parallel paper trail often referred to as VVAT, voter-verified audit trail.

Producing a paper trail is only possible for electronic voting in a controlled environment. In Brazil a law was enacted in 2002 to ensure that each voter would be able to see his or her choices printed on a slip of paper, behind a transparent window. When confirmed, the vote slip would be inserted into a sealed bag, without voter interference, and the bag properly handled for later audit against electronic tally by sampling, or for possible recount. Ref /6/.

According to The New Your Times (18 January 2005), the November 2004 election of an agriculture commissioner in North Carolina had to be invalidated because of defects in the voting machines. The votes that were cast did not tally with the votes registered in the machine. The article concludes: “If the Carteret County voting machine had produced a voter-verified paper record each time a vote was cast, these paper records could have been counted and the matter would be resolved. But electronic voting machines that do not produce paper records make recounts impossible.” In Ohio the commissioner has, in contrast, ordered the voting machines to produce verifiable paper records.

In Norway voting machines were tested out in three municipalities during the 2003 local elections. The report from the evaluation group (Ref /1/) concluded that machines with verifiable paper records should be further investigated. There have been a number of other studies, particularly after the 2000 presidential elections in the US, discussing or recommending the same solutions.

In traditional paper-based voting, there are often claims of irregularities. The checks of the process include independent witnessing of the vote, the count and the possibilities of recounts. Even with sophisticated electronic counting systems, e.g. based upon scanning of ballots, there is always a possibility of a manual re-count, since the ballots are saved. This is similar to an electronic accounting system, where all the transaction records are kept on paper, and alternative manual calculations or simulations may be performed.

In the electronic voting machine, the vote is recorded electronically. A link to the voter would violate the secrecy of the vote. There is therefore not a similar record to go back to if in doubt, unless a paper trail of each vote is being produced as well and the voter is able to check that the printed ballot is correct before placing it in a sealed box.

The paper strip may be used in such cases where doubts have been raised with regard to the electronic result and the paper ballots are counted as an extra control. Paper ballots may also be counted as a random check of the results.

If there should be a serious lack of trust between the EMB and the parties, a printed audit trail of ballots would be the only way of putting such doubts to rest.

Regardless of whether paper strips are being produced or not, reports should be produced for reconciliation against the voter registers and other sources of information about the election process.

For Internet based voting such paper strips would not help. The only audit trails would be those produced electronically by the system. By separating the voter registers and records of voting from the electronic ballot box, electronic reconciliation can take place, but this would not satisfy the need for spot checks and recounts in cases where the technology fails or where the fundamental trust in the electoral process fails.

Internet based voting is being evaluated and tested out in some countries including the UK, the USA and Switzerland. The focus has been on making the access to voting easy and on solving the fundamental security and verification issues.

Page 18: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

16 Observing Electronic Voting

A study performed by Jefferson et al. (Ref /4/) to evaluate a secure system for Internet voting (SERVE) in the US, concluded in 2004:

“DRE (direct recording electronic) voting systems have been widely criticized elsewhere for various deficiencies and security vulnerabilities: that their software is totally closed and proprietary; that the software undergoes insufficient scrutiny during qualification and certification; that they are especially vulnerable to various forms of insider (programmer) attacks; and that DREs have no voter-verified audit trails (paper or otherwise) that could largely circumvent these problems and improve voter confidence. All of these criticisms, which we endorse, apply directly to SERVE as well.

But in addition, because SERVE is an Internet- and PC-based system, it has numerous other fundamental security problems that leave it vulnerable to a variety of well-known cyber attacks (insider attacks, denial of service attacks, spoofing, automated vote buying, viral attacks on voter PCs, etc.), any one of which could be catastrophic.”

The security issues related to the identity of the voter and attacks from outside will no doubt be solved some time in the near future, but that does not mean that it can be fully verified that the system provides the correct result. The lack of paper trails and possibilities for spot checks will still remain.

Observing Electronic Voting

It is first of all the EMB who should secure the electronic voting system and process. It is quite obvious that an election observation mission cannot perform all the tasks that the EMB should initiate or carry out. The observers will not have the capacity to validate the systems in detail, and even if they had, it would be difficult to ensure that the systems are not being altered before placing them in the polling stations.

In particular in those cases where the trust across political lines is missing, a technical review of the observers may create a belief that the observers have certified the system, whereas a full certification in practice would be impossible.

This means that the observer mission should be very conscious about what they want to verify and what they cannot verify. Any conclusion should avoid reference to having verified, validated or certified the system. One should even not claim to have tested the system, unless under very extraordinary circumstances they are set up to do this in a very systematic manner (which is a task that takes months to carry out).

The observer mission may, however, do very useful checks on both the process of acquisition, the overall functionality of the system, and the electoral process based on audit trails.

The Decision Making Process and the Trust in the EMB The most obvious point of departure for an international election observation mission (IEOM) is an assessment of the process that led up to the decision of electronic voting. Was there a consensus behind the decision, or was the decision made by virtue of a majority in the decision-making body (such as the EMB)? In cases where a consensus was not reached, the minority (or opposition) may fail to trust the electronic voting system, regardless of which safeguards have been built into the development process and the validation of the products. In such cases the

Page 19: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

17 Observing Electronic Voting

use of paper audit trails and the willingness to initiate manual recounts may be the only way of evaluating the voting process.

Internet based voting can hardly gain public trust if the EMB itself does not enjoy general confidence. In addition, the general security and secrecy issues make voting in an uncontrolled environment controversial.

Election observation missions should therefore assess the general trust which the public has in the electoral process and in the EMB. If the EMB is seen as a being controlled by the incumbents, electronic voting without paper audit trail would represent a voting process without transparency. The vote is entered into a black box without any means of controlling that the vote will be correctly recorded and counted.

In India there is a court decision to the effect that the machines used there could not be manipulated. That decision is probably based upon an assumption that the Election Commissioner is protecting the integrity of the process and has no intentions of replacing the machines with manipulated ones. Under that assumption the machines may be sufficiently protected from outside manipulation.

The Secrecy of the Vote Election observation missions have to assess elections against international standards for elections. Internet voting constitutes a particular challenge to the secrecy of the vote regardless how the security and reliability issues are being solved. Internet voting (as even postal voting) is normally justified by the possibility for increased participation. The more common postal voting has been widely accepted for special groups which would otherwise be disenfranchised. There does not seem to be a general consensus to what extent voting in uncontrolled environment (postal and Internet voting) should be made available to all voters.

Product Validation by Observer Teams Whether it would be meaningful for an observation mission to carry out such validation is doubtful. It would take a large amount of technical resources to do it right and even then the system may be changed after validation.

A commercial product validated by a third party would be identified by version identification, and it would be a criminal offence if a supplier should change this. The system is based upon trust. If lack of trust is the main problem, a validation by an observation mission technical team may do more harm than good.

However, one of the technical assessments that an audit team should carry out is of the overall design of the system and the possibility for auditing the results. In particular the audit trails should be assessed. What kind of reports will the system offer, what can the reports be reconciled in accordance with, and what are the procedures in place for doing spot checks and reconciliation.

In addition, observers may validate the user-friendliness of the system. Can all voters operate the e-voting system, are there alternatives for those who would have problems, and can staff handle the equipment properly?

Process Verification by Observation Teams The main issue to observe is whether all involved have realised the critical nature of an election process, the complexity of the process, the number of people involved and the need for training, the way the system will behave under load and the need for fall-back solutions.

Page 20: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

18 Observing Electronic Voting

This does not mean that the observer mission should not do anything. In addition to an assessment of the overall design, the process of acquiring and implementing the system should be looked into. The international standards for quality management systems may provide good guidance regarding what to look for, in particular with the guidelines for software development (e.g. the TickIT guide).

Quality systems auditors do this in public and private organisations, and parts of the same techniques may be implied by the observers. Based upon interviews and documentary evidence the processes may be assessed. Particular weight should be put on transparency and trust during the decision-making process, and design, acquisition and test phases. Have all parties been invited to participate in crucial decisions, is there a degree of consensus on the design and do the parties trust the system?

Certificates from accredited certification bodies may replace audits done on behalf of the customer (EMB) in each case. This is particularly the case where such certification refers to IT-related schemes and standards. One should in such cases make sure that the certificate has not expired and that it covers the relevant line of business.

Observing Audit Trails Observers have often found that parties have shown little trust in the computer systems aggregating results from polling stations to the district and national level, in particular if the transfer has been electronic. However, in such cases it is fairly simple to check the results later by adding up results through use of simple tools, as long as the aggregation is published from the polling station level and up to the national level. This is also why observer missions have insisted that election organisers make detailed results available after elections.

Checks of computerised counting systems have been based on audit trails. It must be possible to go back to raw material such as polling station protocols, bypass the computer system and end up with the same aggregated results as with the computer systems.

With electronic voting similar audit trails will be extremely important, in particular in transition democracies. (Even in Norway there has been criticism of a system that was tested out in a few municipalities in 2003 for the lack of auditable evidence from the voting machines being used.)

A comparable example from other IT systems would be accounting systems. For such systems the authorities as a third party have an interest in assuring correct results, where the management may have an interest in manipulating the same for tax reasons (or for the stock market). The ways the authorities can ensure the integrity of the systems are through requirements for:

- The developer of the software;

- The organisation running the software and their auditors;

- The audit trails, with paper verification at each step of the process.

For voting a similar approach should be taken, and the most difficult part is the audit trails. In accounts one may always be able to go back to the invoice of the receipt and follow each one through the system up to the reported profit. With voting machines the basic document may be missing: the ballot paper.

Page 21: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

19 Observing Electronic Voting

Conclusion

Electronic voting is being introduced to reduce the time and effort needed to calculate election results, to improve the reliability and accuracy of the results and to reduce errors in the voting itself. On the other hand, transparency may be reduced if paper ballots are not produced, and there may – in particular in transfer democracies – be a reduced confidence in the results. Such a lack of confidence may stem from the fact that each step of the voting process cannot be verified in the same manner as a paper based system, but may also come from a belief that the election administrators are under the control of the incumbents and that electronic voting is used as a conscious means of manipulating the results.

Electronic voting machines used in a controlled environment may produce a paper record of the vote, which can be validated by the voter and be used for manual recounts if needed. Such additional security cannot be obtained for Internet voting, and all voting in uncontrolled environments raises questions about the security and the secrecy of the vote.

Observing electronic voting may include the following main activities:

The most fundamental question to assess is whether there is a general trust in the EMB and its honest intentions when introducing electronic voting. If such confidence is not in place it is difficult to see that any independent assessment can establish whether the system is correct or not. In India there is a court decision to the effect that the machines used there could not be manipulated. That decision is probably based upon an assumption that the Election Commissioner is protecting the integrity of the process and has no intentions of replacing the machines with manipulated ones. Under that assumption the machines are protected from outside manipulation.

If the observer mission includes IT-expertise, the process of acquiring the electronic voting systems should be assessed. Elements for the assessment would be the selection criteria, the quality and security system of the vendor, possibly underpinned by accredited certificates, the validation process, the process of replication of software, etc.

The overall design of the system should also be assessed. Such assessment would include the user-friendliness of the system, the possibilities for access from outside, the redundancy of the system and other measures to assure that the system has the required availability, the procedures in case of failures, both in terms of capturing the data of the voting already taking place as well as the manual backup systems.

In principle the hardware and software may be validated as a product by independent expertise. This is – if done in a credible manner – a tedious process, which requires a lot of time and resources normally not available to election observation missions. Even if it should be done, the validation would be of a prototype (one selected machine, not each one produced) only. The replication of the software onto firmware or other media would be done by the supplier, and would again be subject to trust. An observation mission should not indicate or claim that they have validated the electronic voting system, and they should be particularly careful not to do this if the general confidence in the election administrators is low in the country.

Page 22: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

20 Observing Electronic Voting

Appendices

Appendix A. Election Related Conventions, Documents and Standards In the following a selection of declarations and conventions are mentioned, and the parts regarding the characteristics of the vote are quoted.

The United Nations Universal Declaration of Human Rights of 1948 is not a binding convention. However, it represents a strong statement and Article 21 states:

1. Everyone has the right to take part in the government of his country, directly or through freely chosen representatives.

2. Everyone has the right to equal access to public service in his country.

3. The will of the people shall be the basis of the authority of government; this will shall be expressed in periodic and genuine elections which shall be held by universal and equal suffrage and shall be held by secret vote or by equivalent free voting procedure.

The 1966 International Covenant on Civil and Political Rights also passed by the UN is, however, binding for the ratifying states. It states in Article 25:

Every citizen shall have the right and the opportunity, without any of the distinctions mentioned in article 2 and the opportunity, without any of the distinctions mentioned in article 2 and without unreasonable restrictions:

(a) to take part in the conduct of public affairs, directly or through freely chosen representatives;

(b) to vote and to be elected at genuine periodic elections, which shall be by universal and equal suffrage and shall be held by secret ballot, guaranteeing the free expression of the will of the electors;

(c) to have access, on general terms of equality, to public service in his country.

Regional conventions and documents include African, American and European documents.

The OAU/AU declaration on the principles governing democratic elections in Africa from 2002 includes the following:

II. PRINCIPLES OF DEMOCRATIC ELECTIONS

1. Democratic elections are the basis of the authority of any representative government;

Page 23: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

21 Observing Electronic Voting

2. Regular elections constitute a key element of the democratisation process and therefore, are essential ingredients for good governance, the rule of law, the maintenance and promotion of peace, security, stability and development;

3. The holding of democratic elections is an important dimension in conflict prevention, management and resolution;

4. Democratic elections should be conducted:

a) freely and fairly;

b) under democratic constitutions and in compliance with supportive legal instruments;

c) under a system of separation of powers that ensures in particular, the independence of the judiciary;

d) at regular intervals, as provided for in National Constitutions;

The American convention on human rights of 1969 states:

Article 23. Right to Participate in Government

1. Every citizen shall enjoy the following rights and opportunities:

a. to take part in the conduct of public affairs, directly or through freely chosen representatives;

b. to vote and to be elected in genuine periodic elections, which shall be by universal and equal suffrage and by secret ballot that guarantees the free expression of the will of the voters; and

c. to have access, under general conditions of equality, to the public service of his country.

2. The law may regulate the exercise of the rights and opportunities referred to in the preceding paragraph only on the basis of age, nationality, residence, language, education, civil and mental capacity, or sentencing by a competent court in criminal proceedings.

Protocol no. 1 to the European Convention for the Protection of Human Rights and Fundamental Freedoms states (1950) in Article 3:

The High Contracting Parties undertake to hold free elections at reasonable intervals by secret ballot, under conditions which will ensure free expression of the opinion of the people in the choice of the legislature.

The document of the Copenhagen meeting of the conference on the human dimension of the CSCE (later OSCE) (1990), states:

(7) to ensure that the will of the people serves as the basis of the authority of government, the participating States will

(7.1) hold free elections at reasonable intervals, as established by law;

Page 24: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

22 Observing Electronic Voting

(7.2) permit all seats in at least one chamber of the national legislature to be freely contested in a popular vote;

(7.3) guarantee universal and equal suffrage to adult citizens;

(7.4) ensure that votes are cast by secret ballot or by equivalent free voting procedure, and that they are counted and reported honestly with the official results made public; …

All of the regional conventions mention the right for people to freely participate in elections. This means that the choice shall be personal without intimidation and threats. All but the African Charter mention explicitly the secret vote.

Based upon the international declarations and conventions, a number of organisations have developed detailed standards for elections, or for observing elections. Some of these are: the EU, OSCE, SADC, International IDEA, Council of Europe, the Inter-parliamentary Union, The Commonwealth, and the Association of Central and Eastern European Election Officials.

Page 25: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

23 Observing Electronic Voting

Appendix B. IT Related Standards

Quality Management Systems

ISO 9000:2000 Quality management systems. Fundamentals and vocabulary

ISO 9001:2000. Quality management systems. Requirements

ISO 90003:2004 Software engineering. Guidelines for the application of ISO 9001:2000 to computer software

ISO/IEC 12207:1995. Information technology. Software life cycle processes

The TickIT Guide. Using ISO 9001:2000 for Software Quality Management System. Construction, Certification and Continual Improvement. (The BSI DISC TickIT Office. Issue 5.0, 2001)

In addition to these standards and guidelines there are methods for assessing an IT organisation such as COBIT and CMMI that are being used in the industry.

Product Standard

ISO/IEC 9126-1:2001. Software engineering. Product quality. Part 1: Quality model (Parts 2 to 4 cover use of measurement – metrics)

Security Standard

ISO/IEC 27001:2005 Information technology - Security techniques - Information security management - Requirements.

Page 26: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

24 Observing Electronic Voting

Appendix C. Abbreviations CMMI: Capability Maturity Model Integration (CMMI). Assessment method established

by the Software Engineering Institute

DRE: Direct Recording Electronic voting system

EMB: Election Management Body

EVM: Electronic voting machine

IEOM: International Election Observation Mission

ISO: The International Organization for Standardization

TickIT: A scheme devised by IT professionals, supported by the British Computer Society and managed by BSI DISC (British Standards Institution)

VVAT: Voter-verified audit trail

Page 27: OBSERVING ELECTRONIC VOTING - Forside · 4 Observing Electronic Voting The Challenges Electronic Voting in Transition Democracies In transition democracies the trust in the elections

25 Observing Electronic Voting

References

1. Christensen D.A., R. Karlsen and B. Aardal: På vei til e-democratiet? (About the trials on e-voting in municipal elections in Norway in 2003.) Institute for Social Research Oslo 2004.

2. Council of Europe: Report on the compatibility of remote voting and electronic voting with the standards of the Council of Europe adopted by the Venice Commission at it 58th Plenary Session, Venice, 12-13 March 2004.

3. Election Reform Society, The Independent Commission on Alternative Voting Methods: Elections in the 21st Century: from paper ballot to e-voting. London 2002.

4. Jefferson, D, A.D. Rubin, B. Simons and D. Wagner: A Security Analysis of the Secure Electronic Registration and Voting Experiment (SERVE). January 20, 2004.

5. The OSCE/ODIHR: Election Observation Handbook. 5th Edition. Warsaw 2005.

6. Rezende, P.A.D: Electronic Voting Systems. Is Brazil ahead of its time? Prepared for the First Workshop on Voter-Verifiable Election Systems, Denver; updated for DIMACS voting workshop, Rutgers University Piscataway, NJ, May 26-27, 2004 (v.7).

7. Sida and European Commission: Handbook for European Union Election Observation Missions, 2002.

8. Oostveen, A-M, and P. van den Besselaar: Security as belief. User’s perception on the security of electronic voting systems. Department of Social Sciences, NIWI-KNAW, the Netherlands.


Recommended