In a recent telephone conference, Chairman Sam Hunt asked me to explain why I thought it was important for Washington State to perform audits. First, I want to be clear on what I mean by an audit — a hand count of the votes on paper ballots to check that the election equipment is tallying correctly. I have three main points that support the need for these audits. Equipment can malfunction without showing noticeable signs. Election-specific ballot programming is a point of extreme vulnerability in every election. Testing is insufficient to ensure that results will be accurate. Equipment malfunctions can occur in either hardware or software. Sometimes the malfunctions are noticeable, and election officials investigate. Audits are needed for the times when malfunctions aren’t noticeable. An example of an optical scanner hardware malfunction: In January of 2003, it was noticed that there were 21,000 missing votes for legislative races in the Snohomish County November 2002 election. That was 25% fewer votes than voters. An investigation showed that the optical scanners were failing to read all the marks on absentee ballots. As the read-lights wore out, they stopped reading some kinds of ink. The Snohomish County Republican party had been noticing the undervote rate climbing “for a while,” so it remains unclear how many elections had been affected. In this case, the 21,000 missing votes didn’t affect the outcome, but there is no way of knowing whether the malfunction impacted results of previous elections. It’s very likely that an audit would have detected this problem long before the GOP noticed it. An example of an optical scanner software malfunction: In November, 2004, the Grays Harbor County Auditor, Vern Spatz, suspected a problem when he saw that the turnout for the election appeared to be 93%. He investigated and found that some of the ballots had been counted twice. So, he did a full recount. Mr. Spatz assumed his staff had downloaded results from some of the disks twice, but his assumption may not be correct. First, the ES&S election management system is supposed to prevent that from happening. And second, similar double-counting was reported in other states (Nebraska and Ohio) using ES&S scanners, and in two cases it was determined that the scanner was reading and counting each ballot as if it were two ballots. The election commissioner in Lancaster County, Nebraska was surprised because they had successfully tested the machine before the election. Ballot programming refers to creating ballot definition files (BDFs), which convert the marks on a paper ballot or touches on a screen into votes counted for a particular candidate or issue. These files are unique for each ballot style in each election. The underlying software uses the BDFs as it counts and tallies votes, and it is the BDF that determines the assignment of votes to candidates. You may have heard, “Never buy version 1.0 software.” This is because it’s new and doesn’t have the obvious bugs worked out. Yet, every ballot definition file is version 1.0. It’s new, specifically created for that one election, and discarded after one use. For many counties, even in Washington, the voting equipment manufacturers provide the ballot programming. ES&S, in particular, is notorious for providing inaccurate ballot programming. Errors like those below occur every election, and they will continue to occur because the ballot programming is always new and untried, until the election provides the first, and only, field test. Clay County, Kansas. August, 2002. The machine showed that the challenger (Jennings) had won, but a hand recount showed that the incumbent commissioner (Mayo) won by a landslide — 540 votes to 175. For one ward, which Mayo carried 242-78, the computer had mistakenly reversed the totals. The ballot programming had reversed the mapping of marks to candidates. Sarpy County, Nebraska. November, 2002. The optical scan machines failed to tally “yes” votes on the Gretna school-bond issue, giving the false impression that the measure failed miserably. The measure actually passed by a 2-1 margin. Responsibility for the error was attributed to ES&S, which provided the ballots and the machines. Craighead County, Arkansas. May, 2004. The chip programmed by ES&S for the county's optical scanner gave one candidate all the votes for constable. A manual recount revealed the error. Pottawattamie County, Iowa. June, 2006. Flawed ballot programming by ES&S reported results of all nine contested primary races incorrectly. Every winner in Pottawattamie County's nine contested races turned out, in retrospect, to be a loser. For example, initial returns showed incumbent Recorder John Sciortino losing by a margin of 1,245 votes to 1,167, but a hand count showed he actually won the election 2,061 votes to 347. St. Francis County, Arkansas. June, 2006. A recount of the State Senate District 16 runoff primary race reversed the initial, incorrect results caused by a ballot programming error by ES&S. Results originally showed Representative Arnell Willis defeating Earle School Superintendent Jack Crumbly by 28 votes. However, a recount in St. Francis County gave Crumbly 100 more votes, making him the winner. Testing is inadequate. In Washington, three phases of testing are performed on election equipment before election day: 1) testing by test labs to ensure the system meets federal standards; 2) testing by the state to ensure that the equipment meets functional requirements and operates correctly; and 3) pre-election testing by county officials. However, each of these phases is inadequate to ensure that votes are tallied correctly. 1) The process of testing to federal standards is dysfunctional. This is not just my opinion. This is a well-known fact agreed upon by state examiners, computer scientists, and experts called in to testify before Congress. Last September, in a Colorado court case, Rice University professor Dan Wallach testified that lab reports indicated the testers simply skipped some of the required tests, for example, checking for protection against viruses and for the security of data being transmitted from polling places back to the central headquarters. Last month, it was revealed that the federal Election Assistance Commission refused to approve an interim accreditation of Ciber test lab, which has tested 68% of the equipment now in use. The New York Times reports that the lab “was not following its quality-control procedures and could not document that it was conducting all the required tests.” 2) Normally, Washington State relies heavily on the flawed testing process described above. Testing at the state level in Washington is minimal, consisting of little more than running test ballots through the system. The State does not even hire professional software testers to design the test ballots; the Secretary of State’s staff constructs and implements each test, in consultation with the vendor. In the test I observed, the set of test ballots was very poorly designed and failed to fully exercise the system. Adding to the problem, the Secretary of State bends the regulations when he deems it appropriate. For example, in the September 2004 primary, six counties — including King — used the new consolidated ballot for the first time. Starting in July, three manufacturers began development of new software to handle the “pick a party” ballot. With only a few weeks allowed for the projects, development was rushed. No independent labs tested the software, so the state relied on software quickly developed and quickly tested by vendors to count the votes of over half the state. Any software developer will tell you that adding new features to software is very risky. Unexpected problems can be introduced in areas that previously functioned correctly, and these types of problems are impossible to detect without thorough testing of the entire system. But the Secretary of State authorized the use of this new, untried software, not only for the September primary, but also for the November 2004 general election. Problems did occur. The Diebold scanners rejected ballots on which the voters chose not to vote in any party primary. That problem was noticeable. Only a robust audit would have shown if there were other, less noticeable, problems. The Secretary refused to conduct an audit. 3) Pre-election testing at the county level is crucial, because it is the only phase that tests the ballot programming. However, pre-election testing is even less robust than the acceptance testing at the state level. The state provides little guidance for creating a test plan, and most auditors have no experience testing software. As a result, their test plans often fail to test for even the most obvious flaws. For example, when I attended the testing in Jefferson County, the test ballots gave the same number of votes to both candidates for a specific office. So, if the ballot programming had mis-mapped the marks for those candidates (as happened in Clay County, Kansas), the testing would not have detected the error. Each candidate would have received the other’s votes, but the results would have been as expected. Checking for this type of mis-mapping is one of the simplest checks. Other more sophisticated, necessary checks were also absent. Even if all the testing were excellent, software always has bugs. In well-tested software, the inevitable bugs show up only in unusual circumstances, that is, situations that might occur in an election but that testers wouldn’t think to test for. A few years ago, I was hired to test a new release of a software product that had been used in the scientific field for many years. I found a bug in a calculation module that had been in the product since its first release, probably because I was new to testing the product and tried some things that other testers had not thought to try. Testing can find bugs, but it cannot determine that there are no bugs. There is never any assurance that software doesn’t have errors in it; and if someone were to insert malicious code, they would be very careful to make it virtually impossible to find. Even diligent, controlled testing does not guarantee that the product will perform reliably during an election. As the report from a June 2004 Harvard University symposium on voting systems points out: Testing is necessary but not sufficient for a well-run election. Testing is never perfect, as it can overlook certain factors or interactions that may be easier to detect in hindsight. Systems interact with each other in unpredictable ways, often impossible to detect in a reasonable battery of tests. The report unequivocally states, “testing before the vote cannot verify accuracy of final tally,” and declares, “Equipment testing does not displace the need for outcome auditing.”   Even auditing isn’t absolute assurance of accuracy, but a robust audit would be likely to show some indication of a problem most of the time. Then, miscounts could be investigated to determine the cause and the real outcome. Demonstrable accuracy in election outcomes is the only way to preserve democracy, and audits are an essential part of ensuring and demonstrating accuracy. We know more today about how to build a machine to take pictures of rocks on Mars than we know about how to build a machine to safeguard the American right to vote. ~ Rev. DeForest Soaries first Chairman of the EAC