Notes regarding the 2006 Primary Election Logic & Accuracy Test.

Test Conducted July 17, 2006, 9 am - 7 pm, and July 18, 2006, 9 am – 5:30 pm.

Test Board

Democrat                                Republican                            

Jean Day Alexander                 Margaret  ?    ?   Anderson

Donald F. Hayden, Jr.              Barbara Olds Anuta

Barbara Longo                         Christine M. Lucas

Nancy Jo Wurl                         Claudia R. Schwartz

The Board operated in teams consisting of one Republican and one Democrat.  . 

Observers

Paul Hultin                                Deb Gardner

Al Kolwicz                               Joe Pezzillo

Tom Morris                              Neal McBurnett

Ivan Meek                                Linda Salas

This report is focused on the public questions and alarming government answers portion of the Logic & Accuracy Test.  It does not go into the many, many technical issues discovered by observers.  For example, the time clock in at least one JBC device was incorrect.  (At 10 AM the clock read 11 AM.)  This can mean that the election workers and test board members were not attentive to detail, or more worrisome, that the equipment is illegally connected to a server that resides in a different time zone.  In either event, the reports and data cannot be synchronized with observer notes for future investigation.

Principal complaints

1.      Election officials have made it impossible for the public, poll watchers, canvass board members and the press to see or hear anything.

The test board, selected by the clerk, performed their duties behind a glass wall.  It is true that observers could see that people were doing something, but it was not possible to see or hear what they were doing. 

That the test board operates behind a glass wall is totally unsatisfactory.
 

2.      That the clerk selects the test board members rather than the political parties is currently legal but totally unsatisfactory.  It appeared that most of the people on the test board were not qualified testing or systems experts.
 

3.      That the test procedures and rules for observation were not published in advance, made it impossible for observers to prepare for the test.
 

4.      It appeared that the Test Board did not design any of the tests; the county did.  This means that the test was not s fair and independent verification of the voting equipment..
 

5.      It appeared that the board had not studied the materials that were handed out, and possibly that the board did not have access to any of the information before the start of the LAT.  This prevented collaboration with political party experts on test designs, and test strategy. 

6.      Observers were not provided with important data including:  (a) names and party affiliation of the members, (b) copies of all forms and reports used by the test board, (c) samples of each ballot style, (d) a list showing how many of which precinct ballots was marked by each board member, (e) samples of all computer screens, and (f) detailed instructions and procedures that were followed by the test board.

Public questions & Government answers

Following the test, which ended at 4:20 pm, I asked for the opportunity to ask some questions – as provided for in the rules.  At 4:45 pm a hallway meeting was convened with Salas, Liss, Hansford, Stahl, Wurl, Hayden, Anuta and me.

Q1.      Will there be a written report of the test that the public can read?

A1.      No, and the materials that were produced during the test are sealed.

Q2.      Were there any deviations from the procedures specified in the handouts (below)?

A2.      No.

Q3.      How many of the various style-precinct ballot combinations were tested?

A3.      All combinations were tested.
 

·    There are 13 ballot styles (combinations of contests) for each party.  Each style is used in one or more precincts.

·    There are 236 precincts (234 plus 2 Federal).

·    There is a Republican ballot and a Democratic ballot for each precinct.

·    In addition to one Republican and one Democratic ballot per precinct (472 total), an additional 13 Republican ballots and 13 Democratic ballots (one of each style) and an additional 3 Republican and 3 Democratic ballots (undetermined styles) were used for testing multiple contestants in a contest, write-ins, and over and under votes.

·    The total number of ballots prepared by the test board was 504. 

·    All ballots are “live” ballots.  There are no “test” ballots.

Q4.      It is our understanding that absentee and precinct paper ballots are pre-printed by a vendor, and early-voting paper ballots are printed on demand.  How many of the test ballots were printed as (1) absentee, (2) precinct, and (3) early?

A4.      Absentee ballots are printed in-house, and are not marked “absentee”.  Precinct ballots are printed by an outside vendor.  Early ballots are printed on demand.

NOTE:  The response did not answer the question.  Also, the response appears to conflict with the statutes which say that absentee ballots are marked “Absentee”:
 

1-8-101. Ballots and supplies for absentee voting.

 (2) The ballots shall be in the same form as other official ballots for the same election. On the stub of the absentee ballot shall be printed "Absentee Ballot No. A. V. ........(number)", and such stubs shall be numbered consecutively, commencing with number 1.

Q5.      Since all of the paper ballots from all of the test board members were counted as “absentee” and all of the electronic ballots from all of the members were counted as “early”, was there any attempt to verify that TALLY (a) correctly totals early votes when they come from both paper and electronic ballots, and (b) correctly totals precinct votes when they come from both paper and electronic ballots?

A5.      No.  Only the GRAND TOTAL mechanism was tested.

            NOTE:  This appears to conflict with the election rule: 11.5.3.7

11.5.3.7 The test ballots shall be tested on each type of voting device utilized in a given election and each method of counting. The tests shall include testing of absentee counting methods, election day counting methods, provisional ballot counting methods, early voting counting methods and audio ballots, if applicable.

Q6.      Are all of the equipment, election data, and operator setup options used for the test identical to those that will be used for the live election?

A6.      Yes.

Q7.      Is the election database used for the test identical to the one that will be used for the live election?

Q7.      Yes.  They are identical.  Each board member used an identical copy of the live election database.

NOTE:  Each test board member marked one set of 63 ballots.  We do not know if they were all for the same political party or mixed.  If they are not mixed, then each eSlate would have been tested for only one of the two parties.

The first member’s test set was scanned on the first optical scanner and written to the first MBB, the second to the second to the second, etc.  No ballots were scanned on more than one optical scanner.  This avoided the problem detected in previous LAT where the same set of ballots produced different results on different scanners.

The votes from ballots in the first member’s test set were then manually entered into the first eSlate and the test set was written to a nineth MBB, etc for all eight sets for a total of sixteen MBB’s.. 

Because testing a DRE is totally different from testing an optical scanner, the fact that the same tests were used for both purposes made them essentially useless.  DRE tests would need to include such cases as when a voter walks away without casting their ballot.  Or a voter who gives up and switches to a paper ballot.  Scanner tests would include such things as markings that are not optimal for the equipment, required duplication, and ballot folds.

After a Test Board member’s two MBB’s were written, the 2 MBB’s were read into the TALLY system.  We could not tell if they were collected in one or two batches.  A report was produced which tallied that member’s paper ballot choices in the absentee column, and that member’s eSlate choices in the early voting columns of the report.

The reports were then verified by the team.

When all Test Board results were correct, all 8 batches were tallied into a combined report.

Q8.      Were the text printed votes on the VVPAT paper roll verified?

A8.      Yes, they were verified at the time each ballot was cast, and again during the verification process.

Q9.      Were the bar-coded votes printed on the VVPAT paper roll verified?

Q9.      No.

Q10.    Was there any attempt to verify that the judges report forms were sufficient to meet the needs of the Canvass Board?

A10.    No.

Q11.    Was there any attempt to verify that the forms used to verify the integrity of batches were able to detect errors?

A11.    No.

Q12.    Were any provisional or duplicated ballots introduced to verify that they do not disrupt the ability to balance reports?

Q12.    No.     

Q13.    Was there any test to verify that the data produced by the election processes are sufficient to meet the needs of the Canvass Board?

Q13.    No.

            NOTE:  The lack of testing here is particularly concerning.  It is not clear that the system can be audited, as required by Colorado law, and as described by Neal McBurnett and Paul Walmsley.

Q14.    Do the reports show for each contest by precinct, and for each method of voting (absentee, early, precinct, and provisional), the number of votes counted for each contestant, with over votes and under votes?

A14.    Precinct level reporting was not tested.

Q15.    May I review the test history?

A15.    Yes.

NOTE:   Tim Hansford had already collected prior reports.

1.       Only optical scanner reports were provided for review.  Some scanner reports for the following elections were provided.

a.       2004 General election

b.      2004 Primary Election

c.       2005 City of Boulder Election

d.      2005 Coordinated Election

2.       No test-input specifications or reports describing any problems that were encountered during these prior tests were provided.

3.       No Acceptance Tests were provided for review. 

It was explained that no real tests were performed.  The vendor-recommended procedure for unpacking the equipment was followed.  The procedure does not include any functional tests.  The procedure does not include any system tests that verify that the equipment integrates with the other election equipment.

4.       Tests that we expected to see that were not provided for review include:

a.       Verification that the voter eligibility and districting data on the voter file is correct

b.      Verification of the absentee ballot process including ballot control, pollbook, and ballot qualification including signature verification

c.       Verification of the election setup tables

d.      Verification of the TALLY system

e.       Verification of the provisional ballot qualification and counting processes

f.        Verification of the voter eligibility procedures at precinct and early voting locations

g.       Verification that the voter gets the correct ballot

h.       Verification that judges can and do follow instructions

i.         Etc.

Materials

The materials available to observers at the time of the LAT were:

1.       Guidelines for Observers

2.       2006 LAT Agenda and Procedures – part 1

3.       2006 Primary Election LAT  Procedures – part 2

4.       CRS 1-7-509

5.       Election Rule 11.5.3

6.       Sample Republican ballot for precincts 28, 29, and 69 thru 123

7.       Sample Democratic ballot for precincts 28, 29, and 69 thru 123

8.       eSlate Voter Instruction Script

9.       Hand tally worksheet – Republican – Federal and State contests

10.   Hand tally worksheet – Republican – County contests

11.   Hand tally worksheet – Democratic – Federal and State contests

12.   Hand tally worksheet – Democratic – County contests