By FairVote -- The Center for Voting and Democracy (www.FairVote.org)
January, 2005

On November 2, 2004, the City of San Francisco used ranked choice voting (RCV) to elect seven member of its Board of Supervisors. The City is scheduled to elect at least one city office with RCV every November. This evaluation of the City’s first use of RCV reviews the measures of success for the election. We assess each measure of success based on data released by the Department of Elections. We also include results from an exit poll study conducted by San Francisco State University’s Public Research Institute, as well as results from an exit poll conducted by Chinese American Voter Education Committee (CAVEC). Finally, we include the results of aggregate precinct-level data analysis conducted by Professor Rich DeLeon, well-known San Francisco political scientist. 

Executive Summary:
In aggregate, multiple sources of data and analysis show that San Francisco’s first ranked choice voting election went remarkably smoothly. Quoting from the SFSU exit poll analysis, “The majority of voters appear to have made the transition to Ranked-Choice Voting with little problem…The overall finding on RCV is positive. Wide majorities of voters knew about Ranked-Choice Voting, understood it, and used it to rank their preferences. Further, most prefer it, with only about one in eight saying they prefer the former run-off system.” This successful use of ranked choice voting cut across all racial and ethnic lines, with only slight discrepancies by race, but none of them rising to the level of disenfranchisement. The SFSU results are reinforced by the results of an exit poll released by the Chinese American Voter Education Committee (CAVEC), which showed that only a small numbers of voters found the new system difficult to use, and in every racial group a majority of voters indicated the system was easy for them, with Chinese-language speakers having a bit more difficulty. 

At a pre-election news conference, the Center for Voting and Democracy established measurements for assessing the level of success for ranked choice voting. We can now demonstrate that, according to those measurements, this election demonstrated a high level of success.  On the flip side, we can also examine what the skeptics and opponents of ranked choice voting had predicted for the first election. The skeptics and opponents had predicted chaos and confused, angry voters; that voters would be so confused and angry it would drive down voter turnout; that the Department of Elections would screw up the election; that it would take weeks to know the winners; that minorities would be disadvantaged; that there would be long lines caused by confused voters; that the sky would fall, the Earth would open, and an earthquake would swallow us all. Needless to say, none of these came to pass.

Part I.

• Evaluation of Measures of Success
• In-depth Analysis of Effective Votes, Voter Turnout and Voter Rankings in RCV

Evaluation of Measures of Success

1. No more December runoff elections.
   San Francisco now has its December's back. There was no December runoff election in San Francisco for the first time since 1998, and voters did not have to trudge out to the polls in the middle of the holiday season.  The Department of Elections will run one fewer election per year, providing more time to prepare for the next election. This will help the Department run better elections.
   
   
2. Quick and timely results.
   Three out of seven of the supervisorial winners were known on election night.  The other four winners were known within 72 hours after the polls had closed.  If not for a brief delay in reporting results caused by a minor programming error by the City's vendor, all winners would have been known within 24 hours after the polls had closed. With that error fixed, the winners in future elections should be known within 24 hours, except for races that are extremely close.
   
   
3. Significant tax savings.
   The City saved approximately $1.2 million by not having to administer runoff elections for four supervisorial districts (According to figures released by the Elections Commission in 2003, citywide runoff elections in San Francisco cost taxpayers at least $3 million to administer. Administering runoff elections in any one of the 11 supervisorial district races costs a prorated amount. Taxpayers also saved the costs of public financing for supervisor races, which provides up to $17,000 in public funds to candidates in runoff elections, a savings of up to $34,000 per supervisor runoff as well as the administrative costs of running the program).

   The City incurred one-time implementation costs of $1.6 million for upgrading the voting equipment. There also were expenses of $800,000 for community education and outreach. Avoiding future runoff elections will quickly repay the one-time costs of implementing RCV, leading to substantial ongoing savings to San Francisco taxpayers.
   
   

4. Winners received significantly higher percentage of total votes cast than winners in December runoffs; more voters had a say in who their supervisor is.
   With the “instant” runoff, winners received significantly more votes and overall support than winners in December runoffs (and especially more than winners in conventional plurality voting elections). By getting the election over in November, during a presidential election year, more votes were cast in the decisive election and winners received more votes both in real terms and as a percent of the vote than the old “delayed” runoff system. And that means more voters had a say in who their supervisor is.

   In contrast, for the previous non-RCV supervisor elections in a presidential election year (2000), relatively high voter turnout elections in November were followed by runoffs with sharply lower voter turnout. The average decline in voter turnout from November to December was 42.3% in 2000. Winning candidates received a majority of the low turnout December electorate in those two runoff elections, but when compared to the total voters who participated in that supervisorial election in November, winning candidates in December received a low of 28% and a high of 45% of the November turnout, with most races in the lower end of this range. But winners in the 2004 RCV races received anywhere from 48.7% to 37.6% of all votes cast in their respective races. In addition, in the RCV races, there were far fewer “exhausted ballots” than in the 2000 non-RCV races (for this comparison, we have counted voters who do not return and participate in the 2000 December runoff as exhausted).  See the tables below.
   
   

Supervisor Races requiring an “instant runoff”, November 2004

Board of Supervisors races, 2000

Thus in the 2004 RCV elections, all winning candidates in the “instant runoffs” had larger vote totals and percentages than winners in 2000 in December’s “delayed runoff.” Winning candidates ultimately won a greater share of the valid ballots (that is, of the original turnout) than most of the winning candidates in the December 2000 runoffs. In any runoff system, the winning candidate must win a majority of valid ballots cast in the final round of counting between the top candidates (e.g., the “continuing ballots”) rather than a majority of total ballots that might have been cast for the race.

Moreover, support for winners was significantly higher if determined by the number of people who ranked the winner with at least one of their three rankings. For example, even though District Five had 22 candidates, winner Ross Mirkarimi was ranked by 47% of voters. Every other winner drew at least one ranking from at least 53% of voters (these numbers are not shown on the charts above).

• Winning candidates in 2000 received between 5,887 and 10,384 votes. Winning  candidates in 2004 received between 10,679 and 14,011, and were ranked on 12,200 to  16,900 ballots.

Valid votes, undervotes/drop-off and overvotes in Nov 2, 2004 RCV Supervisor Races
The two tables below show several important things:  1) the Department of Elections incorrectly reports the numbers of exhausted ballots because they lump into this figure the number of undervotes (that is ‘drop-off’ voters who don’t rank any candidates in supervisorial races) and overvotes (voters who rank more than one candidate for their first ranking, spoiling their ballot) See Appendix One; 2) the number of undervotes/drop-off is generally LESS in RCV elections compared to other non-RCV San Francisco elections; and 3) the number of overvotes in RCV races is quite a bit higher than non-RCV races, but still the numbers are so small as to be insignificant for most elections. A high of 1.1% means 11 out of 1000 voters spoiled their ballots in a RCV race, while the low of 0.1% in non-RCV races means 1 out of 1000 voters spoiled their ballots – a difference of 10 voters/ballots.

Table 1. Valid votes, overvotes and undervotes (also known as DROP-OFF) in RCV races

(Total voters – Invalid ballots = valid ballots) 
Overvote means a voter selected more than two candidates for their first ranking.
Undervote/drop-off means voter ranked nothing on their ballot

Overvote means a voter selected more than two candidates for the same office. Undervote/drop-off means voter selected no candidate for that race.

Effective use of rankings
We looked at the extent to which supporters of losing candidates made effective use of of their #2 and #3 rankings. Of course if voters support one of the final two candidates with a #1 choice, it doesn’t matter whether or not they rank candidates second and third: their vote will count for their #1 choice in all rounds. But if their #1 choice is one of the less popular candidates who gets eliminated before the final two, then it can be important whether the voter used a #2 and #3 choice to support one of the contenders.

What this shows for the hotly contested District 1 race is that most voters made good use of their rankings.  Voters on average cast 2.53 rankings. The number of rankings cast by voters did not vary appreciably among supporters of different candidates, ranging from a low of 2.41 for McGoldrick to a high of 2.69 for Tuchow. Supporters of Asian candidate Lillian Sing ranked an average of 2.56 candidates, showing her supporters made good use of their rankings. Supporters of Asian candidate Rose Tsai ranked an average of 2.58 candidates, her supporters also making good use of their rankings.

These results give a sense of the effectiveness of the City’s voter education, poll worker training, and “error notification” on the Optech Eagles. Did voters effectively use the ranked choice system? Did they rank three candidates? It would appear that in the District One race the answer generally is YES. These are important indicators that point the direction for improvement and for ongoing education about ranked choice voting in San Francisco.

Part II. Summary of Exit Poll analysis.
An exit poll about voter’s attitudes regarding ranked choice voting was prepared by the Public Research Institute at San Francisco State University. This exit poll was commissioned by the City and County of San Francisco and paid for by the City and County and SFSU College of Behavioral and Social Sciences. The exit poll was conducted to gauge the ease or difficulty with which voters expressed their preferences on the new form of ballot. The survey, which was translated into several different languages, included a sample of 2,847 voters from city supervisor districts 1, 2, 3, 5, 7, 9 and 11. Among various findings, the exit poll found that:

• 87% of those San Franciscans polled understood ranked choice voting.
• 61% preferred the new system, and only 13% said they preferred the old runoff system (27% said it made "no difference" to them)

The report concludes that “The majority of voters appear to have made the transition to Ranked-Choice Voting with little problem…The overall finding on RCV is positive. Wide majorities of voters knew about Ranked-Choice voting, understood it, and used it to rank their preferences. Further, most prefer it, with only about one in eight saying they prefer the former run-off system.”

Overall, 52 percent of those surveyed said they understood ranked-choice voting “perfectly well”; 35 percent said they understood it “fairly well,” an impressive total of 87 percent who had a decent level of understanding. About 11 percent said they “did not understand it entirely,” and another 3 percent said they “did not understand it at all.” Results indicate that only 13% of Asians and 15% of Chinese speakers reported a lack of understanding of RCV, compared to 12% of whites and 23% of Spanish speakers. 70% of those who spoke English or Chinese as a first language knew ahead of time they would be using RCV, more than those whose first language was Spanish (22%). Nearly the same percentage of Asians and whites ranked three candidates, 58% to 62%, both higher than Hispanics (53%) and African Americans (49%).  Voters with lower levels of education and income, as well as language minorities, reported less understanding, but even within those categories and demographics the differences were not large.

From the Executive Summary

• Over two-thirds (69%) of voters surveyed knew that they would be asked to rank their choices for the Board of Supervisors, while almost one-third (31%) were unaware prior to coming to the polls.
  
  
• Prior knowledge appears to have lessened the potential for language-based difficulty in using the RCV ballot.
  
  
• A majority (59%) of voters surveyed reported ranking three candidates; 14% reported ranking two, and 23% reported ranking only one candidate.
  
  
• Two-thirds (66%) of those who knew of RCV prior to coming to the polls ranked three candidates versus 47% of those who were unaware of the new development.
  
  
• Sixty-three percent of those who understood RCV at least "fairly well" ranked three candidates, while only 36% of those who did not understand it entirely or at all ranked three candidates.
  
  

The entire study can be viewed at http://pri.sfsu.edu/reports.html. Note that this is a preliminary release of results, SFSU expects to release even more results in early February 2005.

CAVEC Exit Poll
The Chinese American Voter Education Committee (CAVEC) also conducted an exit poll survey of 2,108 San Francisco voters. Their poll mostly confirmed the results of the SFSU poll.  Those respondents expressing an opinion about the system overwhelmingly expressed support for it. Large majorities found that the system was not difficult to use, and that cut across all racial lines (with Chinese-language minorities having more difficulty than other Asians).

Despite the first RCV election taking place in a year with high voter turnout where most media attention was focused on the presidential race, only 18% of voters found the new system difficult to use. In every racial group a majority of voters indicated the system was easy for them.

• Overall, 67% of voters found it easy to use, compared to only 18% who found it difficult.  (The rest did not express an opinion.)
  
  
• 74% of Latinos found it easy to use, only 14% found it difficult.
  
  
• 71% of whites found it easy to use, only 13% found it difficult.
  
  
• 59% of Asians found it easy to use, 27% found it difficult.
  
  
• 57% of blacks found it easy to use, 35% found it difficult.
  
  
• 49% of Chinese-speakers found it easy, 39% found it difficult.
  
  

Obviously there is more work to do in terms of education and outreach, but these numbers are generally positive.

Part III. Precinct data analysis by Professor Richard DeLeon
San Francisco State University professor Richard DeLeon has been analyzing aggregate precinct data from the ranked choice voting elections, particularly with an eye for impact on minorities.  Professor DeLeon sorted all the precincts by racial demographics according to the most recent Census data, and focused on two districts (Districts 1 and 11) where there were Asian candidates, significant number of Asian voters, and multiple rounds of counting to determine a winner. He used multiple regression analysis to study nine hypotheses/scenarios (such as a higher proportion of exhausted ballots, a lower number of rankings used in voting, a greater percentage of "bullet voting" i.e., ranking only one candidate, a lower proportion of "effective ballots" i.e., votes cast that help to place one or both of the final two candidates into the final round) that, if were true, would suggest Asian voters in those districts had more difficulties with the system than whites or other racial groups.  Professor DeLeon concluded: "Nine hypotheses with clear predictions were tested in each district, adding up to 18 opportunities for the available empirical evidence to reveal patterns of data at least consistent with, if not proof of, the arguments advanced by some critics that SF's new RCV system systematically disadvantages the city's Asian voters vis-à-vis voters in other racial/ethnic groups. Based on the evidence presented here, the score is zero for 18." In other words, there is absolutely no proof from this precinct data that Asian voters had more difficulties with ranked choice voting than other racial groups, including whites.

Other findings by Prof. DeLeon:

• Of the 203,009 voters who cast a ballot in the Supervisor races, 83.7% included the winning candidate or runner-up in their first, second, or third choice votes.
  
  
• Of the 203,009 voters who cast a ballot in the BOS election, 63.3% included the winning candidate in their first, second, or third choice votes.
  
  
• Of the 72,826 voters who cast a ballot in the Supervisor races and ranked some other candidate than the winner or runner-up as first choice, 54.5% included the winning candidate or runner-up in their first, second, or third choice votes.
  
  

More analysis will be forthcoming from various sources in the coming weeks and months. Having complete voting data for seven different Supervisor districts is a terrific advantage. In effect, San Francisco conducted 7 semi-independent RCV natural "experiments," with significant variation across the 7 districts in socio-demographic and racial/ethnic composition, local political culture and ideology, and electoral conditions (number of candidates varying from 4 to 22, some incumbents running, some not, etc.). Any findings/generalizations that hold up across those wide-ranging socio-demographic and political contexts must be considered especially robust. Where generalizations can't be made, the data provide insight into the conditional relationships that might be operating.

Conclusion.
Multiple data sources and analytical methods show that San Francisco’s first ranked choice voting election went remarkably smoothly. Voters ranked their candidates, and winners in all seven races were declared within 72 hours after the polls closed (in three races winners were declared on election night), winning candidates won with more votes both in real terms and percent terms, and there were more effective votes, than the winners in the December 2000 runoff election. The results and margins of victory have been recognized by nearly all observers as legitimate and substantial, even as San Francisco gets its Decembers back and saves millions in taxes. Also, several of the races were marked by less mudsling and more coalition-building and issue-based campaigning than in previous San Francisco elections, because with ranked choice voting candidates have incentive to build coalitions rather than attacking opponents as a successful winning strategy, since winners may need to attract the second or third rankings from the supporters of other candidates.  In fact, a New York Times profile of the RCV races was headlined "New Runoff System in San Francisco has the Rival Candidates Cooperating."

Certainly this does not mean there was not some confusion among some voters on election day, or room for improvement. Unsurprisingly, the very first election using ranked choice voting in San Francisco leaves room for refinement. Credible studies and analyses will be helpful in pinpointing where future education efforts should be directed.

Nevertheless, by any objective measurement, the first RCV election in San Francisco was a success.  San Francisco voters, poll workers, and especially the Department of Elections and its vendor Elections System and Software, deserve a big congratulations.

Back to www.sfrcv.com