How did our estimates for CHIP50 compare to what happened on election day? As with 2020, CHIP50 underestimated Trump’s vote share. If we plot our estimates against actual outcomes for the two party vote, we see that we overestimated Harris’ vote share by about 3.4 points (figure 1). In contrast, the estimate of Harris’ vote share from 538 polls was a bit closer, overestimating her vote share by a bit less, 2.1 points, aggregating 538 for the same period of data collection for CHIP50, August 19 to November 3rd. (We note that the final estimate for 538 only overshot Harris by about a little over a point, because there was a downwards trend for Harris over the period we were in the field.) 

Figure 1 suggests an intriguing pattern; CHIP50 overshot Harris far more in heavily Republican states; and by very little in heavily Democratic states, underestimating her vote share in only 6 states, all of them where her vote share was well above 50%. We don’t have a definitive explanation for this pattern (if you have hypotheses, please reach out to us). In contrast, the 538 polls systematically overshot Harris’s vote share by about two points, as noted (Figure 2); with no “tilt” in the relationship, with very low spread in that relationship. (We omit 13 states because there weren’t enough polls to offer an estimate.) Note that our collective prior expectation should be that 538 would do better; it captures the input from many polls per state from professional pollsters whereas CHIP50 is a single set of polls that aren’t optimized to provide predictive signals. Beyond that, taking 538 polls as an indicator for the industry, pollsters on average did very very well in 2024 by historical standards.

Why did CHIP50 systematically overestimate Harris’ vote share? Besides the apparent trend towards Trump during October, we suspect that much of our miss was because of gaps in turnout by Democrats as compared to Republicans. As we noted in our earlier posts, we did not incorporate into our estimates the likelihood of individuals actually voting. (Professional pollsters do this, but to do this well would require far more resources than we have.)

Whether turnout was a driver of the systematic misses in figure 1 will become clearer over the next few months, as final voter files become available. However, analysis of overall turnout indicates that heavily Democratic areas had major drops in turnout relative to 2020 (and Republican areas did not). Thus, to take just a few examples of very heavily Democratic areas with major drops in turnout: Cambridge, MA, had a drop in turnout of 15%; Chicago a drop of about 8%; Cleveland a drop of 10%.These were part of a broader pattern of strongly Democratic areas having lower turnout than in 2020. And, unsurprisingly, if we assume individuals who indicated that they preferred Harris were less likely to vote, we end up with an estimate that is much closer to what actually occurred. Figure 3 provides estimates based on the assumption that voters who preferred for Harris were 10% less likely to turn out as those who preferred Trump. This is a plausible estimate, and would be roughly what would yield unbiased estimates for CHIP50.

We will return to the question of why our estimates deviated after voter data are available. Voter data include party registration and whether people voted; and we will be able to evaluate definitively whether and how much less likely Democrats were to turn out relative to Republicans.

Interestingly, we were correct in the evaluation that the electoral college was no longer biased in favor of Trump, because Harris had a much smaller drop in vote share in the swing states than in other states.

Finally, our demographic prediction tool did quite well in anticipating the demographic shifts that exit polls in turn illuminated. Thus, for example, we predicted big shifts among Hispanics and Asian Americans; that there would be a huge drop among young people in support of Harris relative to Biden in 2020; that there would be further education polarization. (Btw, on some of the broader demographic shifts, and, in particular, the shift towards a Republican, multi-racial working class coalition, we’d recommend Patrick Ruffini’s book, Party of the People; which was quite prescient with respect to racial depolarization + education polarization.) Our demographic prediction tool allows a deeper dive into these patterns, highlighting that these racial/ethnic/gender shifts were much much larger among younger individuals. If you look at men, ages 30 years old and younger, high school or less educated, African American, Hispanic, or Asian American voters, they supported Biden by a 63 point margin in 2020. In 2024, they supported Trump by 3 points; an incredible 60 point swing in margin. By comparison, if we look at the same demographic groups, but 60 and older, the margin was +62 in 2020, and +52 in 2024 for the Democrats; a much smaller 10 point swing in margin.