The joys of averaging

The social sciences often deal with attitudes that are difficult to measure in a robust and consistent way; voting choice, for example, or party identification. One possible solution, offered in a new paper by MIT political scientists Stephen Ansolabehere, Jonathan Rodden and James M. Snyder, is to average large number of survey items. Here is the abstract for the paper, Issue Preferences and Measurement Error (PDF):

We show that averaging a large number of survey items on the same broadly-defined issue area - e.g., government involvement in the economy, or moral issues  eliminates a large amount of measurement error and reveals issue preferences that are well structured. Averaging produces issues scales that are stable over time, and with enough items, these scales are often as stable as party identification.

The scales also exhibit within-survey stability when we construct scales made from disjoint subsets of survey items. Moreover, this intertemporal and within-survey stability increases steadily, and in a manner consistent with a standard common measurement error model, as the number of survey items used increases. Also, when we estimate Converse's "black-white" model, we find that at most 20-25 percent of respondents can be classified as "pure guessers", and 75-80 percent have stable attitudes over issue areas, the reverse of Converse's conclusion.

Finally, in regressions predicting presidential vote choice, the issue scales appear to have much more explanatory power - relatively large coefficients and much larger t-values - than any of the individual survey items used in constructing the scales.

This is an important paper, with uses well beyond political science. After reading it I came across a long post by Andrew Gelman about the paper: Political attitudes look more stable over time if measured by averaging responses to several issue questions. Gelman comments:

This seems like an important paper to me in getting closer to understanding issue attitudes and polarization. It seems that you get a lot by combining questions...

I think this paper is important for substantive reasons (it's interesting that voters are more issue-consistent than researchers have thought), but also statistically-- I'm always trying to make the point to students and collaborators that it's a good idea to combine questions into "scores," and it's nice to have this example. Sometimes items can be combined using factor analysis, but often simple averages are effective (as Robyn Dawes and others have pointed out in the context of psychological measurement).

Comments

[pandora jewellery]

Interesting perspective on India ... thanks!

[nike jordan retro]

Averaging produces issues scales that are stable over time, and with enough items, these scales are often as stable as party identification.

[Nike Jordan]

Honesty may be the best policy, but it's important to remember that apparently, by elimination, dishonesty is the second-best policy.