Have Republicans changed views on evolution? Or have creationists changed party? Pew's (half-released) numbers don't add up ...
Dan Kahan Posted on Wednesday, January 1, 2014 at 2:05PM 
Okay. Something does not compute.
Last few days everybody is chortling about a shift in % of Republicans who say they don't believe in evolution.
According to Pew Research Center, a higher percentage of Republicans agreed with the statement that "humans ... have existed in their present form since the beginning of time" in 2013 than in 2009.
One fairly annoying thing is that the information that Pew disclosed about the survey makes it impossible to determine what percentage of Democrats actually believe in "naturalistic" as opposed "theistic" evolution.
Pew's survey item is bifurcated. First, survey participants respond to the question, "Which comes closer to your view? Humans and other living things have [1a] evolved over time [OR] [1b] Humans and other living things have existed in their present form since the beginning of time?" Those who select [1a], are then asked:
And do you think that [2a] Humans and other living things have evolved due to natural processes such as natural selection, or [2b] A supreme being guided the evolution of living things for the purpose of creating humans and other life in the form it exists today?
As a result, only 32%, in both surveys, indicated that the believed in the "naturalistic" position that "Humans and other living things have evolved due to natural processes such as natural selection."
Pew tells us in the most recent survey (in its web page summary and in its Report ) that only 27% of Democrats selected 1a, the "creationist" position that "Humans and other living things have existed in their present form since the beginning of time." It also tells us that 67% of Democrats, "up" from 65% in 2009, "believe in evolution," or in other words that 2/3 of them selected 1b.
Frankly, that's lame.
It's lame, first, because the answer to that question is really interesting and important if one is trying to make sense of how ordinary Americans reconcile their cultural identities, which are indicated by both their political affiliations and their religious practices (among other things), with belief in science.
Second, it's lame because this sort of deliberate selectivity (make no mistake, it was deliberate: Pew made the decision to include the partisan breakdown for only half of the bifurcated evolution-belief item) subsidizes the predictable "ha ha ha!" response on the part of the culturally partisan commentators who will see the survey as a chance to stigmatize Republicans as being distinctively "anti-science."
If in fact, only a minority of Democrats are willing to endorse "naturalistic" evolution -- if a majority of them refuse to assent to a theory of human beings' natural history without God playing a role in guiding it -- then that makes "ha ha ha ha ha!" seem like an unreflective response to a complicated and interesting phenomenon.
But hey-- I'd love to be surprised, too! An unchanging world is dull.
All that aside, the finding that a greater proportion of Republicans now believe in "creationism" -- & not either theistic or naturalistic evolution -- than in 2009 is pretty darn interesting!
But what exactly has changed?
There are two obvious possibilities: [A] Republicans are "switching" from belief in evolution (naturalistic or theistic) to creationism; or [B] creationists are switching their party allegiances from Democrat or Independent to Republican &/or evolutionalists (theistic and naturalistic) are switching from Republican to Democrat or Indepedent.
Either [A] or [B] would be really interesting, but they would reflect very different processes.
Pew doesn't tell us directly (why?! I don't get the attitude of this Report; very un-Pewlike) but we should be able to deduce the answer from what they do report -- the population %s and the partisan breakdowns on "creationism" in 2009 and 2013.
Logically, if the fraction of the overall U.S. population who identifies as creationist stayed same, & more Rs are now identifying creationists, then [B]-- party-shifts by either evolutionists, creationists, or both -- must be correct.
And in that case,the proportion of Ds & Is who are creationists would have to be correspondingly lower.
Alternatively, If the proportion of Rs who are creationists went up but the proportion of Ds & Is who are creationists stayed same, then [A]-- Republicans are changing position -- would be the right answer.
And logically, in that case, the % of the U.S. public overall who now say they are "creationists" would have had to have gone up.
Now that would be truly surprising -- huge news -- because the %s on creationism-vs-evolution haven't changed for decades.
But not surprisingly, Pew reports that "the share of the general public that says that humans have evolved over time is about the same as it was in 2009, when Pew Research last asked the question.":
The same fraction of the U.S. public -- approximately 1/3 -- believes in "naturalistic" evolution today as did then. The 33% who selected the "creationist" response to the bifurcated survey item in 2013 is statistically indistinguishable from the 31% who did in 2009.
So ... if the population frequency of creationism didn't increase, and the proportion of Republican's who now identify as "creationists" did, either creationists are switching to the Republican party or "evolutionists" (theistic or naturalistic) must be switching to Democrat or Independent -- option [B].
But, logically, then, the proportion of "evolutionsists" who are now identifying as either Democrat or as Independent must have risen by an amount corresponding to the increase in "creationists" now identifying as Republican, right?
Nope. Pew says that the division of "opinion among both Democrats and independents has remained about the same":
Ah.
So if the percentage of Democrats and Independents who identify as creationist has stayed constant, and the proportion of Republicans has increased, [A] --Republicans are "switching" their views on evolution-- must be the answer!
But if the proportion of Republicans who are creationists has significantly increased while the division of "opinion among both Democrats and independents has remained about the same," the total proportion of the population that embraces creationism must be significantly higher. . . . Except that Pew says "the share of the general public that says that humans have evolved over time is about the same as it was in 2009, when Pew Research last asked the question."
So, something does not compute.
At a minimum, Pew has some 'splainin to do, if in fact it is trying to edify people rather than feed the apptetite of those who make a living exciting fractious group rivalries among culturally diverse citizens.
Has anyone else noticed this?
Right away when I heard about the Pew poll, I turned to the results to see what the explanation was for the interesting -- truly! -- "shift" in Republican view: Were Republicans changing their positions on creationism or creationists changing their party allegiance?
And right away I ran into this logical inconsistency.
Surely, someone will clear this up, I thought.
But no.
Just the same predictable, boring "ha ha ha ha!" reaction.
Why let something as silly as logic get in the way of an opportunity to pound one's tribal chest & join in a unifying, polarizing group howl?
Happy New Year, Liberal Republic of Science ....
Dan Kahan
@MW offers a logical & plausible mathematical proof of how the proportion of Republicans who are "creationists" could go up while the proportions of Democrats and Independents as well as the proportion of the general population that is "creationist" all stayed (according to Pew) "about the same."
The short of it is that the "about the same" population frequency of creationism actually includes a statistically nonsignificant increase that is concentrated disproportonately in the portion of the sample that identifies as Republican in 2013.
So a nonsignificant -- statistically & practically -- increase in the proportion of the population who "disbelieves" evolution drives the statistically significant & apparently hugely meaningful decrease in the % Republicans who "disbelieve" it.
Does that mean one can one say that insofar as a greater proportion of Republicans but "about the same" proportion of Democrats and Independents are now creationsits means a greater percentage of the overall public is creationist than in 2009-- my position [A]?
No, at least not at a 0.95 level of confidence.
Well, then, can one say instead that since the proportion of the population who disbelieves evolution hasn't increased, "the drop in Republican belief doesn’t appear to be people changing their minds about evolution so much as people who already didn’t believe in evolution becoming Republicans"-- my position [B]? And that therefore the Pew result corroborates that the Republican Party is falling "increasingly out of step with the rest of America’s political views"?
Obviously, one can say that, but that's definitely not supported by the survey results, because in fact the only way (or only way proposed so far-- take your own shot at solving the paradox) for the proportion of Republicans who are classified as "creationist" to go up and the proportion of Democrats and Indepedents as well as the proportion of creationists in the populaiton all to have remained "about the same" is for the population frequency of creationists actually to have risen -- but by a statistically nonsignificant amount.
But it is predictable that people will overlook the mathematical inconsistency -- which will persist until Pew gives an account that resolves it-- and interpret the data as fitting their own preconceptions, which they will then treat as supported by the data.
Well, if in fact the only way to explain how "Republcans' belief in evolution has plummted!" is to observe that the percentage of creationists in the population as a whole went up by an amount that pretty much everyone would have to agree is statistical noise, then maybe we should reconsider whether anything interesting has happened at all since 2009?
But then again, why let logic drown out the noise of embarrassingly, painfully unreflective jeering at the party a majority of whose members believe God had something to do with the natural history of humans? (What's that you say? A majority of people in our Party believe that, too? Shhhhhhhhhhhhhhhh.).
Dan Kahan
Another thing @MW's helpful calculations brought to my attention is that I was assuming, almost certainly incorrectly, that Pew was including "lean [R/D]" as partisans rather than "Independents." I wrote:
Alternatively, If the proportion of Rs who are creationists went up but the proportion of Ds & Is who are creationists stayed the same, then [A]-- Republicans are changing position -- would be the right answer.
And logically, in that case, the % of the U.S. public overall who now say they are "creationists" would have had to gone up--especially insofar as the proportion of the population identifying as Republican has increased a lot since 2009 (when Republicans were still smarting from the odium of Bush II & responsibility for the economic meltdown).
To reduce the risk that of my leading others into error, I've now removed the bolded language (& also an earlier correction/clarification -- "[note: assuming one ncludes "lean Republican" "independents" in the totals, as one really should if one is trying to give an accurate senes of partisan identification]"-- that I now view as not reducing that risk sufficiently).
The mystery/paradox of how the proportion of Republicans who reject evolution could go up without either the proportion of Democrats and Indpendents who reject it going down or the proportion of the overall population who reject it going up doesn't depend on whether "Republican" and "Democrat" are defined as including "leaning" independents.
Dan Kahan
A very informative addendum from Pew here. Includes an accounting of the partisan & population shifts.
Also partisan breakdown for the 2d half of the "evolition" item, which confirms that only a minority -- 37% -- of Democrats, too, "believe" that "Humans and other living things have evolved due to natural processes such as natural selection."
It also makes clear that there wasn't a meaningful downard "shift" in the proportion of Republicans who belive in Darwinian evolution (i.e., "naturalistic" as opposed to "theistic" evolution): 2009, 23%; 2013, 21%....
I likely will post something separately about what sort of inferences one might draw from the Republican "shift"-- but only after looking carefully at reasoning about the full results; the whole point was that it is impossible to interpret the "shift" otherwise, something that obviously never crossed the mind of those whose style is to happily construe ambiguous evidence as confirming what they already believe.
Dan Kahan
Okay-- now w/ the excellent Pew supplement in hand, & my thoughts (provisional as always) about it worked out in my head, I've offered my views on "what can & can't be inferred" from the "shift"
Reader Comments (21)
I was puzzled too. One thing to note however: in 2009, 36% of Dems were 'naturalists' vs. 22% theists, while Republicans were 23% naturalists vs. 26% theists, so there does seem to be a pattern there (http://www.people-press.org/2009/07/09/section-5-evolution-climate-change-and-other-issues/).
The real mystery to me is that, in 2009, fifteen percent of the total ("all adults") answered "I don't know" (I think?), while in 2013, only seven (or five?) percent of the total did. I'm not sure what that means.
@Mike:
If you can perform the necessary simultaneous equations -- or just brute force it out w/ a computer simulation -- necessary to make the numbers all add up, please do!
It shouldn't be necessary, though.
And I find it very discouraging that Pew wouldn't report the results on partisan breakdown on the *entire* evolution question. It's hard not to suspect that they were worried not to complicate the great "story line" they had created for the poll by singling out the Republican shift for attention. That's so unlike Pew, though ... Weird. & sad.
I have reviewed many PEW reports, & my employer has participated with them in some. I have no faith in any of them. The researchers they put into the field are young, naive, & not only biased but don't know much about interviewing or data collection. They package their results to look slick, but based on experience with them I've given up believing anything they publish. A real shame.
@Kathy--
eek! That's disconcerting.
I haven't had that kitchen's eye view of things.
The reason I really like Pew as a researcher & curious person is that the design of their surveys usually reflects a tremendous amount of sophistication & craft, both psychometric & practical.
For the most part, they don't do issue du jour polls that pretend that it is meaningful to figure out what % of populace believes what about an issue that well over 50% never heard of before the pollster asked them about it.
Instead they do thoughtful *studies*, ones that attempt to extract from public opinon the sorts of evidence that supports valid and edifying inferences, and with *sets* of questions that enable a reader to be confident that the survey is measuring what the researcher says that it is measuring.
My favorite example is the 2009 public attitudes toward science report. It's a masterpiece.
This latest survey is not-- and sort of does a disservice to the organization's commitment to helping to generate knowledge about the complicated & interesting signficance of religion in American culture.
"Second, it's lame because this sort of deliberate selectivity (make no mistake, it was deliberate: Pew made the decision to include the partisan breakdown for only half of the bifurcated evolution-belief item) subsidizes the predictable "ha ha ha!" response on the part of the culturally partisan commentators who will see the survey as a chance to stigmatize Republicans as being distinctively "anti-science.""
Hmm.
" if in fact it is trying to edify people rather than feed the apptetite of those who make a living exciting fractious group rivalries among culturally diverse citizens."
Hmmm.
"It's hard not to suspect that they were worried not to complicate the great "story line" they had created for the poll by singling out the Republican shift for attention. "
Hmmm.
I dunno, Dan. That all sounds a bit conspiratorial. Pew comprises many people. No doubt, a relatively small number are making decisions that explain operational choices, but even still I'd imagine a lot of people are involved. Do you really suppose that there is a deliberative and coordinated decision-making process motivated by the goals such as those you suggest?
In the very least, I'd think that you might consider the role of motivated reasoning to explain the decisions - but even there, why not just - plain ol' confirmation bias or incompetence or a lack of comprehensiveness?
As for the Pew poll, my guess is that there are methodological self-report biases - such as social desirability bias - problems with the data. People might answer questions in inconsistent ways depending on the context of the questions being asked.
But I do still think that there may be some meaning to the data - even if they aren't particularly valid (in the sense of indicating what they are said to be indicating) - because of trends over time. The data, I suggest, show that the question of evolution has become more politicized, and thus views on evolution have become more polarized in correlation with political identification.
I guess I should re-state that. Basically, I think that views on evolution have become more politicized (not polarized). Sure, they were politicized before, but the politicization has become stronger as the "culture wars" have stepped up in intensity along with the "War on Christmas," the enlarged political influence of fundamentalism as reflected by Michelle Bachmann and other highly religious members of the Tea Party, the meme of Republicans being "anti-science," the noted drop in "trust in science" among Republicans (although not with Democrats and Independents), etc.
Agree on the first point and agree that Pew has presentation problems. But doesn't the math work out if we note (1) there are fewer self-identified Republicans than Independents and Democrats (at least per 2009/2012 surveys); (2) the percent of Democrats who believe in creationism did go down by three points (and Independents up by one); and (3) the proportion of the population as a whole that believes in creationism went up by two percentage points?
Check my numbers. Say we have a survey population of 1000 people. And assume in both 2009 and 2013, 24% identified as Republican, 32% identified as Democrat, and 38% identified as Independent (as in the 2012 poll), so we're considering option [A]. Using the data from the party affiliation & belief graphic, I think the survey shakes out as follows:
Of 1000 People Surveyed:
240 Republicans
320 Democrats
380 Independents
= 940 Answerers, 60 Doodlers
In 2009
Believe in Evolution:
130 Republicans (.54*240)
205 Democrats (.64*320)
255 Independents (.67*380)
(130+205+255)/940 = 63% Believe Evolution
Believe Creationism:
94 Republicans (.39*240)
96 Democrats (.30*320)
103 Independents (.27*380)
(94+96+103)/940 = 31% Believe Creationism
In 2013
Believe in Evolution:
103 Republicans (.43*240)
214 Democrats (.67*320)
247 Independents (.65*380)
(103+214+247)/940 = 60% Believe Evolution
Believe in Creationism:
115 Republicans (.48*240)
86 Democrats (.27*320)
107 Independents (.28*380)
(115+86+107)/940 = 33% Believe Creationism
That aligns with the Pew survey on essentially all fronts, right? I certainly agree that portraying everything as staying the same except for Republican belief is problematic and misleading. And it's weird when a few numbers that are within the margin of error get overshadowed by another number that isn't. But do you not think the math works out? (Let me know if my arithmetic is wrong or if I'm missing the point.)
@MW:
Thanks!
There must be some explanation.
And I agree yours helps to put the "shift" in perspective." If Pew comes out & says, "oh, this is the explanation"--great: it will help people to see that a : a nonsignificant ripple in the data is being inflated into a whirlpool of "see! More evidence of what we already knew about ... [Republicans, increasing polarization, tightening of party affiliation & positions on social issue etc]
Also, note that you are using 24% as Republican pct of population b/c you are excluding "lean Republican" Independents. Is that what Pew is doing? If so, they shouldn't be. It is well established (Petrocik, J.R. Measuring Party Support: Leaners Are Not Independents. Electoral Studies 28, 562-572 (2009)) that "lean Independents" are partisans. & Pew knows this!
What is 2009 & 2013 comparison based on? Only "partisans minus lean-partisan" computation of party id or "partisan plus lean partisan" computation? If the former, Pew should tell us that -- & why they did it that way, & what the "partisan plus leaner" proportions are. If latter, then there are considerably more "missing creationists" than you have found through your proof!
And they should just *tell* us enough about the data & how they decided to parse it so that we can answer the question whether the "shift" is a result of "Republicans changing positions" or "Creationists changing parties" -- or whether in fact that question can't be answered b/c the "shift" is too vapor thin to admit of analysis.
There's a wonderful circularity in which weakness in evidence that we cite to justify our preconceptions gets excused by the fit between it & our preconceptions
The reason that Pew didn't say whether the change was compositional or not, most likely, is that they don't know. The questionnaire didn't contain any question along the lines of "have you switched parties in the last four years", so there's no way of determining that from the data. (And, since they probably weren't expecting this swing, they had no way of knowing that such a question would be useful ahead of time).
I checked the theistic evolution question partisan breakdown on the 2009 report, and it's interesting how much difference there was; Republicans were 23/26/39 on natural/guided/no evolution, whereas Dems were 36/22/30 and independents were 38/20/27.
@dmk38:
Again, I agree that this should all be much more transparent.
I don't think including "leaners" is going to change the option A plausibility, though. Doing this more simply, if we take a 9 percentage point shift in creationist belief among the 40% of the population that leans Republican, and a -3 percentage point shift in creationist belief amond the 48% that leans Democrat, we have:
(.09*.40) - (.03*.48) = .022, or a +2 percentage point shift in creationist belief overall. That look right to you?
But yes, it would be good to know what they actually did! (I suspect they didn't include leaners, since they did include Independents.)
@RFlaum:
One doesn't have to ask that question. It would be a bad question to ask, in fact.
Shifts in population frequency of party id & beliefs in evolution from the two periods can be compared and a logical inference formed about whether the "shift in proportion of Rs who believe evolution" is being driven by change in evolution beliefs conditional on being R/D/I or a change in party identification conditional on believing/not believing evolution. That one gets probabilities not adding up to 1 when one tries to perform this exercise is mysterious.
It's possible the shift was too ephemeral to make answring that question possible, partuicularly in light of noise created by shift in party id between 2009 & 2013. If so, that would help to make sense of the "shift" -- it would suggest that the risk of confusing noise as signal is too high to give it much significance at all.
@MW:
I agree that they "probably" didn't include "leans" in party id. But their including "Independent" as a category doesn't itself help to answer that question.
The survey item involves a "push" for those who identify as "Independent" on a 3-point measure ("R," "D" or "I"). So one can define "Independent" as those who say "I" on the three-point measure or those who refuse to "lean" after the push.
The reason to treat "leaners" as partisans rather than as "Independents" is that they are not meningfully less partisan in responses to various policy or attitudinal items than those who identify as "R" & "D" on a 3-point measure. (Pew knows this; they probably were among those who helped people to learn that this is the case!)
But one can define "Independent" however one wants. One should just say how one did. And when making claims about how "partisans" feel about an issue be sure to report data on "leaners" if you collected them, so that people can decide for themselves what the data signify.
Opinion polling is not supposed to be a hide-&-seek game. It is supposed to help people understand the nature of public opinion.
But many thanks for the additonal computations!!
My thoughts re: including Independents as a separate category were:
(a) In other graphs that include leaners (see my earlier link and the graphs you posted above) Pew tends not to include non-leaning Independents; and
(b) If Pew actually asks whether people identify as Independent, some people say "yes," and then those people are not included in the Independent category in a graph without any indication that they weren't or why, that would be somewhat actively misleading.
Agreed on all other stuff.
@MW:
they can/should simply describe who they are classifying & how. Classifying w/o saying how is what's misleading & I'm not proposing that.
INdeed, I'm proposing that they be much clearer -- & disclose rather than hold close the data on how all the groups answered *all* parts of the question.
Also proposing that everyone else read -- not you, in particular; in fact, not you, since you obviously aren't in this category -- read critically rather than just pile on & say "yup-- just as we thought!"
Giving evidence a "free pass" when it fits your predispositions is a great way to get & stay stupid!
As a progressive creationist (life is created by God over a long period of time as opposed to a few 1000 years ago), I would have "agreed" with the first statement, that living things have evolved (which simply means "changed") over time. I would also have agreed with the 2nd, that such processes take place by natural selection. I'm talking about how species develop ("micro-evolution"), as opposed to life spontaneously coming from non-life, or beings of lower orders of complexity & intelligence randomly giving rise to beings of higher orders of complexity/intelligence ("macro-evolution"). The 3rd statement I believe also confuses micro- and macro-evolution, for I would agree in part that God providentially supervises all things, including the (minor) changes in living beings over time, but the part that says "for the purpose of creating humans" etc. would get an overall "disagree" from me. So far, then, my answers would put me in the "agrees with evolution" camp, which I am definitely not. I'm a little surprised that with such misleading questions, so many respondents fell into the "don't believe in evolution" camp at all (I'd guess the respondents basically said, "well, I know what they mean" instead of actually responding to the questions). The questions are very poorly thought out (unless this was an intentional purpose). The proof that they were poorly thought out is the last one: "Humans and other living beings have existed in their present form since the beginning of time." "Since the beginning of time"? That rules out progressive creationism in which most life was created long after "the beginning of time" (whatever that means) so the 31% of people that agreed with this statement is only some fraction of the total number who believe in creation (unless progressive creationists who thought about this poorly worded question decided they'd rather answer in support of creation as opposed to evolution). Overall, I think this is garbage in, garbage out.
I have the answer; it is not pretty. For generations, American students have scored almost dead last globally in math and science. Ergo our adult population is dominated by technically ignorant people. This is amply documented. Further, most Americans do not know the three branches of government or the names of their Senators. Thus to sample, in depth, on the topic, a statistical study of judgments on evolution vs political affiliation is bound to yield Ha Ha results.
But the article points to a worse situation, considering our population. Virtually all reporting today omits half the vital facts, the facts pointing to a conclusion opposed by the editor. This reporting bias also is well documented. One has to dig very hard, as this author has, to identify inherent falsehoods in reports. We read lies by omission. It is common in every scientific conflict, from climate change to nuclear power.
And the worst point. Our modern society is plagued with educated people who value their corner office more than the truth. We drown in polysyllabic lies;there is a high percentage of published fraudulent science papers. This drive our federal policies. In our busy lives, there is no time to sort things out, even if the gnawing feeling, "this does not calculate."resides in technically educated adults.
There is no remedy to a corrupt culture.
On the topic, the bible vs evolution. There is no conflict. I was taught by a priest who spoke 67 languages and was the biblical consultant to Vatican II. The bible is not a scientific treatise; it is poetry, history, sociology and at its base, theology. Man must not tell God what to do. We must learn from Him. He gave two key points governing science, journalism and politics: Thou shalt not lie, or steal.
Don't even bother to tell me how many Democrats or Republicans "believe" in evolution.
Tell me what percentage can coherently explain evolution by natural selection.
I bet it's well under 5 percent.
Response to the Pew Research report on views about evolution (http://www.pewforum.org/2013/12/30/publics-views-on-human-evolution/) in this blog and in a number of other outlets raised some interesting questions about the patterns in the data, particularly regarding the finding that far fewer Republicans believe in evolution than did so four years ago. We put together a brief review of possible explanations along with the relevant data, which is now available at the Pew Research Center's FactTank blog: http://www.pewresearch.org/fact-tank/2014/01/03/republican-views-on-evolution-tracking-how-its-changed/.
As a nonpartisan, nonadvocacy organization, we always strive to be comprehensive, transparent and completely unbiased in our work and our reporting. We cannot always anticipate every potentially interesting line of inquiry, but we are always happy to provide additional information and crosstabs to those who are interested. Typically, there's an email icon with every report on our website that can put you directly in touch with the relevant researchers.
– Cary Funk, Senior Researcher, Pew Research Center
Interesting to see ... thank you it's well done :)
@Cary-- many thanks, not just on my behalf, of course, but on behalf of the zillions of other people who find the results of the survey important, interesting, & worthy of thought & discussion.
I do have some questions about the new data, and about their relevance for conjectures about how the overall shift in the proportion of evolution "nonbelievers" contributed to the shift in the Republican proportions, but I'll raise them in an email
It's life Jim, but not as we know it Dustin