"It's better to light a candle than curse the darkness"

“Let’s put on a Study!”

November 3rd, 2008

Once again, there is a hue and cry to “do a study” about vaccines and autism, this time pushed forward by commentors on David Gorski’s recent post on the “Science-Based Medicine” ‘blog. Whenever I hear people who have never tried to do any study - let alone a study of human children - say “We need to do a study!”, I can’t help thinking of Mickey Rooney and Judy Garland and their “Let’s put on a show!” musical films - ergo the title of this article. 

As I expected, the commentors put forth many of the same arguments that haven’t worked before but this time they are doing their math, something I appreciate. So, in appreciation, I have prepared the following:

The Reason:

The interest in “doing a study” seems to stem from the idea that the only way to find if vaccines are connected with autism is to study unvaccinated children and compare their autism prevalence with vaccinated children. This is not true and is - in fact - not even the most effective way to study the postulated connection, as I will show later.

Much of the blame for this wrong-headed fixation on “Just study the unvaccinated!” can be laid at the feet of Dan Olmsted, who claimed that the Amish don’t vaccinate and don’t have autism. Both claims - it turns out - are untrue. The Amish do vaccinate and they do have autism. Apparently, Mr. Olmsted didn’t look in the right places for his information about the Amish. And if you don’t look, you never find.

The same argument - if you don’t look, you don’t find - has been used to argue for studying the “connection” between autism and vaccines. However, researchers have looked. They looked at thimerosal and autism - and found no correlation. They looked at the MMR vaccine and autism - and failed to find a correlation. Now, with their backs against the wall, the vaccines-cause-autism believers have shifted from distinct vaccines or vaccine components to vague “toxins” or the even more vague “too many, too soon”.

Despite an absolute lack of data supporting either claim (’toxins” or “too many, too soon”), the proponents have argued that “we need to do a study” of vaccines and autism. “We” (i.e. the taxpayers) may indeed end up “putting on a study” in response to the political pressure these groups may bring to bear. I have no illusions that any research will ever be able to convince these people that they are wrong. After all, they haven’t been swayed by any research done so far.

However, since we’re all going to end up paying for this study, we should ask that it be done in the most effective way possible, rather than simply following the uninformed dictates of the vaccines-cause-autism believers.

The Amish:

Let me dispose of one line of “reasoning” that often comes up when these “Let’s put on a study” promoters gather. It is often suggested that we “take advantage” of existing groups of people who do not vaccinate their children. Mr. Olmsted proposed the Amish, although it later turned out that the Amish do vaccinate their children, although not at the level seen in the population at large. Another proposed study group was a health care group in Chicago that practiced non-vaccination and also - we are told - had a low to zero autism rate.

The problem with selecting “special” populations to study is that they are not comparable to the general public. The Amish - probably the most dramatic example - have a much higher prevalence of several genetic disorders as a result of generations of inbreeding. They are not comparable to the US population for this reason and any results obtained from studying them would be irrelevant. I suspect the main reason for wanting to study the Amish was that Mr. Olmsted “found” that they don’t have any autism - which was, as I’ve said above, also incorrect.

Even if we were to study a group like that in Chicago, the question would arise, “What other differences (besides vaccination) are there?” Do they only eat organic foods? What else might account for any differences between them and the general population. And if no difference in autism prevalence were discovered, would that be enough to satisfy the folks crying to “put on a study” or would they simply shift their sights to another group?

No, the only way to correctly do a study of vaccines and autism is to compare children who are matched for age, sex, geographic location, rural/urban/suburban setting, socio-economic group and race. That way, the only differences (we hope) would be vaccination status and autism prevalence.

The Numbers:

If we’re to do a study (as opposed to “putting on” a study), we need to have an idea of the numbers of subjects we’ll need to test in order to come up with a statistically meaningful result.

In 2004, Smith et al published “Children who have received no vaccines: Who are they and where do they live?”. This was an extensive study of vaccination practices in the US, carried out from 1995 to 2001 on over 23,000 children selected at random to be representative of the larger population. Their results were interesting and bear directly on the  desire to “put on a study” looking at the connection (or lack thereof) between vaccines and autism.

What they found was that 62.8% of their study population was “fully vaccinated” (i.e. had received all the vaccines on the then-current vaccination schedule - 15 total in 2001). An additional 36.9% were “undervaccinated” - although 58.5% of them (20.4% of the total sample) were missing only one vaccine. Only 0.3% of the sample - 111 children ages 19 - 35 months over five years - had not received any vaccination.

I’d like to emphasize a point. The Smith et al study only looked at children 19 - 35 months of age, which is well before school age, at which time undoubtedly a few more tenths of a percent of the unvaccinated were whittled away. However, this age includes the ages where children have been reported to have “regressed into autism” following vaccines.

So, any study of unvaccinated vs vaccinated children has to deal with the fact that the “pool” of unvaccinated children is about 0.3% (or less) of the total population. Even if we could extend the Smith et al findings to the entire 0 - 17 year age range (”children”, by most definitions), this works out to only about 223,000 completely unvaccinated children. Given that at least some of the children who were unvaccinated before 35 months will have gotten at least one vaccination by their teen years, this number is sure to be much smaller.

For the purposes of this demonstration, let’s limit the study group to younger children, both to eliminate the uncertainty about the number of unvaccinated children and to keep the diagnostic criteria somewhat comparable. After all, the criteria used to diagnose the 17 year-old autistic children were very different from those used today.

How about we settle on the 3 - 6 year age range? This puts them past the scheduled time for the majority of childhood vaccinations and yet cuts off before the “school vaccination mandate” years. According to the US census bureau, this age group (as of August 2008) contains 16,550,753 children. Of that number 49,652 (or less) might be completely unvaccinated (and over ten million “completely vaccinated”).

What sort of sample numbers do we need to “prove” the vaccine-austism question? Well, let’s make a few assumptions. The first assumption is that we will want to have a sample size large enough to be sure that we have less than a 5% chance of mistakenly saying there is a correlation if there isn’t (alpha error)  - and we’ll want a less than 5% chance of saying there isn’t a correlation if there is one (beta error) in order to satisfy those who want to “put on a study” (normally, we’d accept a 20 - 50% chance of mistakenly saying that there is not a difference when there is one).

Secondly, we’ll have to stipulate what sort of difference we expect to find between the two groups. If the “fully vaccinated” group had 50% more autism than the “unvaccinated” group, would that be enough of a reason to change vaccination policies? Would a 10% difference be enough? More to the point, how small a difference will be small enough for the vaccine-cause-autism proponents to say “I was wrong, vaccines don’t cause autism.”?

That last point may be the one that actually torpedoes the study. Because if there is no number small enough to convince those who want the study that they are wrong, most of the reason for doing the study vanishes.

Let’s say - for the sake of argument - that we decide that a 10% difference in autism prevalence is enough to convince the skeptics that vaccines might cause autism and that a less than 10% difference will convince the believers that vaccines don’t cause autism. [I know, the latter assumption is pure fantasy.]

Well, plugging those numbers in - along with the current 1 in 150 autism prevalence - we find that we need over 360,000 children in each group to detect a 10% difference (you can try it yourself here). Unfortunately, that is more than the total number of unvaccinated children in the US, so that’s not going to happen.

What can we get with our “sample” of 49,652 unvaccinated children? If we manage to include each and every unvaccinated child in the US in the study, we could detect a 26% or more difference in autism prevalence.

Of course, it’s not even remotely practical to expect to get 100% of the unvaccinated children in the country into a study. How about a more practical number - say, 10% of them? That would allow us to detect a 70% or greater difference - about a three-fold difference in autism prevalence between the fully vaccinated and unvaccinated groups.

Now, if a difference of this magnitude or greater were found, that would pretty effectively silence those who say that the data haven’t shown a connection between vaccines and autism. I would certainly change my opinion (from “not shown” to “strong indication”).

But what if the results go the other way? What if the results are “negative”, meaning that the difference is less than 70%? Would that be enough to get the vaccines-cause-autism crowd to stop protesting and stop sending death threats? I doubt it.

Of course, there are those who believe that the results will be unambiguously in favor of their belief that vaccines-cause-autism. They want the study to go forward because they are confident that they will be vindicated. They are also the people who will rant about “bias”, “flaws” and “corruption” if the data don’t go “their way”.

Another other thing to consider is that a study with almost ten thousand children enrolled would be exorbitantly expensive, especially as they would all have to be tested carefully for autism. Even using a very conservative figure of $150 for an evaluation, that’s over $1.5 million just for the autism testing.

Don’t forget that research money, time and facilities are not infinite. Resources spent chasing the vaccines-cause-autism “connection” will have to come from other research areas, and the most fair way to do that would be to take them from other autism research. Thus, much time and money is spent to accomplish little…or nothing.

And is it likely  that you could get the parents of 10% of the unvaccinated children in the country to enroll them in a study? Given that many of these children are unvaccinated because their parents harbor deep suspcions about doctors and “the government”, it seems unlikely.

Finally, let’s “run the numbers” on a more practical study - one where we are able to enroll 500 unvaccinated children and 5000 fully vaccinated controls matched for age, sex, socioeconomic group, geographic location, urban vs rural vs suburban setting and race.

This study - which would still be very difficult and expensive to do - would only be able to detect a more than 15-fold difference in autism prevalence between the two groups. It could detect as little as a 7-fold difference, but only if we were willing to accept a beta error (chance of erroneously saying there is no difference when there is a difference) of over 50%.

I doubt this would “satisfy” the vaccines-cause-autism believers if the results were negative.

So, before someone tells me - again - how easy it would be to study autism in unvaccinated children, first go out and try to find a few thousand completely unvaccinated children and then tell me how easy it is.

Ethical Considerations:

Another suggestion made was to have a study where children are placed into “no vaccination” and “vaccination” groups. This, of course, would be rejected out of hand by any Institutional Review Board because the risks of not vaccinating are well known and quite serious. On the other hand, the connection between vaccines and autism is tenuous at best. It would be unethical to expose adults to a known serious risk in order to test a weakly-supported (again, at best!) possible risk. In children, it would be unthinkable.

Another “modest proposal” was to vaccinate one group per the suggested schedule and vaccinate another group with four times the amount of vaccine. Well, at least this commentor understands the concept of “dose-response”. If - as some people argue - the current amount of vaccine is causing some amount of autism, then it would be reasonable to expect that more vaccine would lead to more autism.

Of course, the idea of doing something with the intent of causing harm to children is unlikely to pass the ethical hurdles of a real IRB. However, even if the hypothesis is incorrect - even if the researchers argue that harm is extremely unlikely, they would still be exposing children to four times the dose used during the safety trials of the vaccines, which would constitute an unreasonable risk, especially to test a hypothesis that is already weak.

Additionally, if the results are “negative”, the vaccines-cause-autism proponents could (and would) argue that their hypothesis is correct but that the threshold for causing autism is lower than the “standard” vaccine regimen (you know, the “too many, too soon” argument). Thus, the argument for doing the study (absent the obvious ethical considerations) has largely disappeared.

An Alternative:

Part of the difficulty in doing a study of unvaccinated children is tracking them down. There is no national or even statewide database that can “spit out” the names of unvaccinated children; you’d have to track them down one by one, a process that can take years to complete, given the numbers needed.

Additionally, the proposed study design wastes the effort used to collect the subjects based on a single variable - vaccination.

A much better study design would be to look at two age, sex, etc. matched sets of autistic and non-autistic children. Autistic children are more “visible” in the medical and educational systems than unvaccinated children (who are often “under the radar”). It would be easier (although not easy) to recruit autistic children and non-autistic control subjects than it would be to track down thousands of unvaccinated children.

Another advantage is that the researchers could collect data on a wide variety of exposures, genetic issues and other variables that could later be “mined” to find other promising avenues of research. This in contrast to the “single issue” study looking at unvaccinated children, which can only compare issues related to vaccination.

Even with those advantages, the study will be “swimming upstream” a bit.

A study of autistic children in the 3 - 6 year age range would need over 683,000 children in each group to detect a 10% difference in vaccination level. It would need nearly 22,000 in each group to detect a 50% difference. With a predicted number of 110,000 autistic children in that age range, that is a sizeable fraction of all autistic children.

A more manageable study - one with 10,000 children in each arm (which is still a HUGE study!) - would only be able to tell the difference between the national average of 0.3% unvaccinated in the non-autistic group and 0.1% unvaccinated in the autistic group (at the specified levels of confidence). If the difference is smaller than that, the results would be considered negative (i.e. that there is no effect of vaccination). For reference, a study with 1,000 children in each arm would show statistical significance (at our specified level of confidence) only when the autistic group was below 0.01% unvaccinated or above 1.7% unvaccinated.

Of course, we wouldn’t have to just look at unvaccinated vs fully vaccinated with this study, which is a large part of its superiority. We could look at a dose response of vaccination - to see if it really is “too many” - as well as the age at youngest vaccination - to see if it really is “too soon”. In fact, a few studies have already looked at those issues and found that there is no difference between the autistic and non-autistic groups. I suspect this is the reason the folks pushing to “put on a study” want to look at vaccinated vs unvaccinated - they hope that the numbers will be different (or, at least, not as definitive) the other way round.

 Conclusions:

The current push to “put on a study” comparing the autism prevalence in vaccinated and unvaccinated children is likely to come to political fruition long before it generates any real scientific interest. Anyone who has actually done a human study will recognize that the proposed study format - find a bunch of unvaccinated kids and compare their autism prevalence to the general population - has serious problems.

If our political leaders eventually find it expedient to “put on a study” in a vain attempt to quiet the protests of the vaccines-cause-autism fringe, then we need to be prepared to insist that it be a proper study, not a rigged “study” pre-ordained to find the “correct” results.

 

Prometheus

Filed under: Autism Policy, Autism Science, Critical Thinking

33 Responses to ““Let’s put on a Study!””

  1. Mu Says:

    Prometheus,

    you latest proposed study would suffer greatly from the bias you’d see for parents with autistic kids who are unvaccinated. This would mean that the most common “it’s not my fault” excuse is gone, and the parents would have to confess to that to get the kid into the study in the first place. I don’t see how you can get the data (autistic, vaccination status) from a truly representative sample, since unvaccinated kids are such a fringe group to begin with, and most likely have parents at the “science opposed” end of the spectrum. Those parents have nothing to gain from participating.

  2. Club 166 Says:

    Great post, Prometheus.

    I would just like to add that when someone suggests we go to another country (that doesn’t vaccinate) to do a prospective study, that current ethical opinion would not allow that, either.

    Joe

  3. Sullivan Says:

    Consider the population of Chicago–2,842,518 (in 2005, according to

    http://www.infoplease.com/ipa/A0763098.html

    Consider 0.3% of the population unvaccinated. That would lead to 8,527 unvaccinated people.

    Either HomeFirst is influencing the population (significantly) to not-vaccinate, or they idea that they have 20,000+ unvaccinated people in their ranks is, well, a stretch.

    Yes, HomeFirst likely brings in a larger area than just the city of Chicago (Cook county has about 5.2M residents http://quickfacts.census.gov/qfd/states/17/17031.html).

    Then again, HomeFirst are not likely to serve 100% of the un-vaccinated, either.

  4. Sullivan Says:

    On the Amish question

    1) The Amish are not just genetically distinct from the main population, the communities are fairly distinct from each other. They are “islands”.

    2) If one were to study the Amish of, say, Lancaster County, PA, one would need to include all of them in the study (similar to the notes you make above). We can not force them to participate. They are people and U.S. citizens. Where do people get the idea that we can just romp through their communities and test them?

    Lastly, in the Omnibus testimony of the petitioner’s epidemiologist–it was noted that present epidemiology would indicate that 96% or more of autism is not caused by thimerosal. If they want to test this vaccine component, they would need much larger numbers than you say are available.

  5. passionlessDrone Says:

    Hi Prometheus -

    They looked at thimerosal and autism - and found no correlation. They looked at the MMR vaccine and autism - and failed to find a correlation.

    Technically true statements; but completely useless if we want to understand anything other than the effect of thimerosal or the MMR.

    Studying these things tells us precious little about the impact of repeated and earlier immune stimulations; it only tells us about the impact of a specific preservative, or a specific immune stimulation. These studies do not constitute a rigorous application of the scientific method.
    This isn’t a back against the wall situation; but rather, a statement of rather plain fact.

    Despite an absolute lack of data supporting either claim (’toxins” or “too many, too soon”), the proponents have argued that “we need to do a study” of vaccines and autism.

    Well folks, the fix is in. I’m curious, what, precisely, do you believe would constitute data to support such a claim, barring exactly the study being proposed? I asked David Gorski this very same question, which he declined to respond to. Perhaps you will. (?)

    What would you consider sufficient data to merit a study of vaccinated versus unvaccinated children? Is there any?

    If some people will be absolutely unconvinced of the results, the inverse, being unconvinced that such a study ever be merited is surely just as hypocritical; and indeed, psuedoscientific.

    Another “modest proposal” was to vaccinate one group per the suggested schedule and vaccinate another group with four times the amount of vaccine. Well, at least this commentor understands the concept of “dose-response”. If - as some people argue - the current amount of vaccine is causing some amount of autism, then it would be reasonable to expect that more vaccine would lead to more autism.

    LOL! I didn’t submit this proposal as a serious consideration, but merely, to illustrate the increasingly obvious; we are being sold a bill of goods wherein it is impossible to evaluate the risks of an innoculation program that involves every single child in the United States. Completely impossible. Now, if we want to theoretically prove that a child can get 10,000 vaccines at once, and publish it under a paper headed by ‘Addressing parents concerns’, well that is quality science. But if we want to reduce that number by a factor of thousands and try on real children, it is too dangerous. That is a powerful disconnect. And what of our potential ability to creep our rates of immunity upwards half a percent? Is this not a desirable goal?

    The current schedule is like the middle bowl of porridge, just right! Our evidence for this is not actual analysis, but rather, by the relative weakness of other arguments. Of course, when Gardisil, or shingles, or the common cold vaccine comes along, we can be absolutely positive that we won’t have any long term impact by piling on more stimulated insults. How? By the stated weakness of the arguments that there could be a relationship. For all of the hemming and hawing about how ‘anti vaxxers’ are ‘anti science’, this is a piss poor application of the scientific method.

    I also asked David Gorski if there would ever, ever, be a number of shots on a vaccine schedule that might merit some evaluation; another question he felt unnecessary to respond to. If our vaccine schedule quadroupled, would there be no need to see if we might be having unforseen impacts on our children? What if it increased by ten times? This is a genuine question that I’ve had ducked on me dozens of times. Will you duck it as well?

    Anyone who has actually done a human study will recognize that the proposed study format - find a bunch of unvaccinated kids and compare their autism prevalence to the general population - has serious problems.

    Hm, I found someone who thinks they have an idea. Here is a quick summarization of their proposal. What do you think of it?

    This is a study that could be done rather quickly and with a minimum of expense. It would eliminate many of the sources of bias and would fairly easily generate balanced study populations that would be a good match to most of the general population.

    [a] Contact a large HMO with actual facilities (e.g. Humana or Kaiser) and arrange to get access to their patient medical records. This is routinely done, although the HMO will want assurances that patient confidentiality will be maintained.

    [b] Obtain a list of patients with autism diagnosis in the proper age range (I would suggest 6 - 12 years).

    [c] Select one thousand of these patients at random. This would allow you to detect a difference if the prevalence of unvaccinated children is less than 1/3 that in the general population (alpha error level 5%, beta error level 5%). If the difference is less than that, you’ll need to select more subjects.

    [d] Confirm the diagnosis by having a child psychiatrist or psychologist review the records.

    [e] For each of the remaining children, select a non-autistic control child from the HMO database that is of the same age, sex, geographical region, etc.

    [f] Determine how many of the children in each group have received all, none or some of their vaccinations (keep track of which vaccines, when, etc.). If the GR “survey” was right (a very big “if”) about the number of children unvaccinated, each group should have around 30 unvaccinated children, unless there is a correlation between vaccination and autism.

    [g] If the autism and non-autism groups have statistically significant differences in their vaccination rates, then a correlation can be claimed. If the study shows no correlation, then the relative risk is less than 3. You’d have to have twice as many subjects to bring the minimal relative risk to below 2.

    Using HMO patients eliminates any issues of affordability (which is minimal) or access to health care. Although the population of people who have HMO coverage is not necessarily the same as the overal US population, using the case control design ensures that the two groups are as similar as possible.

    Oh wait. That was YOUR proposal written on July 18, 2007, on the photoninthedarkness.blogspot url. Now that is what I’d call changing the goalposts! LMAO!!!!

    - pD

  6. Phil Schwarz Says:

    Don’tcha know, “Zack and Miri Make A Study” is going to be released real soon into those markets which flinched at advertising that, uh, other movie…

  7. Prometheus Says:

    pD,

    For some reason, any time I see someone use “LOL” as a response to something, I can’t help thinking that they are a pre-teen. Or that they are expressing nervous laughter in the face of an argument they can’t understand or can’t refute.

    Maybe it’s just me.

    My proposed study of 19 July 2007 would have worked if the percentage of unvaccinated children had been as high as those found in the GR telephone poll (3%) - a point I explicitly made in the post, if you read it. Since a better study showed the rate to be only 0.3% (one-tenth that found in the GR poll - another indicator of the validity of their “data”), I have subsequently used that figure.

    You seem to find it amusing that scientists revise their hypotheses when presented with new data (as suggested by your pre-adolescent “LMAO” response), which explains why you have remained so imperturbable and immutable in the face of data contrary to your position.

    Prometheus

  8. The Perky Skeptic Says:

    This whole post is awesome as usual, but:

    “More to the point, how small a difference will be small enough for the vaccine-cause-autism proponents to say “I was wrong, vaccines don’t cause autism.”?…. Because if there is no number small enough to convince those who want the study that they are wrong, most of the reason for doing the study vanishes.”

    THIS, for me, is the money-quote right here– the point at which ANY study can teeter on the brink of either worthy science or crank-sputum! This is the exact question everyone should ask themselves when weighing evidence in making decisions!

    May I quote you all over the place for this one? :)

  9. passionlessDrone Says:

    Hi Prometheus -

    For some reason, any time I see someone use “LOL” as a response to something, I can’t help thinking that they are a pre-teen. Or that they are expressing nervous laughter in the face of an argument they can’t understand or can’t refute.

    Well, I am, literally, laughing out loud again, right now. But it isn’t because I am a pre teen. Instead, it is because despite the fact that you have obviously poured a lot of effort, and indeed, thoughtful analysis, into your original post; yet you have decided to dedicate the majority of your response to me musing on my age and fancifully incorrect assumptions as to my position. It is funny because it shows that despite your claim that you represent ’science’ you are completely unwilling to address legitmate questions I posed to you. None of these things make me nervous.

    Speaking of arguments that cannot be refuted; I can’t help but notice you ducked out on the same questions I put to David Gorski. Big surprize there.

    Listen, I’ll try to only use adult language to keep from throwing you for a loop. I’m curious to see if you can put the same level of detailed analysis into a question set with someone that thinks back, as opposed to a punching dummy of your own design.

    What can we learn by analyzing thimerosal other than the impact of thimerosal? What can we learn by analyzing MMR rates other than the effect of the MMR? Is there a chance that there might be other things in the vaccine schedule other than these two components that merit investigation? Do you believe there are any potential pitfalls towards espousing the notion that studying these two components provides a sufficient model for understanding the impact of the existing schedule as a whole?

    What would you consider sufficient data to merit a study of vaccinated versus unvaccinated children? Is there any?

    If our vaccine schedule quadroupled, would there be no need to see if we might be having unforseen impacts on our children? What if it increased by ten times?

    In regards to a vaccinated / heavily vaccinated study, what of our potential ability to creep our rates of immunity upwards half a percent by doubling, or tripling the number of shots a child receives? Is this not a desirable goal?

    These aren’t necessarily that difficult of questions. Now it does occur to me that your nervousness may have led you to decline to respond to these questions; it is your choice, but the casualty will be your credibility. If you do not feel they are valid questions, I’d wonder why not. (?)

    You seem to find it amusing that scientists revise their hypotheses when presented with new data (as suggested by your pre-adolescent “LMAO” response), which explains why you have remained so imperturbable and immutable in the face of data contrary to your position.

    You have provided no data contrary to my position! My position is not that vaccines cause autism! My position is that the area is drastically understudied, and further, that the existing studies suffer from very broad and deep over simplifications that hinder our ability to make quality conclusions about the possible ramifications of our vaccine schedule. My position is that it is insane, completely crazy, that there is no mechanism by which we can ever evaluate the long term effects of aggressively stimulating the still poorly understood immune system in our children; and yet, no one has ever been able to propopse a mechaniism by which we can accomplish this. My position is that is is absolutely antithetical to the tenets of the scientific method that we can add one, ten, or dozens more vaccines onto our schedule without ever contemplating a need for evaluating the cumulative effect. You have provided absolutely no data contrary to any of these positions, so don’t be surprized when I haven’t changed my position.

    - pD

  10. BisserR Says:

    Prometheus,

    Great post as always. I am glad we are actually having a constructive conversation here. It is the first time I see anyone get into the numbers required to do a study at a certain level of confidence.

    The main point of your post is that a study comparing vaccinated vs. unvaccinated children with sufficient power to detect even a significant correlation between vaccinations and autism would be extremely difficult to do. I agree. Given that such study has clearly not been done in the past, on what do you base your confidence there is no connection between vaccines and autism?

  11. Prometheus Says:

    pD,

    I presumed your questions were rhetorical, since you don’t seem to have listened to any answers I have provided in the past. If you are sincerely interested in gathering information, let me see if I can provide answers to your questions.

    What can we learn by analyzing thimerosal other than the impact of thimerosal? What can we learn by analyzing MMR rates other than the effect of the MMR? Is there a chance that there might be other things in the vaccine schedule other than these two components that merit investigation?

    Is there a chance that that there might be “other things in the vaccine schedule” that might “merit investigation”? Of course there’s a chance - there’s a chance (albeit a very small one) that our “vaccine schedule” includes alien implants, but I don’t see much point in spending time or money investigating that either.

    The question - as I addressed above - is what level of negative result will you be willing to accept as sufficient to accept the current “vaccine schedule” as safe. If your threshold is “any chance at all”, then there is no point in doing a study at all, since your expectations are unrealistic.

    Do you believe there are any potential pitfalls towards espousing the notion that studying these two components provides a sufficient model for understanding the impact of the existing schedule as a whole?

    Thimerosal and the MMR vaccine were the first two “components” put forward as putative “causes” of autism. When they failed to “pan out”, other, less well-defined “components” were proposed. I’d say that the people proposing that some “component” of the “vaccine schedule” causes autism or other disorders have failed to show that their hypothesis has merit.

    I see little reason to continue a fruitless search for an unspecified something in vaccines or their administration schedule that “causes” autism or “other neurological disorders” unless and until there is some data pointing that direction.

    What would you consider sufficient data to merit a study of vaccinated versus unvaccinated children? Is there any?

    Show me the data you have that you think merits a study. So far, all I’ve heard is that the “autism epidemic” and “parental intuition” are the only data supporting the alleged “connection”. The data supporting the “autism epidemic” is terribly weak and adding parental opinion doesn’t make the connection to vaccines any stronger.

    If our vaccine schedule quadroupled, would there be no need to see if we might be having unforseen impacts on our children? What if it increased by ten times?

    You reasoning seems to imply that the current vaccination “schedule” just popped into being - that there were no vaccines on one day and “Pop!”, the next day kids were getting 15, 20, 25 vaccinations. However, the reality is that every new vaccine was tested on the background of all the previous vaccines - they were tested on children who had received the other vaccines already.

    As a result, your question is not only rhetorical, it is countersensical. Nobody is proposing to “quadruple” the number of vaccines children receive at one go - nor are they proposing to increase them by ten, twenty or one hundred-fold. And that is not how we got to the current vaccination schedule, either.

    In regards to a vaccinated / heavily vaccinated study, what of our potential ability to creep our rates of immunity upwards half a percent by doubling, or tripling the number of shots a child receives? Is this not a desirable goal?

    You will probably be surprised to discover that researchers have looked at those questions. The dosage, timing, virus strains and adjuvants have been adjusted many times in an attempt to find an optimal amount of antigen to include in a vaccination.

    Contrary to what you might think, it is better (i.e. more effective) to give smaller doses at intervals than to give a single larger dose. There is also little advantage to increasing the amount of antigen - it causes only a small increase in the immunity and increases the side effects.

    You also haven’t addressed the issue that if we did double or quadruple the amount of vaccine and didn’t find any effect in regards autism or other disorders, the study still wouldn’t answer your question: “Does the current vaccine schedule cause autism or other disorders?” It could still be argued that both arms of the study were over the threshold for causing autism (etc.).

    I suspect that there is nothing I or anyone else can say that will displace you from your opinion that the current vaccination schedule is “drastically understudied”, even though you clearly have little understanding of vaccination, the immune system or even the research that has been done on vaccine safety. As a result, I am unsurprised that you feel that I have provided no data contrary to your positions.

    Cheers!

    Prometheus

  12. Prometheus Says:

    Perky Skeptic,

    You may quote me as much as you like.

    I’d like to get that one point spread as widely as possible - I’ll even repeat it here:

    There is no point in doing a study if the people who claim to want the study will not accept a negative result.

    Every study in biology and medicine has a chance that the results came out the way they did (”positive” or “negative”) because of random events.

    If this degree of “uncertainty” will be used by people with fixed ideas or a political agenda to discredit any result they disagree with, there is little point in doing a study to “please” or “appease” them.

    Prometheus

  13. Prometheus Says:

    BisserR,

    I believe that my position is that the data don’t support the connection between vaccines and autism. From what I know of the available data, that is still an accurate statement. My confidence in that statement is based on the preponderance and quality of the data supporting that position.

    If I have written that vaccines don’t cause autism, please tell me where it was so I may correct it.

    The two positions are not equivalent. Saying that there is no reason to believe that vaccines cause autism is not the same as saying that they can’t (or don’t).

    That said, there is always the possibility that in haste or fatigue I may have written that vaccines don’t (or can’t) cause autism. That is not my position. I remain open to the data.

    I am curious why some people - given that there has not been a study showing that vaccines do cause autism - are so confident that there is such a connection. That seems to fly in the face of the available data.

    Prometheus

  14. hera Says:

    Prometheus,
    For many parents, the association: ie “They gave their child a shot, watched them run a high fever immediately afterwards, scream for hours, and never speak again..”seemed pretty conclusive.
    Anecedotal, of course.
    Of course there was recently a case in Georgia where the government conceded a childs autism disorder was linked to vaccines gave money and the the government sealed the records,( why would they do that??) and said this case was rare because it involved a mitochondrial disorder.Interesting and very possible study;the percentage of autistic children with mitochondrial disorder; is it above the norm? The only study done ( small group) seemed to indicate yes.
    The latest in genetics seems to indicate that one of the chromosomal sites indicated is used in gluatione transport, which is involved in heavy metal removal. I’m beeing lazy and not citing the exact study, but a quick search of google scholar will find them for you.
    When you can’t do good population studies, it is useful to be aware of animal studies.Monkeys, who sadly were killed in a recent study, showed inorganic mercury deposits in the brain following thimerosal injection.Looking at what thimerosal and other heavy metals do to mitochondria,is also interesting.
    So :so far it appears that two possible sub groups who may react badly to vaccines; are those who have genetic structure that predisposes them to problems with heavy metal removal, and those with mitochondrial problems.
    You seem a bright man, and hopefully brave enough to follow the data wherever it may lead:maybe you could design some studies that look at the prevelence of these problems in normal/autistic population.

  15. Prometheus Says:

    Hera,

    The case in Georgia is a legal decision, not scientific data. Why “the government” sealed the record almost certainly has to do with privacy - “the government” cannot release the data without the consent of the parents, which is why all the information we have on the Poling case is that which was “leaked” by a non-governmental source.

    There are a number of chromosomal sites linked to autism - over fifty the last time I counted. Some of them are for mitochondrial genes, others for neurotransmitter receptors, others even stranger (including a few in regions that don’t code for any enzyme or structural protein and are probably regulatory regions).

    I suggest that you re-read the Burbacher et al study (the “monkey study”) - it doesn’t say what you think it does.

    I’m very interested in autism research, which is precisely why I don’t want to see money and time flushed down the loo chasing after vague “toxins” in vaccines. If there was any reliable data that showed that “vaccines” (which ones? when?) caused or were correlated with autism, I’d be all over it. However, the data currently suggests that there is no correlation between vaccines and autism.

    I have my own “hunches” about autism and I am pursuing research into them in my own way. It may eventually turn out that a vaccine or a component of vaccines contributes to autism, but the genetic and epidemiologic data don’t point that direction at present.

    I sincerely hope that some good researchers are pursuing the mitochondrial “link” to autism, even though I think it has a low probability of being real. After all, autism has even less in common with the signs and symptoms of mitochondrial disorders than it does with mercury poisoning. Still, the data out of Portugal are intriguing and should be followed up.

    I have proposed - on a couple of occasions - a study of vaccines and autism that pretty much follows the one I’ve outlined above. So far, nobody wants to fund it - not the NIH and not Autism Speaks or any of the other groups that say they want this type of study.

    The NIH gave me the courtesy of explaining why they wouldn’t fund the study. They said (and I paraphrase) that it was a good study but they had other, better studies to fund. I’m preparing to resubmit it to the NCCAM. We’ll see what they say.

    Prometheus

  16. passionlessDrone Says:

    Hi Prometheus -

    I presumed your questions were rhetorical, since you don’t seem to have listened to any answers I have provided in the past. If you are sincerely interested in gathering information, let me see if I can provide answers to your questions.

    Perfect. But, in all seriousness, with my sarcasm meter absolutely at zero; I am always interested in a real conversation. I may not have frequently agreed with your answers in the past, but it would be a mistake for you to believe that I have no interest in your answers, or indeed, that I haven’t listened to them.

    Is there a chance that that there might be “other things in the vaccine schedule” that might “merit investigation”? Of course there’s a chance - there’s a chance (albeit a very small one) that our “vaccine schedule” includes alien implants, but I don’t see much point in spending time or money investigating that either.

    OK. This is where I think my major disconnect is with most folks that seem to take a different spin on this than I do. In my case, I’ve been initiated into this debate through my experiences with autism; but what I’ve found while researching is that the potential problem is much broader.

    We are currently experiencing a wild and completely unexplained upsurge in a variety of still poorly understood diseases with auto immune and/or inflammatory properties. A short list of these things would include type 1 diabetes, food allergies, ashtma, juvenile arthritis, autism, ADHD, and multiple schlerosis. [There are likely more]. There can be debates as to if autism is more prevelant; but I’m really struggling to see how we could fail to notice things like ashtma, food allergies, or type 1 diabetes in the past. What is the alternative diagnosis for asthma, or peanut allergies?

    At roughly the same time we began noticing increases in these conditions, we concurrently began more aggressively stimulating the immune system of every child in the country at an earlier age.

    It seems to me that one of the main reasons frequently given is similar to your ‘alien implants’ reason; because people cannot imagine a way in which there could be an association, it makes no sense to look for one. But this seems to ignore several very important (important to me, anyways) things:

    1) It ignores the very real, repeatedly observed abnormal immune function in the very same conditions we are observing increases in; and vaccines, if nothing else, operate by manipulating the immune system.

    Whatever is happening in autism, food allergies, juvenile arthritis, type 1 diabetes, or asthma; a not normal immune response is either causative, or otherwise associated. I cannot for the life of me understand why we would write off comprehensively evaluating intentional, repeated, and completely novel stimulations of the system that creates cytokines at the same time we observe unexplained increases in diseases associated with abnormal cytokine presence. If there were not abnormal immune system functions within this suite of diseases we are eperiencing, the alien implant argument carries more weight to my mind.

    2) We still have much to learn about how the immune system works, the impact of polymorphisms on immune response, and indeed, how vaccines actually trigger an immune response. It isn’t that I think I’m smarter than immunologists; it is that I think we are all much too dumb to truly understand the impact of our actions without doing evaluations. It was just within the past few years that the mechanism of how aluminum adjuvants actually worked to initiate an immune resopnse has begun to be understood; given that, I am absolutely shocked that a real scientist can say with confidence that we know we cannot be initiating problems far down the line by stimulating the immune system so drastically, and so early; without any decent evaluations.

    Consider the warnings we recently heard regarding tylenol and asthma. It mandated no conspiracy or big pharma shills for this association to remain unseen for so long; it was just that for most of the time, no one could really see how tylenol usage could result in increased risk of developing asthma. Inflammation reduction is a relatively simple process when compared to what we are doing when we intentionally stimulate the immune system in vaccination; I find it incredibly arrogant to believe we understand all of the impacts of our actions. Do you honestly believe differently?

    3) There are too many over simplifications in this discussion. Alien implants is a funny line, but it ignores the fact that I discussed above; there is a clear immune system component to the exact same diseases we are encountering. Likewise, I’ve been told countless times that the immune system is bombarded with antigens everyday; which while no doubt true, has very little to do with the act of bypassing our natural defenses and directly stimulating a response with adjuvants.

    The question - as I addressed above - is what level of negative result will you be willing to accept as sufficient to accept the current “vaccine schedule” as safe. If your threshold is “any chance at all”, then there is no point in doing a study at all, since your expectations are unrealistic.

    How about no increased risk for developing chronic inflammatory and/or autoimmune related diseases? I’ve seen different things on how much of an increased relative risk is meaningful. I am open to suggestions as to what you would feel a meaningful value would be.

    Another thing that would convince me would be for someone to provide a realistic explanation as to why we seem to be experiencing wild increases a variety of diseases with abnormal immune functioning. If such a thing were done, my concerns over the vaccine schedule would ameliorate considerably. No one seems to be able to do this. I am trying not to be unrealistic.

    I see little reason to continue a fruitless search for an unspecified something in vaccines or their administration schedule that “causes” autism or “other neurological disorders” unless and until there is some data pointing that direction.

    You seem to be unable to specify what data might convince you of that, even though I’ve asked you a couple of times now. Not to belate the point, but what type of data would you like to see? Is there any? Above, I have done my best to answer your question as to what would convince me. Humor me back, what would convince you?

    Show me the data you have that you think merits a study. So far, all I’ve heard is that the “autism epidemic” and “parental intuition” are the only data supporting the alleged “connection”. The data supporting the “autism epidemic” is terribly weak and adding parental opinion doesn’t make the connection to vaccines any stronger.

    That is really all you’ve heard? In any case, the clear autoimmune / inflammatory response component of the great number of diseases we are suddenly experiencing now is sufficient evidence for me to evaluate the vaccine schedule is reasonable. I believe the cause for alarm reaches far beyond autism. If you do not feel this is sufficient for some reason, could you tell me why?

    You reasoning seems to imply that the current vaccination “schedule” just popped into being - that there were no vaccines on one day and “Pop!”, the next day kids were getting 15, 20, 25 vaccinations. However, the reality is that every new vaccine was tested on the background of all the previous vaccines - they were tested on children who had received the other vaccines already.

    They were tested for safety issues that focus on things like the frequenty of immediately visible accute reactions as opposed to chronic or neurological conditions which take years to develop. By way of example, I’d bet you cannot show me a single vaccine safety study wherein the researchers performed analysis of autism diagnosis in children who got a new vaccine versus those that did not. That isn’t a retrospective study, but rather one where before licensure and mass distribution the analysis was conducted. The existing safety mechanisms are not well designed to detect chronic problems; largely, I’d say, due to the unimaginable problem I discussed above.

    And even when chronic conditions are found to be associated, it does not always seem to be sufficient to stop a vaccine from being marketed. By way of example, did you know that the Prevnar vaccine is associated with an increased risk of being hospitalized for asthma? Completey true. Take a look at the fda safety sheet for it:

    In analyses of secondary safety outcomes, the adjusted relative risk of hospitalization for reactive airways disease was 1.23 (95% CI: 1.11, 1.35). Potential confounders, such as differences in concomitantly administered vaccines, yearly variation in respiratory infections, or secular trends in reactive airways disease incidence, could not be controlled.

    (I especially love the first confounder they list!)

    One wonders how many parents were told about this before their child was vaccinated with Prevnar? This in of itself, to my mind, is sufficient for us to start asking if we might be having unanticipated impact with our vaccine schedule. I cannot believe that we are so clever that this is the only hidden gem for us left to discover.

    I suspect that there is nothing I or anyone else can say that will displace you from your opinion that the current vaccination schedule is “drastically understudied”, even though you clearly have little understanding of vaccination, the immune system or even the research that has been done on vaccine safety. As a result, I am unsurprised that you feel that I have provided no data contrary to your positions.

    I’m genuinely curious as to where you believe I’ve displayed a poor understanding of vaccination and/or the immune system. Have I in this post?

    One does not need to be an immunologist to understand if there has been a rigorous application of the scientific method or not. I also believe that you are greatly overselling our collective understanding of the immune system, vaccination, and indeed, vaccine safety.

    Whew!

    - pD

  17. Prometheus Says:

    pD,

    Let me answer your last question first - then you might have the answer to the rest of your questions.

    First off, any time I see someone say things like “overwhelm the immune system with vaccines”, I can be pretty sure they don’t understand either vaccines or the immune system, just as you might be able to tell that someone doesn’t understand, say, basketball if they referred to a score as a “touchdown”. [note: I am not a sports "fan", so sports analogies are not my forte]

    Vaccines actually stimulate the immune system a lot less than either the disease they are intended to prevent or - in most cases - the average, run-of-the-mill “cold” virus. As it turns out, the majority of the symptoms of viral and even most bacterial infections are the result of your immune system’s response. The runny, stuffy nose, watering eyes, fever, aches, etc. are all the result of various cytokines released by your immune system. The massive death toll from the 1918 influenza pandemic - especially the unusual mortality among young adults - was the result of their young, healthy, strong immune system “over-reacting” and killing them. Infants and older people - with their “weaker” immune systems - fared better.

    If you’ve ever had a vaccine, you’ll have noted that even the worst of them (in my case, typhoid fever) make you feel no worse than a bad “cold”. Most of them cause even less reaction. This is because the antigen load in vaccines is low. It’s not overwhelming or even “massively stimulating” the immune system - it is “tickling” your immune system, trying to get an anamnestic response without triggering a serious cytokine release.

    As for “overstimulating the immature immune system” - which do you think causes a bigger response in a newborn child: the Hepatitis B vaccine or the hundreds of fungal spores, the thousands of bacteria and the tens of thousands of viruses the child inhales on their very first breath?

    Here’s another question for you to ponder - if vaccines are such an “overwhelming stimulus”, why is it that an infection by the wild-type virus (e.g. measles or rubella) gives a much larger (and longer lasting) immune response than a single vaccination?

    And before you try out the “injected directly into the bloodstream” canard, vaccines are not injected into the bloodstream - they are given intramuscularly or subcutaneously. They are given this way - rather than by ingestion or inhalation (exception: certain live virus vaccines can be given orally or by inhalation) because the gastrointestinal and respiratory tracts are “guarded” by strong “generic” antigen defenses that would eliminate many vaccine antigens without causing an anamnestic response.

    Finally, the idea that autoimmune disorders might be caused by vaccines is not something recent - it has been postulated and explored (so far, without finding any connection) a number of times for a number of autoimmune disorders (e.g. multiple sclerosis) and vaccines (e.g. hepatitis B).

    You seem to think that it is the arrogance of scientists and their reputed unwillingness to challenge fixed dogma that causes the reluctance to investigate the “connection” you propose. Nothing could be further from the truth - scientists who stick to “fixed dogma” are soon devoured by their competitors. There are lots of scientists out there who stand to gain - grants, fame, possible Nobel prizes - by overturning the status quo.

    In fact, there are people investigating exactly the connections you propose. They haven’t found a connection yet. Stay tuned.

    Although it is not my field, I talk to and go to conferences with people investigating autoimmune disorders and one of the more interesting hypotheses is that at least some of them may be due to inadequate stimulation of the immune system. With modern hygiene (”germ”-phobia), antibiotics, cleansers, etc., we aren’t being exposed to enough antigens. It’s an interesting hypothesis and I look forward to seeing the data.

    You see, pD, you have a very superficial understanding of the current knowledge about the immune system and vaccines. It’s probably much greater than the “average” US citizen has (not saying much there), but you don’t even know a tenth of what is known about the human immune system - probably not even one percent. How then can you say that I - or anyone else - is “overselling our collective understanding of the immune system” if you know only a small fraction of what is known? It’s like the Captain of the Titanic saying “It’s just a little piece of ice.”

    Now, I don’t put myself forward as an expert on the immune system - far from it. I consider myself moderately well-versed in the subject but I would never venture to say that I had a complete grasp of it. Even though I freely admit that I am not an expert in the field - or even a researcher in the field - I know far more about the subject than you do. This isn’t “arrogance”, it’s plain, hard fact, as you have demonstrated in your comments. I’d say that you are not well-positioned to give an opinion on the state of knowledge in that field of immunology.

    Now, I have no doubt that this frank discussion and comparison of expertise will sting and cause pD to lash out with some diatribe on the “arrogance of scientists” and how they are “blind” etc, yada, yada, yada.

    The truth often hurts and it’s time that someone talked about “the elephant in the living room”, i.e. how ridiculous it is for someone who has demonstrated that their knowledge of the immune system and vaccines is superficial to insist that they know as much about the subject as the people who studied, trained and research in the field.

    It is true that you don’t have to be an immunologist to “…understand if there has been a rigorous application of the scientific method or not.”, but you do have to know enough about the field to even know what work has already been done. PubMed and Google searches can only get you so far. Remember, not all negative results get published (few journals will publish an article that says, in essence, “We looked for something that was unlikely and failed to find it.”), so you have to be familiar with research in the field to know what has been tried and failed.

    Frankly, I think that I’ve expended enough effort on pD. I welcome questions, but I think that pD is looking for an education. I’d suggest looking at the local university or community college to see if they have a course in immunology. Or, he can simply convince himself that he already knows enough. Either way, I think I need a “time out” from tutoring this reluctant student.

    Prometheus

  18. laurentius-rex Says:

    To all those feckless “somebody ought to do a study” people, why don’t you go do it, you have all the answers, you have the impeccable logic at your fingertips, all you need do is write up a PhD proposal demonstrating how it can be done, then submit it to various medical schools and research bodies.

    That is the established way of doing things, not waiting or dictating that someone else should.

    Of course if your proposal is not accepted it might just mean that it is crap, and not worth a hill of beans, it might be a bit of a come down to discover your logic is not so impeccable after all.

  19. Arthur Golden Says:

    Prometheus,

    Did you receive my short closing comment to your previous blog entry?

    Arthur Golden

  20. Prometheus Says:

    Arthur,

    Yes.

    Prometheus

  21. Joseph Says:

    Perhaps in a third-world country it would be more feasible to do such a study, although you still have the same confounds for the most part.

    An alternative would be to do a VSD-type study (something more thorough like Thompson et al. (2007) is what I’m thinking) except instead of looking at thimerosal dosage, you could look at number of vaccines. Obviously, there should be a dose-response if there’s a real effect.

    There won’t be a “natural experiment” like there was with thimerosal, however. That was a lot more convincing than the individual-level studies in my view. And even that didn’t convince everyone.

  22. storkdok Says:

    Prometheus,

    Thanks for this post, but thank you even more for your last reply to pD. Your succinct and well written reply is a thing of beauty. I only wish I had the time and energy to write something like this! It’s kind of hard with my 3 year old hanging on me.

    I would love to go back to graduate school on the subject of immunology, it is fascinating, and the knowledge in the field since I got my Master’s degree in 1989 in Microbiology/Immunology (before medical school) has increased so much, I would love to study it again!

  23. wfjag Says:

    Another excellent post, with additional excellent responses to comments.

    Personally, as the father of an ASD child, I’m against funding any more studies on any supposed link between vaccines (including components and vaccination frequency) and autism. There are only so many top quality researchers, who have only so much time, and there’s only so much research money. This horse has been beaten to death. There may come a time when there is evidence that some particular vaccine or component may have a correlation to autism (or an ASD subset population) so that a study focused on that is warranted. Until then, there are better places to devote limited resources.

    Dr. Novella proposed one recently. See “Somali Autism Cluster”, Nov. 19. 2008 (Lack of Vitamin D as a possible causal factor in higher ASD rates in Somali immigrants’ children to Minnesota and Sweden). http://www.theness.com/neurologicablog Like any good study, it’s focused on a hypothesis that can be nullified. Further, he suggests a medically plausible mechanism. Insufficient Vitamin D is associated with a number of neurological disorders, and the dark skin of Somalis may prevent them from making enough Vitamin D in the weaker sunlight of northern climes. Finally, he notes that like any cluster, it may just be a statistical fluke. I’d suggest another reason for such a study — doing field work in Minnesota and Sweden has a certain amount of attraction.

    But, why the “Why not another study?” refrain. Again, I believe that Dr. Novella recently summarized the gist of the issue:

    “The skeptical movement, in my opinion, serves a vital role in modern society. We are increasingly dependent upon cutting edge science for our quality of life, and even just to run our complex civilization. And yet, while there seems to be broad respect for science – or at least the fruits of science – in the general public, there is also widespread distrust and overwhelming scientific illiteracy.”

    “Skeptical Battlegrounds: Part I – Background” Dec. 1, 2008.

    His points (which Prometheus has also made any number of times) are our education system falls woefully short in teaching science and the scientific method, but, at the same time there’s a marked tendency to want to believe people who are “experts.” People fail to understand that a good scientific study isn’t designed to prove a hypothesis — it is designed to disprove it. As Prometheus pointed out in a response, he doesn’t say that the studies do not show that vaccines do not cause autism. Rather, he says that the studies fail to show a correlation between vaccines and vaccination and autism. However, the public generally (and politicians, in particular — I fear) do not understand science or the scientific method, and so fail to understand that the negative results (no correlation shown) are what researchers get excited about. And, when multiple studies using multiple approaches get negative results, good researchers want to move on to testing other hypotheses. This isn’t “moving the goalposts”, it is progressing based on accumulating knowledge.

    Finally, Storkdoc — loved your site. Thanks for the link. I’ve added it to My Favorites.

  24. Daisy Says:

    [Note: I've taken the liberty of posting my replies on Daisy's comment in order to increase clarity. Prometheus]

    I do see the difficulty posed in creating such a study, no doubt especially since Autism does not discriminate among the poor or rich, color, religion etc.

    Who would the control subject be?

    There is the biggest problem - who do we use for a control group? Given the nature of the “groups that don’t vaccinate” offered up for consideration, the control group would need to be either a technology-shunning religious group that practices a simple farming lifestyle BUT vaccinates their children (and, actually, the Amish DO vaccinate their children, with an uptake rate of about 70% compared to the 90+% of the general population.) OR an urban organic-lifestyle group that vaccinates their children.

    On top of that, another problem is that the proffered “test” groups (the Amish, a Chicago health-care group) may have OTHER reasons - apart from vaccination - that give them a low autism rate. Or, as we have found with the Amish, it could be that despite their low (or zero) vaccination rate, they actually DO have autism rates comparable to the general population.

    The FIRST thing we need to do is determine if there IS a group that has low autism rates, and we can’t do that by sending a journalist around Lancaster county asking at stores and schools, “You got any autistics around these parts?”. When I asked the nice folks at the Clinic for Special Children (Lancaster County, PA) if they had any patients with autism, their answer was “Yes, plenty - not to mention all the kids with Maple Syrup Urine disease, Crigler-Najjar syndrome ….. etc.”

    What I do know for certain is that my son did react to vaccines even at the age of 11 with regression.

    A lot of people say that - even people with “typical” children. I’m not doubting your truthfulness, just saying that a lot of parents - including myself - have blamed the wrong things for behavioral changes. The medical literature is FULL of studies showing that we mere humans are particularly good at making the WRONG connections.

    I alos know that I have gotten sick after getting the flu vaccine and so have many others.

    The influenza (”flu”) vaccine does not prevent other viral illnesses - the “common cold” is caused by a dizzying number of different viruses. The difference between them and influenza is that influenza KILLS a large number of people every year.

    Vaccines are and can be harmful.

    Nobody actually says that vaccines are “completely safe”. They ARE, however, significantly safer than the diseases they are intended to prevent.

    The question is how many vaccinated kids will it take to prove that vaccines can and do harm just as much as they can save.

    So far, the answer is that vaccines can and do SAVE more than they HARM. If you have data showing otherwise, feel free to present it.

    It is not a one size fits all.

    That’s a great slogan, but what does it mean? Do you REALLY think that children are just lined up and given their “shots” without consideration of their health? Didn’t your children see a pediatrician or other health care provider immediately prior to vaccination?

    Sure I was vaccinated and turned out OK but I did not get as many vaccines as my son nor did I get several at one time.

    How many viruses, bacteria and fungi is a newborn exposed to in their first breath? How many is a child exposed to when the get on an airliner, ride the bus or go to school? These aren’t specially weakened, domesticated (or even DEAD) viruses and bacteria, either - they are the full-on “wild-type”. Do you want to rethink your statement?

    I’m not trying to make fun of you or humiliate you, Daisy. You have been told things that simply AREN’T true by people who - in all liklihood - didn’t understand what they were talking about. In the end, the people who STARTED these “urban legends” are deceiving you and thousands of others.

    We may not be able to accomplish such a study but what must happen is that parents must be informed and educated. I was never educated, or warned nor given optioins. Doctors need to stop telling parents that this is the only option and only schedule to follow.

    I agree that parents need to be educated - they need to know WHY we give the vaccines we do and when we do. You DO know that the vaccine schedule is not just made up out of thin air, don’t you? There have been dozens of changes in the mix and schedule of vaccines over my lifetime, all in response to some new piece of data. The current vaccination schedule is a RECOMMENDATION, not a mandate. It is based on the best science we have to date. If better information comes out tomorrow, the vaccine schedule will be changed.

    My son is an only child so I have no way to compare what might have happened to his siblings. Vaccines are not the only cause for Autism.

    The data show that vaccines are not a cause of autism at all. This doesn’t mean that they CAN’T cause autism or DON’T cause a very small percentage of autism, but there is no data right now showing that vaccines cause ANY autism.

    I am open to the fact that for some genetic factors outweigh the environmental factors.

    Again, we don’t know if the genetic factors are greater than environmental or vice versa. What we DO know is that two “environmental” factors have been pretty thoroughly ruled out: mercury and vaccines.

    This is what makes Autism so complex. I wish I had the answers or the means and way to have such a study not for my son’s sake for it is too late for him but for the sake of all other kids and for the sake of our furture generation. Thank you for listening and thank you for your time.

  25. Science-Based Medicine » The perils and pitfalls of doing a “vaccinated versus unvaccinated” study Says:

    [...] the unvaccinated vary in significant ways from the vaccinated. Skeptical blogger extraordinaire Prometheus tells the tale. First, he points out how few completely unvaccinated children there are to study, perhaps around [...]

  26. The National Autism Association’s Lies & Distortions About Paul Offit « Confutata Says:

    [...] study would be nearly impossible to do with any sample size that would provide a useful result (see this for elaboration.) It would be a waste of resources, and in my opinion, would almost certainly not [...]

  27. Phyllis Wheeler Says:

    I am happy to see this rational discussion of a subject dear to my heart. Thank you, Prometheus and commenters, for providing it!

    It seems logical to me that an epidemic can’t have solely a genetic cause, and so that there must be some environmental trigger or cause involved. Of course we don’t know whether vaccines might be involved. But I REALLY would like to see a study that evaluates this possibility.

    As I understand it, there has been research that refutes an autism-thimerosal link, and research that refutes an autism-MMR link, but no overarching research refuting something else about vaccines that could be a trigger or cause.

    You, Prometheus, are arguing that the numbers aren’t there to compare unvaccinated vs. vaccinated and get a meaningful result (unless the results are overwhelmingly in one direction or the other).

    I have a suggestion to make. What about studying younger unvaccinated or under-vaccinated siblings of autistic kids? I believe there must be a significant number of them, because of the prevalent suspicion that vaccines may cause autism. These kids are already “matched” to their older sibling, sort of. Can you make a statistically significant study out of these kids?

    Finding them might not be too hard: just putting out the word through Generation Rescue might bring them out of the woodwork.

    I think there must be a way to solve this problem of designing a study.

  28. Prometheus Says:

    Ms. Wheeler,

    I completely agree that an “epidemic” is unlikely to have a solely genetic cause. However, there is stiill a great deal of debate over whether or not the rise in autism prevalence is a real rise or an artifact of improving awareness and shifting diagnostic criteria. Before using the “no genetic epidemic” argument, it would be best to determine if there actually is an epidemic.

    Secondly, you acknowledge that the thimerosal-causes-autism and MMR-causes-autism hypotheses have been refuted by several studies. Since these were the two components originally thought by the supporters of vaccines-cause-autism hypotheses to be the most likely causes of autism, you are also acknowledging that you (and other supporters of the vaccines-cause-autism hypotheses) are grasping at straws.

    Having already shown that the most likely “culprits” aren’t associated with autism , are we now committed to run through each and every vaccine, vaccine component and all possible permutations and combinations? Or, should we focus our resources on areas of inquiry that are more likely to yield useful results?

    Your suggestion of using unvaccinated siblings of autistic children is interesting. However, using “Generation Rescue” as a recruiting source is flawed because its members would be more likely than the general population to have unvaccinated siblings that don’t have autism, based on the bias of the organisation (i.e. parents whose subsequent children were unvaccinated and developed autism are not likely to find Generation Rescue’s message convincing).

    If the autistic children were chosen at random - which would be necessary to avoid selection bias - it could be made to work, as long as the siblings were old enough to be reliably evaluated for autism. Of course, there would still be the criticism that the study compares “apples to oranges”, since the subjects (case and control) would be of different ages.

    A bigger concern - and one I’ll have to “run the numbers” on - is how the known genetic contributions to autism would confound such a study. As a rough estimate of the problem, let’s look at a recent twin study of autism [Rosenberg et al (2009)]. This study showed a MZ concordance of 88% and a DZ concordance of 31%, which indicates a high degree of genetic heritability. Lauritsen et al (2005) showed that siblings of autistic children had an autism prevalence of 1.76%, while the general population had an autism prevalence of 0.08% - a 22-fold increased risk of autism for siblings of autistic children.

    This means that if the prevalence of autism in the general population is 1% (as the latest data suggest), we could expect 22% (or so) of unvaccinated siblings of autistic children to develop autism if vaccines have no impact on autism.

    Since - as you suggest - the younger siblings of autistic children (especially those whose parents follow Generation Rescue) are less likely to be vaccinated than the general population, the study you propose is likely to (falsely) show that being unvaccinated (or partially vaccinated) increases the risk of developing autism.

    Still, it might be possible to use the sibling comparison to both look at vaccination effects and the heritability of autism. When I get the chance, I’ll do some modeling of the study you propose and post the results.

    Finally, I want to reiterate that I’m not saying that a study of “vaccinated vs unvaccinated” or some other variation looking at vaccination status and autism is impossible - far from it! In fact, I’ve described above exactly how I would propose to do such a study, were I given the funding.

    All it would take to do this study is funding (and a willingness to accept that vaccination effects on autism prevalence will have to be fairly large to be detectable). If GR, AS or any other group is willing to provide the funding, I’m willing to do the study or help arrange to get it done.

    Prometheus

  29. Autism Blog - Generation Rescue’s Vaccinated/Unvaccinated Study « Left Brain/Right Brain Says:

    [...] the limitations of these studies and, also how they could be done. One early example is in his post “Let’s put on a Study!” Dr. David Gorski goes into the details as well in The perils and pitfalls of doing a [...]

  30. Tanja Says:

    Hi!
    I would ask a little bit of help with the sample size calculator. I tried to run Your numbers, so that I would learn how to use that calculator, but I just can’t seem to get the same results, or even any “reasonable” ones. I actually got a sample size of 112 for each group, which doesn’t really make sense since there might be no autistic children in the other group at all with that a small size… So, I would appreciate if You could go through the calculation a little bit more thoroughly. I tried the “percentages, two samples” calculator, and used 0,6 (1 / 150) and 10,6 values (10% difference) for samples. Where did I go wrong?

    Thanks!

  31. Prometheus Says:

    Tanja,

    The problem is that 10.6% isn’t 10% more than 0.6% - it’s nearly 18 times more (1767% larger, to be precise). A difference that large would be relatively easy to detect, which is why you came up with such small sample sizes.

    If your hypothesis is that - for example - unvaccinated children will have 10% less autism than vaccinated children, you would enter 0.6% for one group and 0.54% for the other. With alpha and beta errors set to 5%, that would give you sample sizes of over 300,000 subjects (340,746) in each group (681,492 total).

    If you choose values that are a little more reasonable - say 0.6% in the vaccinated group and 0.3% on the unvaccinated group - then the sample sizes are only 10,771 for each group. If you’re willing to accept a 50% chance that you’ll erroneously find that there is no difference when a difference exists (beta error) - which is the more standard value - then you can get by with a bit under 2,700 subjects in each group.

    I hope this clears up the matter for you.

    Prometheus

  32. Tanja Says:

    Aaaa, of course! Oh no, I feel quite stupid now… I had a feeling that there was some fundamental error in my “logic” but I just didn’t get what it might be. Well, I’ve never been good with numbers, especially with %-numbers.
    But thanks, now I can play with the tool on my own also :-).

  33. Science-Based Medicine » Vaccines and infant mortality rates: A false relationship promoted by the anti-vaccine movement Says:

    [...] “vaxed versus unvaxed” study, despite the known difficulties with such a study and the low likelihood of finding anything without huge numbers of [...]