The New York Times is running a Room for Debate forum on whether the federal government should provide a tax credit to families who home school their children. The Fordham Institute chief Checker Finn and Home School Legal Defense Association legal counsel William Estrada are among the contributors.
In my contribution to the forum, I express no quarrel with a tax credit so long as it comes with accountability strings attached. The main point I tried to make is that we know astonishingly little about the academic performance of children who are homeschooled. The sad truth about home schooling is that we have little more than glorified anecdotes.
The word limit for contributors is strict, so I could only baldly assert this in the NYT forum. Here’s the argument and evidence for the assertion.
For a quick overview, first check out the comprehensive and detailed site maintained by Professor Robert Kunzman of Indiana University. Kunzman has written sympathically about homeschooling, but he explains why the frequent claims that the “average homeschooler” outperforms public and private school students are unjustified.
For a more detailed argument, read an overview on why home schooling needs to be regulated a few years ago. Download a version of it here. It contains a discussion of the lack of evidence on the outcomes of home schooling.
Why do we lack such evidence? The reason is related to the massively de-regulated environment for homeschoolers. Because existing regulations for home schooling are either so minimal or so little enforced, many parents do not notify local educational officials when they decide to home school. At least ten states do not even require parents to register their home schools. A great deal of home schooling occurs “under the radar”, so to speak, so that even if local officials wished to test or monitor the progress of home schooled students, they wouldn’t even know how to locate them. Researchers and public officials have, quite literally, no sense of the total population of home schooled students. This is the primary obstacle to studying home schooling.
A further concern is that an appalling amount of the research conducted on home schooling and given publicity in the media is undertaken by or sponsored by organizations whose explicit mission is to further the cause of home schooling. Of course, that research is conducted by persons whose pay comes from organizations dedicated to promoting home schooling is no reason to reject the findings out of hand. I would suggest, however, that we treat the findings of their research on home schooling in the same way the people treat the research on nicotine addiction funded by tobacco companies: with a very large dose of skepticism.
Consider one of the most widely publicized studies in the home school research literature, the 1999 report by Lawrence Rudner entitled “Scholastic Achievement and Demographic Characteristics of Home Schooled Students in 1998.”
Rudner’s study was funded and sponsored by the Home School Legal Defense Assocation. It analyzed the test results of more than 20,000 home schooled students using the Iowa Test of Basic Skills, and it was interpreted by many to find that the average home schooled student outperformed his or her public school peer. But Rudner’s study reaches no such conclusion, and Rudner himself issued multiple cautionary notes in the report, including the following: “Because this was not a controlled experiment, the study does not demonstrate that home schooling is superior to public or private schools and the results must be interpreted with caution.” Rudner used a select and unrepresentative sample, culling all of his participants from families who had purchased curricular and assessment materials from Bob Jones University. Because Bob Jones University is an evangelical Christian university (a university which gained a national reputation in the 1980s for its policy of forbidding interracial dating), the sample of participating families in Rudner’s study is highly skewed toward Christian home schoolers. Extrapolations from this data to the entire population of home schoolers are consequently highly unreliable. Moreover, all the participants in Rudner’s study had volunteered their participation. According to Rudner, more than 39,000 contracted to take the Iowa Basic Skills Test through Bob Jones, but only 20,760 agreed to participate in his study. This further biases Rudner’s sample, for parents who doubt the capacity of their child to do well on the test are precisely the parents we might expect not to volunteer their participation. A careful social scientific comparison of test score data would also try to take account of the problem that public school students take the Iowa Basic Skills Test in a controlled environment; many in Rudner’s study tested their own children.
Rudner himself has been frustrated by the misrepresentation of his work. In an interview with the Akron Beacon Journal, which published a pioneering week-long investigative series of articles on home schooling in 2004, Rudner claimed that his only conclusion was that if a home schooling parent “is willing to put the time and energy and effort into it – and you have to be a rare person who is willing to do this – then in all likelihood you’re going to have enormous success.” Rudner also said, “I made the case in the paper that if you took the same kids and the same parents and put them in the public schools, these kids would probably do exceptionally well.”
Absent rigorous, social scientific data on the outcomes of home schooling, we are left in the realm of anecdote – the home schoolers who win the National Spelling Bees – and the occasional ethnographic study of small populations of home schoolers. But neither can give us any picture of whether home schooling “works”. The very best research on home schooling – the combination of random samples of large populations and ethnographic studies, yields some good information about the reasons why people home school and demographic characteristics of their households. But when we look at the academic performance of home schooled children, the bottom line is that we know virtually nothing.
 Lawrence Rudner, “Scholastic Achievement and Demographic Characteristics of Home Schooled Students in 1998,” Educational Policy Analysis Archive, Vol. 7, No. 8, 1999.
 Doug Oplinger and Dennis J. Willard, ‘Claims of Academic Success Rely on Anecdote, Flawed Data Analysis”, Akron Beacon Journal, November 15, 2004.
 For a good study with ethnographic components, see Mitchell Steven’s The Kingdom of Children (Princeton: Princeton University Press, 2001).