Monday, April 18, 2011

Educational Waste

I'm mostly cribbing from the blog Kitchen Table Math, but here are a few items I found troubling:

1. A comment about elementary math classes:
In the last few months, I visited over a dozen elementary schools. Mostly I visited kindergartens, but whenever possible, I visited the 1st, 2nd, 3rd and 4th grades as well.

Over and over I saw schools where "math class" was the same template: children doing activities from Everyday Math on their own in chaotic, loud classrooms where students didn't have individual desks but had to sit at group tables (sometimes putting up their books and folders to act as little cubicle walls) while they waited for a teacher or an aide to interact with them. Uniformly, I saw half a dozen kids doing nothing at all in those times; another half a dozen chatting or playing but obviously not doing anything, and a precious few trying to block out the stimulus. Some read cheap fiction books.

No one could have learned anything in such a room even before you find out that the task at hand is some bizarre manipulative task in Everyday Math that had no goal or explained purpose anyway.
2. A New York Times story about the vacuity of some collegiate work:
"IN “Academically Adrift,” Dr. Arum and Dr. Roksa looked at the performance of students at 24 colleges and universities. At the beginning of freshman year and end of sophomore year, students in the study took the Collegiate Learning Assessment, a national essay test that assesses students’ writing and reasoning skills. During those first two years of college, business students’ scores improved less than any other group’s. Communication, education and social-work majors had slightly better gains; humanities, social science, and science and engineering students saw much stronger improvement.

What accounts for those gaps? Dr. Arum and Dr. Roksa point to sheer time on task. Gains on the C.L.A. closely parallel the amount of time students reported spending on homework. Another explanation is the heavy prevalence of group assignments in business courses: the more time students spent studying in groups, the weaker their gains in the kinds of skills the C.L.A. measures.

Group assignments are a staple of management and marketing education. In dorm lounges and library basements around the country, small cells of 20-year-olds are analyzing why a company has succeeded or failed (Drexel University); team-writing 15-page digital marketing plans (James Madison University); or preparing 45-minute PowerPoint presentations on one of the three primary functions of management (Tulane University).

The pedagogical theory is that managers need to function in groups, so a management education without such experiences would be like medical training without a residency. While some group projects are genuinely challenging, the consensus among students and professors is that they are one of the elements of business that make it easy to skate through college.

Donald R. Bacon, a business professor at the University of Denver, studied group projects at his institution and found a perverse dynamic: the groups that functioned most smoothly were often the ones where the least learning occurred. That’s because students divided up the tasks in ways they felt comfortable with. The math whiz would do the statistical work, the English minor drafted the analysis. And then there’s the most common complaint about groups: some shoulder all the work, the rest do nothing.
3. This 2006 article (Richard Elmore, "Three Thousand Missing Hours," Harvard Education Letter):
One of the most remarkable things about American classrooms is how little real teaching goes on there. Over the past five years or so, I have spent at least three or four days a month in schools studying the relationship between classroom practice and school organization. I observe classrooms at all levels—primary, middle, and secondary grades—and in all subjects. One of the most striking patterns to emerge is that teachers spend a great deal of classroom time getting ready to teach, reviewing and reaching things that have already been taught, giving instructions to students, overseeing student seatwork, orchestrating administrative tasks, listening to announcements on the intercom, or presiding over dead air — and relatively little time actually teaching new content.

When my fellow researchers and I code our observations for teaching new content, it is not unusual to find that it occupies somewhere between zero and 40 percent of scheduled instructional time.

[He describes videos of American and Japanese lessons.] When American educators watch these two lessons they are shocked by the difference. Students in the Japanese lesson are fully engaged in new content for the entire class, while in the American lesson it is difficult to discern what the new content actually is, much less how much time is dedicated to it.
4. My son is in what seems to be a well-regarded public middle school. In the past few weeks, here are the assignments I've seen him working on at home: making a video about Edwin Hubble (it had very little information about Hubble in it, but the teacher said his was one of the best videos in the class); making a fake Facebook page about Edwin Hubble (same); and writing up a description of a fake dinosaur that he had imagined. I'm not too worried about his science knowledge (when he was 9, he demanded that I subscribe to Scientific American for him to read), but I'm not confident that the school is doing as much as it could to instill knowledge in its students.

Labels:

Saturday, March 12, 2011

Value-Added Has Errors . . . Which Means What?

A recent New York Times article profiled a dedicated and well-reviewed middle school teacher who may not get tenure thanks to the New York City value-added model that places her at the 7th percentile (judging by how much growth her students showed on tests).

Assuming that the article is correct, and that the value-added model here is being unfairly applied for any number of reasons, that means that value-added models can make Type II errors (false negatives).

OK, so we should ditch value-added models and keep on with the "last in, first out" system in which the newest teachers get laid off first (in the case of layoffs). No messing around with incomprehensible math that could be wrong.

But wait, the "last in, first out" model can make Type II errors too. Look at what happened last year in Wisconsin, where the zero-deductible health insurance plan was so expensive (note: this is one reason why Governor Walker wanted to strip collective bargaining over such benefits) that it required Milwaukee to lay off some new teachers acknowledged as good:
Megan Sampson was named outstanding first-year teacher by the Wisconsin Council of Teachers of English last week.

Second-year social studies teacher Kevin Condon, also at Bradley Tech High School, has four licenses and can command the attention of 40 students in an open-concept classroom.

Both are among 482 educators - more than 12% of the full-time teachers in the district - who have received layoff notices from Milwaukee Public Schools.

On Monday - the last day of the year for schools in MPS and the first day teachers reunited after hearing the news of the layoffs - some teachers expressed frustration at losing their jobs because of experience, not performance. . . .

Meanwhile, the Milwaukee School Board president and the Milwaukee Teachers' Education Association leadership continued to disagree on how to handle the tightened $1.3 billion district budget, and whether teachers should accept a lower-cost health-care plan to avoid layoffs. . . .

Bonds said if all teachers switched to the lower-cost plan, about $48 million could be saved, enough to pay for 480 educators.

"I'm not aware of any place in the nation that pays 100% of teachers' health-care benefits and doesn't require a contribution from those who choose to take a more expensive plan," Bonds said.
. . .

Sampson and her laid-off colleagues, all who have less than three years of teaching experience, also expressed frustration that their jobs would be filled by more veteran, but not necessarily better, educators.

Sampson and Emily Kaphaem, a world geography and citizenship teacher at Tech, said they have received exemplary performance reviews.

"I feel kind of let down by my city today," said Kaphaem, 25, as she lost the fight to hold back tears in Principal Ed Kupka's office.

Kupka is equally frustrated. He hand-selected the new teachers because of their talent and enthusiasm for turning around Tech, recently designated as one of the worst-performing high schools in the state.

"Based on the pressures we're under as a low-performing school, I absolutely would have chosen a different nine (for layoffs)," Kupka said.
So now what? Well, for one thing, it's silly for ideological advocates on either side to act as if they have a perfect system that will preserve the best teachers, either by 1) improperly equating quality with years of experience or 2) improperly assuming that each teacher can be given an exact number that represents her quality. It's equally silly for ideological advocates to act as if they've proven the opposite side's system to be some horrible inequity just because it results in an occasional good teacher getting fired. Unless God is running the system, there are going to be errors.

What we need to know is which system makes more errors. But what constitutes an "error"? One way to think about this would be to assume, for the sake of argument, that teachers really are good teachers when they have received outstanding (not merely adequate) performance reviews from their principals. So then, if layoffs are going to occur, would more teachers deemed outstanding by their principals be laid off under 1) a value-added model, or 2) the "last in, first out" standard?

Labels:

Wednesday, March 09, 2011

Charter Schools and Averages

We often hear that something has no overall effect on something else -- education, health, whatever -- but it seems to me that in a world with widely varying individuals and situations, the overall mean effect isn't very interesting or useful. At least not as often as some people seem to think.

For example, we might hear that a certain kind of medicine has no overall survival benefit, which is the reason that the FDA moved to block to approval of Avastin to treat metastatic breast cancer. But even if there is little overall mean effect, Avastin could still cause a remission in a few people:
Christi Turnage of Madison, Miss., said her cancer has been undetectable for more than two years since starting therapy with Avastin. She was diagnosed with breast cancer in June 2006 and began taking the drug in 2008 after the tumors spread, or metastized, to her lungs. Breast cancer that spreads to other parts of the body is generally considered incurable.
If it turns out that you're one of the few people for whom the medicine works, it's no comfort to be told that you can't take it because not enough other people would benefit if they took the medicine.

The same is true for charter schools. We've heard about several studies indicating that charter schools don't have a higher average effect than regular public schools. Take, for example, this Mathematica study of charter schools in 15 different states. It found no overall impact on the students. But this masks an important variation in who benefited and who didn't:
We found that study charter schools serving more low income or low achieving students had statistically significant positive effects on math test scores, while charter schools serving more advantaged students—those with higher income and prior achievement—had significant negative effects on math test scores.
In other words, charter schools were doing very different things for different students — raising up low income and poor-scoring students while actually harming richer and higher-scoring students' test scores.

Average them all together, and you find no effect. But if you want to expand charter schools in impoverished urban areas, the “no overall average benefit” finding would be completely beside the point.

Consider as well this very recent study of charter schools in Milwaukee. The author (Hiren Nisar) says that "charter schools on average have no significant effect on student achievement." An opponent of charter schools (say, a Diane Ravitch) would cite that finding as if it represented the entirety of the study.

But Nisar goes on to find that the overall average is hiding a critically important distinction:
Charter schools with higher level of autonomy from the district in terms of financial budget, academic program, and hiring decisions, are effective. I show that students in these charter schools would read at a grade level higher than similar students who attend a traditional public school in three years. Irrespective of the type and the age of the charter school, race of the student, or grade level, attending a charter school has a positive effect on low achieving students. I show that these effects on low achieving students are substantial and are more than enough to eliminate the achievement gap in two years.
Once again, the overall average is completely meaningless if you are interested in expanding the very charter schools that are most likely to work, i.e., the ones that serve low-achieving students and that have more autonomy from their competition (the school district).

Labels:

Monday, February 14, 2011

The Law of Conservation of Mathematical Symbols

One of the frustrating things about math is that some symbols are reused again and again, as if someone was afraid of running out of symbols.

For example:

1. ' can mean a derivative of a function, or the transpose of a matrix or vector. This is confusing when a book refers to the derivative of a vector. Hmm, which is it?
2. | | can mean the absolute value of something, or the determinant of a matrix. This is confusing when transforming random variables and you need to refer to the absolute value of a Jacobian determinant.
3. ~ can mean "scales as" or "is distributed as."
4. Σ can mean the operation "sum the following" or the item "covariance matrix."

I'm reminded of Demetri Martin's line about vitamins: "When they were naming vitamins they must have thought there were going to be way more vitamins than there ended up being. OK let's name these: Vitamin A, Vitamin B...ok man slow down we've got a lot to cover here. B2, B3, B4, B5, B6, B12. Then they got to E and they were like 'We're pretty much done. We've got all those damn B's. This is embarrassing. Let's just skip to K and get the hell out of here."

Labels:

Saturday, February 05, 2011

Good article

My good friend and law school classmate Mark Rienzi (of Catholic Univ. School of Law) has a new article on the horizon. It argues as follows:
The Fourteenth Amendment rights of various parties in the abortion context – the pregnant woman, the fetus, the fetus’ father, the state – have been discussed at length by commentators and the courts. Surprisingly, the Fourteenth Amendment rights of the healthcare provider asked to provide the abortion have not. Roe and Casey establish a pregnant woman’s Fourteenth Amendment right to decide for herself whether to have an abortion. Do those same precedents also protect her doctor’s right to decide whether to participate in abortion procedures?

The Court’s substantive due process analysis typically looks for rights that are “deeply rooted” in our history and traditions. Accordingly, this article addresses the historical basis for finding that providers do indeed have a Fourteenth Amendment right to refuse to perform abortions. This historical analysis shows that the right to refuse passes the Court’s stated test for Fourteenth Amendment protection. In fact, the right to refuse actually has better historical support, and better satisfies the Court’s stated tests, than the abortion right itself.

Beyond this historical case, a healthcare provider’s right to make this decision also fits squarely within the zone of individual decision-making protected by the Court’s opinions in Casey and Lawrence v. Texas, and protects providers from the types of psychological harm that the Court recognized in Roe and Casey. For these reasons, under Roe and Casey, a healthcare provider has a Fourteenth Amendment right to refuse to participate in abortions.
Worth checking out.

Wednesday, February 02, 2011

C.S. Lewis on 16th Century English


I've been slogging my way through C.S. Lewis's book English Literature in the Sixteenth Century, which is the only Lewis book that I haven't yet managed to read. It's been slow going, and I now understand why Lewis sarcastically referred to the book as OHEL (i.e., the Oxford History of English Literature), pronounced "O Hell."

The book is, of course, immensely learned, as one might expect (Alan Jacobs notes that “Lewis read every single sixteenth-century book in Duke Humfrey’s Library, the oldest part of Oxford’s great Bodleian Library”). But that's what makes it so difficult. The book is written as if you, the reader, are already familiar with all of the authors, political movements, theological and philosophical developments, etc., from the 16th century, and therefore don't need much explanation. Thus, nearly every page has numerous sentences that throw out names and terms in passing that are left undefined and unexplained, as Lewis immediately moves on to something else. (An example: "Wilder and more 'eldritch' even than this is the Dreme of 'Lichtoun Minocus' (Bannatyne CLXV); a dream, which has, for once, no allegorical significance.")

The book is nonetheless enjoyable if only for those flashes of insight that I do understand.

For example, this passage:
It may or may not have been noticed that the word Renaissance has not yet occurred in this book. I hope that this abstinence, which is forced on me by necessity, will not have been attributed to affectation. The word has sometimes been used merely to mean the 'revival of learning,' the recovery of Greek, and the 'classicizing' of Latin. If it still bore that clear and useful sense, I should of course have employed it.

Unfortunately, it has, for many years, been widening its meaning, till now 'the Renaissance' can hardly be defined except as 'an imaginary entity responsible for everything the speaker likes in the fifteenth and sixteenth centuries'. If it were merely a chronological label, like 'pre-Dynastic' or 'Caroline' it might be harmless. But words, said Bacon, shoot back upon the understandings of the mightiest. Where we have a noun we tend to imagine a thing. The word Renaissance helps to impose a factitious unity on all the untidy and heterogeneous events which were going on in those centuries as in any others.

Thus the 'imaginary entity' creeps in. Renaissance becomes the name for some character or quality supposed to be immanent in all the events, and collects very serious emotional overtones in the process. Then, as every attempt to define this mysterious character or quality turns out to cover all sorts of things that were there before the chosen period, a curious procedure is adopted. Instead of admitting that our definition has broken down, we adopt the desperate expedient of saying that 'the Renaissance' must have begun earlier than we had thought. Thus Chaucer, Dante, and presently St. Francis of Assisi, became 'Renaissance' men. A word of such wide and fluctuating meaning is of no value. Meanwhile, it has been ruined for its proper purpose. No one can now use the word Renaissance to mean the recovery of Greek and the classicizing of Latin with any assurance that his hearers will understand him. Bad money drives out good.

It should also be remembered that the word Renaissance is in a curiously different position from the general run of historical terms. Most of these, when not merely chronological, designate periods in the past by characteristics which we have come, in the course of our historical studies, to think distinctive or at least convenient. The ancients were not ancient, nor the men of the Middle Ages middle, from their own point of view. Gothic architecture was not 'Gothic' at the time, it was merely architecture. No one thought of himself as a Bronze Age man.

But the humanists were very conscious of living in a renascentia. They claimed vociferously to be restoring all good learning, liberating the world from barbarism, and breaking with the past. Our legend of the Renaissance is a Renaissance legend. We have not arrived at this conception as a result of our studies but simply inherited it from the very people we were studying.

If the earlier modern scholars had not themselves been bred in the humanist tradition it may be doubted whether they would have chosen so lofty a name as 'rebirth' to describe the humanist achievement. The event, objectively seen, would perhaps have appeared not quite so important nor so wholly beneficent.

But, once established, this glowing term inevitably linked itself in the minds of English scholars with those two other processes which they highly approved in the sixteenth century -- the birth of Protestantism and the birth of the physical sciences. Hence arose, as it seems to me, that strong prejudice, already more than once alluded to, which predisposed our fathers to see in this period almost nothing but liberation and enlightenment; hence, too, by reaction, and among scholars of anti-Protestant sympathies, the opposite tendency to see in it little else than the destruction of a humane and Christian culture by kill-joys and capitalists. Both views perhaps exaggerate the breach with the past; both are too simple and diagrammatic. Both thrust into the background things which were, at the time, important.
Better yet, consider this passage on discerning the "spirit" of historical periods:
Some think it the historian's business to penetrate beyond this apparent confusion and heterogeneity, and to grasp in a single intuition the 'spirit' or 'meaning' of his period. With some hesitation, and with much respect for the great men who have thought otherwise, I submit that this is exactly what we must refrain from doing. I cannot convince myself that such 'spirits' or 'meanings' have much more reality than the pictures we see in the fire.

Whether the actual content of the past or (less plausibly) of some artificially isolated period in the past has a significance is a question that need not here be raised. The point is that we can never know that content. The greater part of the life actually lived in any century, any week, or any day consists of minute particulars and uncommunicated, even incommunicable, experiences which escape all record. What survives, survives largely by chance. On such a basis it seems to me impossible to reach the sort of knowledge which is implied in the very idea of a 'philosophy' of history. There is also this to be said of all the 'spirits', 'meanings', or 'qualities' attributed to historical periods; they are always most visible in the periods we have studied least. The 'canals' on Mars vanished when we got stronger lenses.
And this was even better:
It is impossible not to wonder at this sudden extinction of a poetical literature which, for its technical brilliance, its vigour and variety, its equal mastery over homely fact and high imagination, seemed 'so fair, so fresshe, so liklie to endure'. . . . But however we explain the phenomenon, it forces on our minds a truth which the incurably evolutionary or developmental character of modern thought is always urging us to forget. What is vital and healthy does not necessarily survive. Higher organisms are often conquered by lower ones. Arts as well as men are subject to accident and violent death. . . .

We ask too often why cultures perish and too seldom why they survive; as though their conservation were the normal and obvious fact and their death the abnormality for which special causes must be found. It is not so. An art, a whole civilization, may at any time slip through men's fingers in a very few years and be gone beyond recovery. If we are alive when such a thing is happening we shall hardly notice it until too late; and it is most unlikely that we shall know its causes.
On a lighter note, some of the original sources that Lewis describes seem humorous, intentionally or unintentionally. Consider the following Scottish comic poetry from some time prior to 1501:
It relates (in rough four-beat couplets) the creation of the first Highlander. God and St. Peter were out for a walk in Argyll one day when St. Peter, observing a certain unsavoury object on the path, jokingly suggested that God might like to create something from it. One stir of the almighty 'Pykit staff' and "vp start a helandman, blak as ony draff'. Questioned about his plans, the new creature announced that he would be a cattle-thief. God laughed heartily, but even while He was doing so (it is like Mercury and Apollo in Horace's ode) the Highlander had contrived to steal His pen-knife.
Or this:
Indeed it is impossible to read Mulcaster long without smiling. Sometimes he meant us to smile. He recalls innumerable scenes in headmasters' studies when he writes of the father who will 'very carefully commend his silly poor boy at his first entry, to his maisters charge, not omitting euen how much his mother makes of him, if she come not her selfe and do her owne commendation' (Positions, IV). But he is quite serious when he recommends holding the breath as a beneficial physical exercise 'though all men can tell what a singular benefit breathing is' (ibid. XV) or writes 'Consider but the vse of our legges, how necessarie they be (ibid. XX).

In other Lewis news, I see that someone finally reissued Lewis's co-authored 1939 book The Personal Heresy: A Controversy, which had long been out of print. In fact, I corresponded a few times with Douglas Gresham (Lewis's stepson) several years ago to lament that very fact.

Sunday, January 23, 2011

Battle Hymn of the Tiger Mother


Yale law professor Amy Chua has certainly raised a firestorm with her new book Battle Hymn of the Tiger Mother, which shot to fame with this Wall Street Journal preview. A sample:
A lot of people wonder how Chinese parents raise such stereotypically successful kids. They wonder what these parents do to produce so many math whizzes and music prodigies, what it's like inside the family, and whether they could do it too. Well, I can tell them, because I've done it. Here are some things my daughters, Sophia and Louisa, were never allowed to do:

• attend a sleepover

• have a playdate

• be in a school play

• complain about not being in a school play

• watch TV or play computer games

• choose their own extracurricular activities

• get any grade less than an A

• not be the No. 1 student in every subject except gym and drama

• play any instrument other than the piano or violin

• not play the piano or violin.
So far, so good, it seems to me. (OK, mostly good, and often just in theory as to my own family . . .) Nonetheless, many reactions to the book/article are very hostile, primarily because of vignettes in which she calls her daughter "garbage" for being terribly disrespectful, or threatening to burn another daughter's stuffed animals.

I got the book one afternoon last week, and devoured the whole thing by that evening, which is unusual for me only in that I don't usually have enough spare time to do that anymore. I was surprised to find myself constantly laughing over her self-deprecating and ironic remarks. I later saw that Chua herself told the Wall Street Journal that "much of my book is tongue-in-cheek, making fun of myself."

To take just a few examples:
  • Chua explaining why she made one daughter learn the piano:

    I wanted her to be well rounded and to have hobbies and activities. Not just any activity, like "crafts," which can lead nowhere -- or even worse, playing the drums, which leads to drugs -- but rather a hobby that was meaningful and highly difficult with the potential for depth and virtuosity. And that's where the piano came in.

  • Chua explaining why third-generation Chinese Americans turn out poorly:

    Finally and most problematically, they will feel that they have individual rights guaranteed by the U.S. Constitution and therefore be much more likely to disobey their parents and ignore career advice.

  • Chua explaining why she ultimately decided not to focus too heavily on her dog:

    I had finally come to see that Coco was an animal, with intrinsically far less potential than Sophia and Lulu. Although it is true that some dogs are on bomb squads or drug-sniffing teams, it is perfectly fine for most dogs not to have a profession or even any special skills.

  • Chua explaining why she didn't fit in at Harvard Law School:

    But I always worried that law really wasn't my calling. I didn't care about the rights of criminals the way others did.

That last line was especially hilarious to me. I'll never forget one of the women I knew (and was actually good friends with) at Harvard Law School: She got into a heated argument with me once over her contention that rape was a systematic patriarchal tool that benefited all men, but then she would spend her spare time working for the Prison Legal Assistance Project (known as "PLAP") where one of her projects -- I kid you not -- was helping a local rapist to get out on parole.

Anyway, because the Wall Street Journal selected the most seemingly outrageous portions of the book in a way that made the self-deprecating tone less obvious, most of the reviewers seem to be completely missing out on Chua's real theme: that she ultimately had to transform her own ideas of motherhood when faced with a daughter who simply refused to comply. (One clue, from the book's very front cover: "This was supposed to be a story of how Chinese parents are better at raising kids than Western ones. But instead, it's about a bitter clash of cultures, a fleeting taste of glory, and how I was humbled by a thirteen-year old.")

One non-ironic line that I liked very much, and that partially explains my visceral hostility to most children's television, was this:
America seems to convey something to kids that Chinese culture doesn't. In Chinese culture, it just wouldn't occur to children to question, disobey, or talk back to their parents. In American culture, kids in books, TV shows, and movies constantly score points with their snappy backtalk and independent streaks. Typically, it's the parents who need to be taught a life lesson -- by their children.

Thursday, January 20, 2011

Should a Teacher's Value-Added Last Forever?

In the middle of a review of a recent Gates Foundation report on teacher value-added scores, economist Jesse Rothstein writes this:

An extensive literature makes clear that students assigned to high-value-added teachers see higher test scores in the year of that assignment, but that this benefit evaporates very quickly.

He cites, among other things, a Jacob/Lefgren/Sims paper arguing that:
the vast majority of the contemporary test score effect attributed to teacher value-added is transitory. This suggests that the teacher value-added literature overstates the effect of teachers on longrun learning and, therefore, the ability of policies that target teacher value-added to change ultimate student outcomes.
For some reason, the short-lived effect of teacher value-added scores is seen as somehow disproving their worth, and the worth of policies that would "target teacher value-added."

But to what other realm of human endeavor would such a standard be applied?

Consider another aspect of human development: physical fitness. Would anyone suggest that the value of vigorous exercise is somehow discredited because a vigorous exercise program has the most effect only in the year that it is actually followed, but its effects mostly disappear after the individual stops exercising? Of course not. Everyone who has ever exercised knows that if you stop for even a few months, you will lose much or most of your conditioning. Does that mean that vigorous exercise wasn't effective? No: it was effective as long as you kept exercising.

The same is true for medicines and vitamins. Would anyone find it important to observe that the effectiveness of medicines and vitamins disappears after you stop taking them for a year or more?

Or take weight loss. Would anyone be surprised to find that a healthy diet enabled people to lose weight only when they actually followed the diet, but didn't have permanent effects that allowed people to quit the diet and eat whatever they wanted therafter?

Shouldn't the same logic apply to teachers? A great teacher can have a powerful effect on your learning of a specific subject. If the effect of good teaching fades out over time, that may simply mean that teaching is like nearly everything else in life: it doesn't magically guarantee permanent improvement.

But it can still be important to figure out what good teachers are doing and who the good teachers are, just as it's important to know which weightlifting coaches produce Olympic champions and which ones are incompetent -- even if the effect of good teaching in either field fades out once an individual moves on and does other things.

Indeed, the fadeout of high value-added teachers may actually mean that it's all the more important to implement well-thought-out policies that encourage such teachers. Why would a 5th grade math teacher's effect fade out in 6th and 7th grade? Yes, it could be because the 5th grade teacher was artificially gaming her scores by teaching topics or problems that are completely idiosyncratic to the 5th grade math test in a particular state (although the blame really would lie with the state standards and tests in such a case). But it could well be because a great 5th grade math teacher was followed by mediocre 6th and 7th grade math teachers who failed to capitalize on the 5th grade teacher's accomplishments, and students' performance -- no surprise! -- wasn't magically and permanently raised.

In other words, the fadeout of high-value added teachers might just mean that we need to be finding the mediocre 6th and 7th grade teachers who are managing to erase their students' prior knowledge and achievement, and either helping those teachers to improve or else gently moving them into other professions for which they are better suited.

UPDATE: I'd apply the same logic to all of the other educational interventions whose effect fades out or disappears over time -- Head Start, class size reduction, rewarding students for reading books (Roland Fryer's experiment). In all these cases, why would anyone ever have expected a one-time treatment to have a permanent effect? Nothing about our bodies or minds works that way.

Labels:

Statistical Architecture

I liked this passage from a 2008 article by Gary King and Eleanor Neff Powell:
There are fields of study that have not yet been revolutionized by increasing quantification and modern statistics, but its an easy prediction that this will eventually happen given enough enterprising scholars, wherever it would be useful (and unfortunately, at other times too!). Certainly the opportunities for intellectual arbitrage are enormous.

To take one example, for clarity outside our field, consider architecture. By far, the most expensive decisions universities make are about buildings and their physical plant. Yet architecture as a field is composed primarily of engineers who keep buildings up and qualitative creative types who invent new designs: quantitative social scientists do not frequently get jobs in schools of design.

Imagine instead how much progress could be made by even simple data collection and straightforward statistical analysis. Some relevant questions, with associated explanatory variables might be: Do corridors or suites make the faculty and students produce and learn more? Does vertical circulation work as well as horizontal? Should we put faculty in close proximity to others working on the same projects or should we maximize interdisciplinary adjacencies? (Do graduate students learn more when they are prevented for lack of windows from seeing the outside world during the day?) And if the purpose of a university is roughly to maximize the number of units of knowledge created, disseminated, and preserved, then collecting measures would not be difficult, such as citation counts, the number of new faculty hired or degrees conferred, the quality of student placements upon graduation, etc. A little quantitative social analysis in architecture could go a long way in putting these most expensive decisions on a sound scientific footing.

Labels:

Monday, December 27, 2010

Pasi Sahlberg has an op-ed in the Boston Globe making the following claim:
What could the United States learn from the Finns? First, reconsider those policies that advocate choice and competition as the key drivers of educational improvement. None of the best-performing education systems relies primarily on them. Indeed, the Finnish experience shows that consistent focus on equity and cooperation — not choice and competition — can lead to an education system where all children learn well.
The weasel word (or term) in the above is "relies primarily," because it certainly isn't true that none of the best-performing education systems offer school choice. Many of them do.

Consider Hong Kong, the Netherlands, and Belgium, all of which are significantly above the OECD average for the most recent PISA scores in reading, math, and science. Hong Kong is at about the same level as Finland (and ahead in math); the Netherlands and Belgium are somewhat below Finland, but are still among the world's top performers.

These nations all have extensive school choice.

Netherlands: "Almost 70 percent of schools in the Netherlands are administered by private school boards, and all schools are government funded equally." (Source.)

Belgium: All schools in Belgium, including privately-operated schools, are publicly funded. (Source.)

Hong Kong:
Hong Kong’s educational system, although dominated by the Education Bureau, certainly does offer genuine choices. In fact, compared with many western countries, the range of choices is quite broad. Just to recap, there are:

* a surprisingly small number of purely government-funded and operated schools;
* a very large number of schools that are government-funded and supervised, but that are run by private organizations, mostly religious ones;
* an increasing number of what are essentially charter or magnet schools, e.g. the Direct Subsidy Scheme schools; and
* genuinely private schools that receive no government funding.
(Source.)
In addition, consider New Zealand, also one of the world's top performers on all subjects in PISA. Although it has a checkered history, New Zealand has most definitely had school choice, including funding for private schools.

Labels:

Tuesday, December 14, 2010

What Does the Ruling Against the Health Care Mandate Actually Accomplish?

A federal district judge in Virginia recently issued an order declaring unconstitutional that portion of Obama's health care plan that required individuals to purchase health insurance. This decision is unsurprisingly controversial, and has already been the subject of much commentary.

What I don't see anyone mentioning is the fact that the judge's ruling applies only to the specific plaintiff -- the Commonwealth of Virginia -- not to anyone else nationwide. Indeed, it's not clear to me that the judge's ruling would apply to any of the private citizens of Virginia.

Why do I say this?

First, the district court did not issue an injunction here that would purport to prevent the federal government from enforcing the statute elsewhere.

Second, it seems very questionable to me whether the district court would have had the power to issue such an injunction in the first place. As the Ninth Circuit has noted in a case involving the INS, district courts are not supposed to issue injunctions that protect non-plaintiffs, absent a class action:
We must vacate and remand, however, because the scope of the injunction is too broad. On remand, the injunction must be limited to apply only to the individual plaintiffs unless the district judge certifies a class of plaintiffs. National Center for Immigrants Rights, Inc. v. INS, 743 F.2d 1365, 1371 (9th Cir.1984). A federal court may issue an injunction if it has personal jurisdiction over the parties and subject matter jurisdiction over the claim; it may not attempt to determine the rights of persons not before the court. . . . The district court must, therefore, tailor the injunction to affect only those persons over which it has power.
In another case, the Ninth Circuit similarly held:
In addition to rescinding Meinhold's discharge, the district court permanently enjoined DOD from "discharging, changing [the] enlistment status of or denying enlistment to any person," from maintaining files, and from "taking any actions" against gay or lesbian servicemembers based on sexual orientation in the absence of sexual conduct which interferes with the military's mission. The Navy argues that even if the district court did not err on the constitutional issue, its nation-wide injunction cannot stand. We agree.

An injunction "should be no more burdensome to the defendant than necessary to provide complete relief to the plaintiffs." Califano v. Yamasaki, 442 U.S. 682, 702, 99 S.Ct. 2545, 2558, 61 L.Ed.2d 176 (1979); see also Bresgal v. Brock, 843 F.2d 1163, 1170-71 (9th Cir.1987). This is not a class action, and Meinhold sought only to have his discharge voided and to be reinstated. Effective relief can be obtained by directing the Navy not to apply its regulation to Meinhold . . . .
Third, federal district judges do not have the power to issue binding precedential orders, even within their own district. As the Third Circuit has noted:
First, it is clear that there is no such thing as "the law of the district." Even where the facts of a prior district court case are, for all practical purposes, the same as those presented to a different district court in the same district, the prior "resolution of those claims does not bar reconsideration by this Court of similar contentions. The doctrine of stare decisis does not compel one district court judge to follow the decision of another."
Given this lack of precedential power, it would be startling if a federal district judge could nonetheless issue an order preventing the federal government from enforcing a federal law as to anyone in the entire country. Put it this way: if a private citizen had filed a lawsuit against the federal government in the same district court (the Eastern District of Virginia), there could be another ruling tomorrow that went the opposite way: upholding the individual mandate. Whatever federal district judge happens to be the first to issue a ruling shouldn't have the effective power to make all future rulings by other courts a nullity.





Sarah LaFon