Go to the Aspen Ideas Festival – or to any similar confab of affluent elites gathered to solve the problems of the world in luxurious, remote hamlets – and you can be sure that a dominant theme will be a lament for the vanishing political center.

Where, panel after panel will ask, are the wise moderates, able to seek compromise and rise above partisanship in pursuit of the public good? America’s biggest problems, and its inability to tackle them head-on, will usually be cited as the consequence of this lack of a sensible middle.

Most of the wealthy and well-positioned people in the rooms where these sorts of discussions are conducted see themselves as members of that sadly disempowered middle, so reflections along these lines are generally well received.

But the problem with this approach is its implicit assumption that politics, or at least policy, is a win-win game. Policies that serve the collective good are out there to be found, if only we publicly minded moderates were in charge.

But what if it isn’t just the political battle at the voting booth that is partisan, but the policies themselves, and their outcomes, too?

Take healthcare. The fierce battle over the U.S. healthcare overhaul is framed by both sides as an argument over which type of system would work better “for America.”

But what you think “works” depends very much on who you are. If you are uninsured, or fear you could be, expanding coverage is obviously a very good thing. But if you already have Cadillac coverage through your employer, and are confident you will keep it, then paying for more people to get on board might not seem like such a good idea.

A favorite issue for believers in the moderate middle is the budget deficit. It is held up as an example of the rabid partisanship and short-termism that threatens to destroy the U.S. economy. The usual consensus on how to avert impending doom is to put the wise moderates in charge and let them hammer out a compromise deal to both raise taxes and cut spending. That’s fair enough – and is almost certainly the eventual path the United States must take.

But what’s really striking about discussions of the budget deficit by the moderate middle is how overwhelmingly the issue itself has come to dominate the public debate about the economy, especially when those wise, nonpartisan people get together. A study last year by the National Journal found that the number of articles in five major American newspapers about the budget deficit eclipsed the number of articles about unemployment.

What’s especially remarkable about that imbalance is that while the deficit is certainly an issue the United States needs to tackle in the medium term, at the moment interest rates for 10-year bonds are near historical lows. The markets may not be wise, but they are nonpartisan, and as far as they are concerned there is no urgent fiscal crisis in the United States.

Meanwhile, unemployment remains above 8 percent and is having a dire and long-term impact on the people without jobs and on their families. That’s why Alan B. Krueger, an economist at Princeton who is the head of President Barack Obama’s Council of Economic Advisers, focused his talk at Aspen (which I moderated) on “the middle-class jobs deficit.”

“What I wanted to do in using that term is to highlight that this is a deficit, just like our fiscal deficit,” Krueger said. “We need to raise the jobs deficit to the same level as the fiscal deficit, if not above it.”

The U.S. jobs problem is even worse than the headline unemployment numbers suggest.

Heidi Ewing, a filmmaker whose documentary about Detroit was aired at Aspen and who was in the audience for Krueger’s talk, identified what is going on: “What we’ve noticed, from workers who got rehired recently, is they took a 50 percent wage cut during the bailout. So they’re coming back at $11 to $14 an hour. Do we really have to accept that this group of people is no longer the middle class we had in our minds before, but just the working poor?”

Thanks to what economists call the “polarization” of the labor market, if you are lucky enough to have a job in the United States you probably fall in either the top or the bottom of income distribution, but not in the middle.

Krueger touched on one of the most worrying consequences of this economically divided country: It will get worse over time.

Americans still imagine their country to be the land of opportunity, but the numbers tell a different story. Your parents’ income correlates more closely with your chance of finishing college than your SAT scores do – class matters more than how you do in class.

Which brings us back to the affluent moderates who congregate at conferences like the Aspen Ideas Festival. Whether they realize it or not, these sensible centrists are guided by their own self-interests, too. It just so happens that deficits are dangerous for them – they could bring higher taxes – while the jobs deficit has little impact on them or their families.

Today, the U.S. political center lives at the top. We shouldn’t be surprised by the shrieks of those who have been forced to the economic bottom and have chosen to move to the political fringes.