Felix Salmon

The Nanex HFT chart I posted last week went viral, becoming by far my most popular post of the year; I even did a version of it for Buzzfeed. In the comments to that post, Kid Dynamite defended high-frequency trading by saying that spreads have tightened in substantially for everyone as a result of HFT. But neither of us really had the numbers — until now.

The NYT’s Nathaniel Popper, today, runs this pair of charts, which basically tells us everything we need to know. The main thing you need to notice is that the x-axis is the same on both charts, running from 2000 to the present day; my HFT chart from Nanex ran from 2007 to the beginning of 2012.

What’s clear from the top chart in the NYT (and from my Nanex chart) is that the explosion in HFT took place from 2007 onwards. And what’s clear from the bottom chart in the NYT is that benefits to small investors more or less stopped at that point.

First, let’s be clear about what these charts are showing. HFT is maybe a bit misnamed, since what we’re seeing here is two separate eras. From 2000 to 2006, trading got faster and cheaper. From 2007 to date, trading itself hasn’t actually risen much, or got faster. the huge spikes are in quotes, rather than trades, and it’s not uncommon for certain stocks to see more than a million quotes over the course of a single day, even when they are only traded a couple of dozen times.

You know the track cycling at the Olympics, where the beginning of the race is entirely tactical, and the trick is not to go fast but to actually position yourself behind the other person? HFT is a bit like that: the algorithms are constantly putting up quotes and then pulling them down again, in the knowledge that there’s very little chance they will be hit and traded on. The quotes aren’t genuine attempts to trade: instead, they’re an attempt to distract the rest of the market while the algo quietly trades elsewhere.

As such, the vast number of quotes in the market is not a genuine sign of liquidity, since there really isn’t money to back them all up. Instead, it’s just noise. But don’t take my word for it. Here’s Larry Tabb, the CEO of Tabb Group, and a man who knows vastly more about HFT than just about anybody else:

Given the events of the past six months, the SEC should think hard about the market structure it has created, and do its utmost to rein it in. While the SEC can’t stop computers from getting faster, there is no reason it can’t reduce price and venue fragmentation, which should slow the market down, reduce message traffic and lower technology burdens.

Until we can safely manage complex and massive message streams in microseconds, fragmentation is making one of the greatest financial markets of all time about as stable as a McLaren with its RPMs buried in the red.

HFT causes stock-market instability, and stock-market instability is a major systemic risk. No one’s benefitting from the fact that the entire market could blow up at any second. So why isn’t anybody putting a stop to it?

Dennis Kelleher of Better Markets has responded to my post in which I said, inter alia, that he was wrong about high-frequency trading. He, of course, says that I’m wrong — indeed, that I’m “over the top and just plain wrong in many ways”, and that the post is “self-discrediting”. Blogfight! So, fair warning: this post is my response to his response to my post; if you’re not into that kind of thing I fully understand, and you’re probably much more grown-up than either of us. Anyway.

First, Felix totally overlooks the fact that some of the biggest banks in the world knowingly committed multiple very serious crimes by rigging the Libor rate.

It’s true I didn’t dwell on this, because it really wasn’t the subject of my post. My point was that whenever something scandalous or unacceptable happens in the financial markets, it’s not enough that the activity is scandalous or unacceptable: the financial press also feels the need to demonstrate that the little guy was being ripped off somehow. Even if he wasn’t. In this case, I’m perfectly happy to agree with Kelleher that rigging Libor was a very serious crime.

Kelleher accuses me of ignoring other things, too, like the difference between the two separate parts of the Libor-rigging scandal. Again, yes, I didn’t mention that. I also didn’t mention Standard Chartered, or HSBC money-laundering, or, for that matter, the Olympics badminton scandal. Kelleher has made it his life’s work to rail against such things, so maybe he feels that I should mention them in every post I write. But I can hardly be wrong about something if I didn’t even mention it.

He does, however, say that I’m “dead wrong that no one was harmed by the banks rigging the Libor rate”. This is a bit of a nasty accusation, because if I’d said that no one was harmed by the Libor rigging, then indeed he would be quite right to call me wrong. But I never said anything like that. He also says that I don’t understand interest-rate swaps, and proceeds to give a perfectly accurate explanation of how they work. And again, his explanation doesn’t contradict anything I said. But I do think he misses my point, so let me try again.

Kelleher uses an example of a municipality which has entered into an interest rate swap and is paying a fixed rate while receiving a floating rate linked to Libor. Such swaps are designed to protect borrowers from rising interest rates; the flipside of the deal is that if rates fall, then the borrower will end up losing money. And as it happened, rates fell, and the borrowers ended up losing money.

Now here’s the thing: the municipalities didn’t insist on linking the interest-rate swap to Libor because their borrowing costs are particularly bank-like. They just used Libor because it was the market standard, a proxy for interest rates more generally. The Libor scandal — and, yes, it is a scandal — is that the banks ended up printing a rate for Libor which was closer to prevailing interest rates than it should have been. Because Libor is tied to the interest rate on unsecured bank debt, it can actually rise when interest rates are falling, if the credit spread on bank debt rises fast enough. From the point of view of borrowers engaging in interest-rate swaps, that’s a bug, not a feature. What they want is a simple proxy for interest rates; they don’t want a proxy for interest-rates-plus-financial-sector-credit-spreads.

So Kelleher is right, in a narrow sense, when he says that if you were receiving floating-rate interest payments linked to Libor, then you got less money than you should have got. Because according to the contract, your payments should have included that extra bank-credit-spread component, on top of the interest-rate component. But my point is that no one ever entered into an interest-rate swap because they were making a bet on bank credit spreads rising. As a result, the losses here are losses of windfall, unexpected revenues. And of course there are just as many borrowers who entered into floating-to-fixed interest-rate swaps: they ended up winning just as much as the fixed-to-floating borrowers ended up losing.

It’s worth taking a step backwards here. In the grand scheme of things, borrowers gained rather than lost from the Libor manipulation, because it meant that they paid less interest on floating-rate debt. The real losers here are investors who bought floating-rate debt, and who should have been paid more than they were. My point is that if you’ve found someone claiming to have lost money as a result of the Libor manipulation, and they’re a borrower rather than an investor, you’re pretty much scraping the barrel. The Libor scandal is scandalous for many reasons, first and foremost that it involved banks lying in order to manipulate a hugely important interest rate. You don’t need to show borrowers losing money in order for there to be a scandal here: there would be a huge scandal even if no borrowers lost any money at all.

Kelleher then moves on to the main subject of my post, which was high-frequency trading. I said he was wrong when he said on a TV show we were on that shops like Knight rip off small investors. He replies:

Mr. Kelleher distinguished between high speed trading (really high speed market making) and predatory high frequency trading (HFT). Maybe not the most precise way to talk about these activities, but not too far off the mark for a general audience. It was the later practice not the former that Mr. Kelleher said rips off small investors, frequently referred to in the market as dumb money. (Not mentioned was that, because shops like Knight pay for order flow from retail brokers and pick off what they want, there are fewer natural buyers and sellers in the market and only professional or toxic retail flow actually gets to the market.)

OK, let’s make a distinction between high-speed market-making, on the one hand, and HFT, on the other. If you’re making that distinction, then Knight absolutely falls into the former category: it’s one of the helpful market-makers, rather than one of the predatory algobots. This part of the show hasn’t made it onto the internet, but I can assure you that Kelleher never explained that his distinction, at the margin, actually makes Knight look better rather than worse.

But in any case, the high-frequency algobots don’t rip off small investors, because the two never come into contact with each other. If a small investor puts in a stock trade, it ends up being filled by Knight, or one of the other high-speed market-makers. The algobots are whale-hunting: they’re looking for big orders from institutional investors, which they can game and front-run and otherwise prey upon. If small investors ever found themselves naked in the open oceans of the markets, the same thing might happen to them, but they don’t: they’re protected from those waters by companies like Knight, which will give them exactly what they want at the national best bid/offer price.

You’d think that Kelleher, having made the distinction, would be happy that small investors don’t end up being picked off by predators, but he’s not: he reckons that because they’re not out in the open ocean, that means “there are fewer natural buyers and sellers in the market”. Well, you can’t have it both ways. And frankly if retail investors did return to the market, it wouldn’t help matters: there wouldn’t be more volume or more liquidity or any visible positive effect.

So why did Kelleher even make his distinction in the first place? Just so that he could then come out and say that “HFT is a liquidity taker, not a liquidity provider”. In order to say that, he needs to exclude high-speed market-makers like Knight, who clearly do provide liquidity to retail investors. When I said that high-frequency shops provide liquidity to the market, I was very much talking about Knight, and I can assure Kelleher that everybody who was watching TV on Monday night thought that he was talking about Knight as well. After all, it’s Knight that’s in the news right now.

Finally, Kelleher pushes back against my “anti-regulation stance”, which is quite hilarious; he also informs his readers that “Felix also sees HFT as nothing but a force for good.” Maybe he didn’t see my post on Monday, where I talked about how HFT is “quite literally out of control”. I concluded that post by saying that “the potential cost is huge; the short-term benefits are minuscule. Let’s give HFT the funeral it deserves.” So obviously I don’t consider HFT to be “nothing but a force for good”.

The fact is, however, that I don’t need to go back to Monday’s post to demonstrate my anti-HFT bona fides. In the very post that Kelleher’s responding to, I write this:

I do think that the amount of HFT we’re seeing today is excessive, and I do think that we’ve created a large-scale, highly-complex system which is out of anybody’s control and therefore extremely dangerous.

Kelleher, then, is a man who, immediately after reading those words, can turn around and describe me as someone who sees HFT as nothing but a force for good. It’s very hard to know how to respond to such a person, but I guess that does at least explain why he thinks I said so many things I never said. He might think he’s responding to me, but in fact he’s just creating a straw man and putting my name on it. Which, frankly, is a little bit annoying.

One of the problems with financial journalism is its rather kludgy attempts to appeal to a general audience. If something bad happens, for instance, it has to be presented as being bad for the little guy. This was a huge problem with the Libor scandal, since anybody with a mortgage or other loan tied to Libor ended up saving money as a result of it being marked too low.

But don’t underestimate the imagination of the financial press. For instance, what if there was a New York county which put on Libor-linked interest rate swaps to hedge its bond issuance? In that case, if Libor was understated, then the hedges would have paid out less money than they should have done — and presto, the Libor scandal is directly responsible for municipal layoffs and cuts in “programs for some of the needy”.

This is all a bit silly. The Libor understatements actually brought it closer to prevailing interest rates; the fudging basically just served to minimize the degree to which the unsecured credit risk of international banks was embedded in the rate. And in any case, the whole point of a hedge is that it offsets risks elsewhere: it’s intellectually dishonest to talk about losses on the hedge without talking about the lower rates that the municipality was paying on its debt program as a whole.

We’re seeing the same thing with the fiasco at Knight Capital, where a highly-sophisticated high-frequency stock-trading shop lost an enormous amount of money in a very small amount of time, and small investors lost absolutely nothing. On the grounds that we can’t present this as news without somehow determining that it’s bad for the little guy, it took no time at all for grandees to weigh in explaining why this really was bad for the little guy after all, and/or demonstrates the need for strong new regulation, in order to protect, um, someone, or something. It’s never really spelled out.

The markets version of the Confidence Fairy certainly gets invoked: Arthur Levitt, for instance, said that recent events “have scared the hell out of investors”. And Dennis Kelleher of Better Markets goes even further: I was on a TV show with him last night, where he tried to make a distinction between “high-frequency trading” and “high-speed trading”, and said that shops like Knight rip off small investors. He’s wrong about that: they absolutely do not. Yes, Knight and its ilk pay good money for the opportunity to take the other side of the trade from small investors. But those investors always get filled at NBBO — the best possible price in the market — and they do so immediately. Small retail investors literally get the best execution in the markets right now, thanks to Knight and other HFTs. And those investors want companies like Knight to compete with each other to fill their trades as quickly and cheaply as possible. If Knight loses money while doing so, that’s no skin off their nose.

So Andrew Ross Sorkin is right to treat such pronouncements with skepticism. The argument that “investors are worried about high-frequency trading, therefore they’re leaving the market, therefore stocks are lower than they would otherwise be, therefore we all have less wealth than we should have” just doesn’t hold water at all. Sorkin has his own theories for why the stock market doesn’t seem to be particularly popular these days, which are better ones, but the fact is — he doesn’t mention this — that the market is approaching new post-crash highs, and that if investors follow standard personal-finance advice and rebalance their portfolios every so often, they should probably be rotating out of stocks right now, just to keep their equity holdings at the desired percentage of their total holdings.

The calls for more regulation are a bit silly, too. Bloomberg View says that “if any good comes out of the Knight episode, it will be a commitment by Wall Street’s trading firms to help regulators design systems that can track lightning-speed transactions” — but regulators will always be one step behind state-of-the-art traders, and shouldn’t try to get into some kind of arms race with them. More regulation of HFT is not going to do any good, especially since no one can agree on the goals the increased regulation would be trying to achieve. If what we want is less HFT, then a financial-transactions tax, rather than a regulatory response, is the way to go.

I do think that the amount of HFT we’re seeing today is excessive, and I do think that we’ve created a large-scale, highly-complex system which is out of anybody’s control and therefore extremely dangerous. Making it simpler and dumber would be a good thing. But you can’t do that with regulation. And let’s not kid ourselves that up until now, small investors have been damaged by HFT. They haven’t. The reasons to rein it in are systemic; they’re nothing to do with individuals being ripped off. Sad as that might be for the financial press.

>

This astonishing GIF comes from Nanex, and shows the amount of high-frequency trading in the stock market from January 2007 to January 2012. (Which means that the Knightmare craziness of last week is not included.)

The various colors, as identified in the legend on the right, are all the different US stock exchanges. You might think there are only two stock exchanges in the US, but you’d be wrong: there are only two exchanges where stocks are listed. There are many, many more exchanges where stocks are traded.

What we see here is relatively low levels of high-frequency trading through all of 2007. Then, in 2008, a pattern starts to emerge: a big spike right at the close, at 4pm, which is soon mirrored by another spike at the open. This is the era of traders going off to play golf in the middle of the day, because nothing interesting happens except at the beginning and the end of the trading day. But it doesn’t last long.

By the end of 2008, odd spikes in trading activity show up in the middle of the day, and of course there’s a huge flurry of activity around the time of the financial crisis. And then, after that, things just become completely unpredictable. There’s still a morning spike for most of 2009, but even that goes away eventually, to be replaced with sheer noise. Sometimes, like at the end of 2010, high-frequency trading activity is very low. At other times, like at the end of 2011, it’s incredibly high. Intraday spikes can happen at any time of day, and volumes can surge and fall back in pretty much random fashion.

It’s certainly fair to say that if you take a long, five-year view, then you can see a clear rise in trading activity. But it’s also fair to say that there’s something quite literally out of control going on here. Just as the quants at Knight found themselves unable to turn off their machines for 30 long minutes last week, the HFT world in aggregate seemingly has a mind of its own when it comes to trading patterns. Or, to put it another way, if there’s a pattern here, it’s one incomprehensible to human minds.

Back in 2007, I wasn’t a fan of a financial-transactions tax; today, I am. And this chart shows better than anything why my opinion has changed. The stock market is clearly more dangerous than it was in 2007, with much greater tail risk; meanwhile, in return for facing that danger, society as a whole has received precious little utility. Are spreads a tiny bit tighter than they might be otherwise? Perhaps. But that has no effect on stock-market returns for long-term or even medium-term investors.

The stock market today is a war zone, where algobots fight each other over pennies, millions of times a second. Sometimes, the casualties are merely companies like Knight, and few people have much sympathy for them. But inevitably, at some point in the future, significant losses will end up being borne by investors with no direct connection to the HFT world, which is so complex that its potential systemic repercussions are literally unknowable. The potential cost is huge; the short-term benefits are minuscule. Let’s give HFT the funeral it deserves.

It’s rogue algo day in the markets today, which sounds rather as though the plot to The Fear Index has just become real, especially since the firm at the center of it all is called The Dark Knight, or something like that. At heart, however, is something entirely unsurprising: weird things happen when you get deep into the weeds of high-frequency trading, a highly-complex system which breaks in entirely unpredictable ways.

In fact, it’s weirder than that: HFT doesn’t just break in unpredictable ways, but works in unpredictable ways, too. Barry Ritholtz has an excerpt from Frank Partnoy’s new book, Wait, all about an HFT shop in California called UNX:

By the end of 2007, UNX was at the top of the list. The Plexus Group rankings of the leading trading firms hadn’t even mentioned UNX a year earlier. Now UNX was at the top, in nearly every relevant category…

Harrison understood that geography was causing delay: even at the speed of light, it was taking UNX’s orders a relatively long time to move across the country.

He studied UNX’s transaction speeds and noticed that it took about sixty-five milliseconds from when trades entered UNX’s computers until they were completed in New York. About half of that time was coast-to-coast travel. Closer meant faster. And faster meant better. So Harrison packed up UNX’s computers, shipped them to New York, and then turned them back on.

This is where the story gets, as Harrison put it, weird. He explains: “When we got everything set up in New York, the trades were faster, just as we expected. We saved thirty-five milliseconds by moving everything east. All of that went exactly as we planned.”

“But all of a sudden, our trading costs were higher. We were paying more to buy shares, and we were receiving less when we sold. The trading speeds were faster, but the execution was inferior. It was one of the strangest things I’d ever seen. We spent a huge amount of time confirming the results, testing and testing, but they held across the board. No matter what we tried, faster was worse.”

“Finally, we gave up and decided to slow down our computers a little bit, just to see what would happen. We delayed their operation. And when we went back up to sixty-five milliseconds of trade time, we went back to the top of the charts. It was really bizarre.”

Partnoy has a theory about what’s going on here — something about “optimizing delay”. But that sounds to me more like ex-post rationalization than anything which makes much intuitive sense. The fact is that a lot of the stock-trading world, at this point, especially when it comes to high-frequency algobots, operates on a level which is simply beyond intuition. Pattern-detecting algos detect patterns that the human mind can’t see, and they learn from them, and they trade on them, and some of them work, and some of them don’t, and no one really has a clue why. What’s more, as we saw today, the degree of control that humans have over these algos is much more tenuous than the HFT shops would have you believe. Knight is as good as it gets, in the HFT space: if they can blow up this badly, anybody can.

I frankly find it very hard to believe that all this trading is creating real value, as opposed to simply creating ever-larger tail risk. Bid-offer spreads are low, and there’s a lot of liquidity available on a day-to-day basis, but it’s very hard to put a dollar value on that liquidity. Let’s say we implemented a financial-transactions tax, or moved to a stock market where there was a mini-auction for every stock once per second: I doubt that would cause measurable harm to investors (as opposed to traders). And it would surely make the stock market as a whole less brittle.

It’s worth recalling what Dave Cliff and Linda Northrop wrote last year:

The concerns expressed here about modern computer-based trading in the global financial markets are really just a detailed instance of a more general story: it seems likely, or at least plausible, that major advanced economies are becoming increasingly reliant on large-scale complex IT systems (LSCITS): the complexity of these LSCITS is increasing rapidly; their socio-economic criticality is also increasing rapidly; our ability to manage them, and to predict their failures before it is too late, may not be keeping up. That is, we may be becoming critically dependent on LSCITS that we simply do not understand and hence are simply not capable of managing.

Today’s actions, I think, demonstrate that we’ve already reached that point. The question is whether we have any desire to do anything about it. And for the time being, the answer seems to be that no, we don’t.

In February 2009, Deutsche Bank announced that its Credit Trading desk had managed to lose €3.4 billion in the fourth quarter of 2008, with €1 billion of those losses directly attributable to the bank’s prop desk.

The losses in the Credit Proprietary Trading business were mainly driven by losses on long positions in the U.S. Automotive sector and by falling corporate and convertible bond prices and basis widening versus the Credit Default Swaps (CDS) established to hedge them.

In English, Deutsche Bank had put on a basis trade: it owned credit instruments, like bonds, and it also owned credit default swaps designed to hedge against those loans. And then the trade blew up.

The Deutsche trader responsible for the monster losses was Boaz Weinstein, who eventually left the bank to start his own hedge fund, Saba Capital. His first job, obviously, was to make sure he didn’t blow up a second time. But his second job, it seems, was to use his experience at Deutsche to be able to notice when someone else was about to blow up on a massive basis trade. In this case, JP Morgan.

Go back to early February, long before the articles about the “London Whale” came out in Bloomberg and the WSJ, and you’ll find Weinstein revealing his biggest trade at the Harbor Investment Conference:

The derivatives trader and legendary hedge fund manager said his trade idea is to buy Investment Grade Series 9 10-Year Index CDS (maturing on 12/20/2017).

“They are very attractive,” he explained adding that they can be bought at a “very good discount.”

At the time, Weinstein didn’t know — or necessarily even suspect — that his big trade would involve a zero-sum bet with one of the biggest hedge funds in the world, JP Morgan’s Chief Investment Office. But over time, as he bought more and more protection but the price stubbornly refused to rise, he began to learn just how big the other size of the trade was. Whale big.

Tracy Alloway and Sam Jones have pieced together the best account yet of what exactly JP Morgan was up to. Yet again, it was a basis trade, although this one was horribly complex even by basis-trade standards. Essentially, that CDX.NA.IG.9 position was a second-order hedge, designed to offset volatility in JP Morgan’s first-order hedge, which was designed to offset credit risk in the rest of the bank’s portfolio.

The first-order hedge itself doesn’t make a great deal of sense — Iksil seems to have bought “tranches” of CDS indices, which would pay off if some (but not all) credits suddenly got into trouble. For a bank which had broad economic exposure to European meltdown and/or a US double dip, that seems like a pretty narrow hedge.

But if the first-order hedge is weird, the second-order hedge is downright scary. Do you remember the notorious Howie Hubler trade at Morgan Stanley, where he made a smart bet against dangerous subprime securities, but then put on a much larger “hedge” which ended up costing him $9 billion? Iksil’s trade seems a bit like that:

Because of the mechanics of the trade, in order to achieve a “market neutral” position, whereby JPMorgan hedged the bet against volatility as best it could and offset the cost of its short positions, the bank had to sell far more units of cheap protection on the IG.9 as a whole than it bought on short, more expensive tranches.

Inevitably things started to go wrong. There are two things you can do when something starts to go wrong in the markets. You can unwind your position at a loss. Or you can try to fix it. Iksil, and Drew, chose the latter:

The two legs of JPMorgan’s trade did not move according to the relationship the bank had expected, meaning the position became imperfectly hedged. Like many credit models before it, JPMorgan appeared to misjudge correlation – one of the hardest market phenomena to accurately capture in mathematics.

In order to try and stay risk neutral, the dynamic hedge required even more long protection to be sold. The bank continued to write swaps on the IG.9, causing a pricing distortion that was spotted by more and more hedge funds seeking profit.

The rest, pretty much, is history.

Iksil, we’re told, is going to leave JP Morgan, while taking his own sweet time doing so: “although a spokeswoman for the bank said Mr. Iksil is still employed, he is no longer trading on behalf on the bank and is expected to be gone by the end of the year”. I’m sure he’ll use the intervening months to feel out his chances of being able to raise a few billion dollars for a hedge fund of his own, and weigh them up against simply joining a fund like Saba. Iksil’s now learned a $2 billion lesson — and as Boaz Weinstein can attest, once learned, those lessons can be surprisingly valuable.

We’re still not much the wiser on exactly how the London Whale managed to lose $2 billion this quarter, but I think Matt Levine has the smartest take. (This is why the blogosphere is so great: it’s full of people who used to do this kind of thing for a living, rather than just people who write about people who do this for a living.)

The key thing to note here is that while the monster hit to the P&L is what got all the headlines, the real problem here lay with JP Morgan’s risk models. A hint of far out of whack they are is given in the difference between the bank’s earnings release, which showed $67 million of value-at-risk in the Whale’s division in the first quarter, and the new SEC filing, which showed that number as actually being $129 million. Here’s Levine:

This was attributed to modeling changes made over the last year, and someone asked on the call “why did you change the VaR model?,” but I’m not convinced that’s exactly the right question. This, I suspect, is not an issue of a thing called a “VaR model” that sits in a central location and spits out numbers for regulators and 10-Qs; rather, this looks like the CIO’s trading desk modelling the actual P&L and risks of the trade wildly wrong. That seems to me like the simplest way to lose a billion dollars without noticing it.

I’d put this another way. JP Morgan’s Bruno Iksil, it seems, managed to find an incredibly profitable way of hedging the bank’s positions. Like any other economically rational actor, when he saw a lot of dollar bills lying on the sidewalk, he decided to pick them up. But in Iksil’s highly-complex world, a dollar bill isn’t really a dollar bill. Instead, it’s the output of a model. And if a trader can’t trust his model, he’s flying blind.

The problem is that pretty much by definition, it’s impossible to model model risk. We now know that Iksil’s model was deeply flawed. And indeed the minute that the rest of the world found out about his positions, they didn’t really pass the smell test: it’s very hard to see how writing an enormous amount of protection on an off-the-run CDX index would hedge anything much.

This is where grown-ups like Jamie Dimon are meant to step in. If they see billions of dollars in super-senior mortgage exposure, or in off-the-run CDX exposure, they’re meant to say “I know that your highfalutin’ models say that these exposures are risk free, but I don’t understand how this isn’t risky, so go unwind this trade”. Dimon has historically been very good at that — very good at refusing to simply trust that superstar traders earning eight-figure bonuses are doing nothing that might blow up in their faces. In this case, however, for some reason, he had blind faith in Iksil — and in Iksil’s models, which proved to be very faulty.

A modern trading desk is a bit like a high-tech airplane: nearly all of the time, you’re better off trusting your instruments than trusting your gut. But at the same time, if your instruments are broken, then trusting them can lead you to fly straight into the ocean.

This is why Basel I turned out to be much more robust than Basel II. Your sophisticated platform needs to be built on a foundation of dumb rules: simple limits on how big any one position can get, on how much exposure you can have to any one counterparty, or in general on any trade which is based on the hypothesis that your desk is smarter than anybody else on Wall Street.

Those kind of rules won’t prevent all blow-ups, of course, but they’ll help. They would have prevented this one, and they would have put an end to Jon Corzine’s disastrous MF Global trades, as well.

The problem is that traders hate dumb rules, because they cap the amount of money they can make. And traders have enormous power at investment banks these days, because they make the lion’s share of the profits. That’s why it’s important that the CEO of an investment bank not be a trader. And certainly it’s crucial that the CEO shouldn’t have his own trading account and buy and sell from his Blackberry during meetings, as Corzine did. That’s just a recipe for disaster.

Kara Swisher says that Twitter might start selling access to its “firehose” — the full stream of all public tweets from its tens of millions of users — to Google and Microsoft. Such companies, she says, might be willing to pay “several million dollars” for such a product.

Which raises the obvious question: if the Twitter firehose is worth millions to a search engine, how much would it be worth to algo traders and data miners? And how much of a premium would they be willing to pay to get that information a few milliseconds before anybody else? Indeed, would they be willing to pay Twitter a huge amount of money just for the privilege of hosting its servers in a the same location as their own proprietary stock-trading black boxes?

There’s been a lot of speculation of late about how on earth Twitter could be worth $1 billion. Maybe this is it! After all, the stock market, like Twitter, is basically a reflection of real-time sentiment. If you could somehow mine Twitter to isolate changes in sentiment, that could be worth billions.

Equity Private has a stinging and hilarious response to my post about the Zero Hedgies, which accused them of being day-traders:

It is not completely clear what “day trader” means in this context (it is possible this refers to individuals who manage their own portfolios rather than dump their assets into long-only 401k funds like smart, patriotic investors are supposed to) but I have reason to believe that they are generally considered “losers” and otherwise unsuitable degenerates. This appears to be connected to the dot-com crash… I have, therefore, undertaken an anecdotal analysis of our audience to pinpoint other weak points and pools that may contain high concentrations of short-sellers, precious metal devotees, former index fund investors, “retail investors” (I have used 5 definitions of this term to be expansive) gypsies, jews, homosexuals and former members of the band “Air Supply.” Obviously, if word leaked out of these associations our readership would plummet and we may face regulatory sanction or even prison.

She’s right: commenters are not necessarily representative of a site’s overall readership, and I’m sure that there are many smart and important finance and regulatory types who read ZH. A large part of the blog’s function is disintermediation: it comprises market participants talking directly to each other without being filtered by professional (or even amateur) journalists.

But if you follow this particular thread too far, you end up with an even more unsettling conclusion than my original one. Far from reflecting the conspiracy-minded and often-disjointed ramblings of harmful-only-to-themselves retail day-traders, could ZH actually be holding up a mirror to what the market’s really like, once you strip away the artificial polish of the PR departments and the urbane investment-banking types? Are finance types, in general, much more Howard Lindzon than Robert Rubin?

That’s what the wisdom-of-crowds hypothesis says: that if you take a million idiots, all trading with and against each other, yes you’ll get occasional mass delusions, and bubbles and busts, and ad-hoc groupings of vaguely like-minded individuals, like ZH. But somehow, in aggregate, those million idiots will be more right, more often, than any regulatory panjandrum or media pooh-bah. And anybody who’s spent any time with bankers and buy-siders will tell you that there’s precious little correlation between intelligence, on the one hand, and success in the markets, on the other. Yes, there are smart people who make a lot of money, but for every one of those I can show you two equally-smart people who have lost just as much.

It’s kinda ironic that there’s so much bad blood between ZH and CNBC, because in many ways both of them are very good at unveiling the market’s id. Look beneath the pat explanations of the daily market reports (“markets rose on optimism that oil prices might” etc etc), and what you see is a roiling, chaotic mess. It’s something which is almost completely invisible to readers of the Wall Street Journal, while occasional visitors to the Yahoo message boards can be forgiven for thinking that the only people who frequent such places can’t possibly be representative of the market as a whole. But with the advent of ZH and CNBC, it’s getting harder to escape the conclusion that this really is what the market is like, and the sanitized version found in middlebrow publications and mutual-fund reports is at heart a fiction.

I’m somewhere in the middle. On an intraday timescale, I think that noise traders really do set prices and move markets. If you look at markets over a period of months, however, their movements are much more a function of macroeconomic activity, global capital flows, and long-term decisions made by large institutional investors. These things are almost invisible on a day-to-day basis: they’re like gravity, which can happily be ignored at the quantum level, but which is the only thing that matters when you scale up a lot.

So ZH is probably a pretty accurate representation of many people who pay close attention to what the markets are doing on a minute-to-minute or even day-to-day basis. Which is as good a reason as any not to do that.

The existence of arbitrage, and arbitrageurs, is a necessary precondition for having a reasonably efficient market. Arbitrage allows the law of one price to become roughly true, and in turn belief in the law of one price is the central faith of any arbitrageur, who will pick up on price discrepancies safe in the knowledge that sooner rather than later the law will turn those trades into profits.

The problem is that the law of one price is not some kind of physical law with replicable effects, and sometimes it disappears entirely. So this kind of thing is actually true of all arbitrage:

DLC arbitrage is characterized by substantial idiosyncratic return volatility and a high incidence of large negative returns.

DLC (dual-listed company) arbitrage — where you look at the share price of the same company in different countries, and bet that they’ll converge — is one of the purest forms of arbitrage there is. But it’s not surprise to learn that it comes with “a high incidence of large negative returns”: any arbitrage strategy is ultimately a game of picking up nickels in front of a steamroller. Unless you have unlimited liquidity and never need to worry about margin calls, the market is likely to move against you just until you give up, at which time it will snap back to where you would have made a huge profit. Just ask the guys at LTCM, or the stat-arb hedgies who blew up in 2007.

This is why only the biggest and most liquid companies tend to try their hands at arbitrage: it’s very much a don’t-try-this-at-home strategy. Even if you’re convinced that the trade is risk-free, it really isn’t.

Aaron Lucchetti has a 2,000-word front-page WSJ story today which appears online under a “management” heading and with the headline “Wall Street’s B-List Firms Trade on Bigger Rivals’ Woes”. Really, however, it’s all about the obscene displays of wealth being perpetrated by Steven Schonfeld, the new poster boy for conspicuous consumption.

Schonfeld’s new $90 million house, on Whitney Lane in Old Westbury, Long Island, not only has “a poolside cabana designed to look like the Cove Atlantis resort in the Bahamas” but also sports a 9-hole golf course which is off-limits to anybody if Schonfeld isn’t at home:

“It’s not a private golf course,” he explains. “It’s a personal golf course.”

And then there’s this:

At one dinner with traders, he said that anyone who looked at the menu for more than 90 seconds was in the wrong business…

At high-end restaurants, Mr. Schonfeld has been known to order one of everything on the menu, with his party leaving much of the food uneaten.

Schonfeld got into a spot of bother (including $1.1 million in fines from the NYSE) when he tried to be a broker, so now he’s basically just a trader, hiring laid-off employees from big Wall Street firms and seeding them with his own money. Or, as Forbes put it in 2005,

Schonfeld oversees a harem of semi-independent traders who use his equipment, his software and his capital, sharing profits with him and paying him trading commissions.

It’s possible that he’s exaggerating the amount of money he’s spent on his house, and is giving obnoxious interviews to the WSJ, in order to stand out from a crowded field of small trading shops. Or else he’s maybe just a leveraged day-trader who made lots of money from the volatility of 2008 and now wants to flash his cash. Either way, I suspect that Schonfeld is going to be spending much more time on his private personal golf course than at the Old Westbury Country Club down the street. This kind of attitude tends not to sit well with one’s upscale neighbors.

Steve Forbes interviewed Duncan Niederauer, the CEO of the NYSE, and asked him about high-frequency trading. Unsurprisingly, Niederauer came out in favor of high-frequency traders:

They’re actually probably the most consistent source of liquidity provision in the market today. I actually don’t think they added to the volatility in the crisis. If anything, the studies that we’ve done would suggest it’s a pretty consistent provision of liquidity that would dampen volatility.

The NYSE has done studies on this? Does anybody know where I might be able to find them?

Michael Wellman has an intriguing idea for solving all the issues surrounding high-frequency trading at a stroke: switch from a market with continuous clearing to a market which clears once per second.

Orders accumulate over the interval, with no information about the order book available to any trading party. At the end of the period, the market clears at a uniform price, traders are notified, and the clearing price becomes public knowledge. Unmatched orders may expire or be retained at the discretion of the submitting traders.

Even with a period as short as one second, the call market totally eliminates any advantage of HFT systems. It does not eliminate the opportunities for algorithmic trading in general–just those that come from sub-second response time. No party has privileged information about order flow, and no party benefits by getting a shorter wire to the “trading floor”.

According to Wellman, there would be other advantages to discrete-time trading too, including lower volatility.

Would this plan essentially give everybody in the market the advantages of being in a dark pool which only exists for one second? On its face, I think it’s a good idea. What would the downside be?

I have a bit more clarity on the $20 billion figure for total profits from high-frequency trading: it comes from the TABB Group. In a recent publication, TABB’s Robert Iati writes:

TABB Group estimates that annual aggregate profits of low latency arbitrage strategies exceed $21 billion, spread out among the few hundred firms that deploy them. While we know all the large investment banks such as Goldman Sachs are committed to prop trading profitability, the hundreds of smaller, private high frequency prop shops extend much greater influence in the marketplace by providing liquidity that keeps activity flowing.   

The Bloomberg article, meanwhile, explains the figure thusly:

The firms compete for a slice of $21.8 billion in annual profits from equities and derivatives market making and arbitrage, according to Tabb. Among the largest are hedge funds Citadel Investment Group LLC, D.E. Shaw & Co. and Renaissance Technologies Corp., as well as the automated brokerages Getco LLC, Hudson River Trading LLC and Wolverine Trading LLC.

When John Hempton, then, says that “quantifications of this as a $20 billion issue are insane”, I think there are two questions: firstly, what is “this”, and secondly, how profitable is it, in aggregate.

It would be most convenient if the HFT algorithms were split nicely into a “trading” bucket and a “quant arbitrage” bucket, so that Hempton could complain mildly about the “trading” algos while saying at the same time that they’re not all that big of a problem, while ignoring the stat-arb shops and other high-frequency, low-latency traders. But in reality there’s very little difference: the traders all have strategies, and the stat-arb strategies are all implemented so as to maximize trading profits.

To put it another way, I don’t think people are making billions of dollars in profit just by being fast. But there are definitely people making billions of dollars in profits through strategies for which being fast is a necessary precondition.

Which leads us to the second question: if you tot up all the profits from high-frequency, low-latency traders, including big shops like Citadel, Renaissance, and Goldman, can you get to $20 billion? My gut feeling is that you can, and that the TABB estimate is not obviously unreasonable.

I also got a note from Jon Stokes yesterday which is worth disseminating more widely:

It’s quite remarkable to me that many of the econ and finance folks who insist that “HFT is the same thing we always did, just way faster” don’t seem to realize that frequency and amplitude matter a whole lot, and that for any given phenomenon when you suddenly increase those two factors by an order of magnitude you typically end up with something very different than what you started with. This is true for isolated phenomena, and it’s doubly true for complex systems, where you have to deal with systemic effects like feedback loops and synchronization/resonance.

What I’ve noticed anecdotally is that engineers and IT pros are more concerned about HFT than people who just handle money for a living. These guys have a keen sense for just how fragile and unpredictable these systems-of-systems are even under the best of conditions, and how when things go wrong they do so spectacularly and at very inconvenient moments (they get paid a lot of money to rush into the office to put out fires at 4am).

There’s an analogy here with e-voting, which I did quite a bit of work on. In the e-voting fiasco, you had people who were specialists in elections but who had little IT experience greenlighting what they thought was an elections systems rollout, but in actuality they had signed on for a large IT deployment and they had no idea what they were getting into. To them, it was just voting, but with computers, y’know? They found out the hard way that networked computer systems are a force multiplier not just for human capabilities, but for human limitations, as well.

This is why I’m sympathetic to Paul Wilmott’s view of all this: there’s simply too much complexity here for comfort, and too many things which can go wrong. When the stat-arb shops imploded in the summer of 2007, the systemic consequences were mild-to-nonexistent, and that does provide a certain amount of reassurance. But we can’t be sure that if and when such a thing happens again, the consequences won’t be much worse.

There’s an interesting debate in the comments to my post on high frequency trading about the widely-cited $20 billion figure for the profits attributable to HFT. In Jon Stokes’s Ars Technica article on the subject, he writes this:

At least two different groups, the TABB Group and FIXProtocol, estimate that high-frequency trading generated around $20 billion in profits for the financial sector last year. Goldman Sachs accounts for some 20 percent of global high-frequency trading activity, and the bank recently had a blow-out quarter in which its HFT-heavy trading operation racked up a record number of days where profits topped $100 million.

If Goldman Sachs alone can make $100 million a day from HFT, then $20 billion globally seems reasonable. But that’s a very big if, and I’d love to see how TABB Group and FIXProtocol arrived at their figures. (It would also be nice if HFT was clearly defined, which it isn’t, although I think most people agree that it’s a superset of flash trading.)

Elsewhere, Paul Wilmott (con) and Tyler Cowen (pro) join the debate. I’m more convinced by Wilmott than Cowen, although both make good points: Wilmott says that the complex algorithms driving HFT are prone to spectacular failure, while Cowen notes that “the correct judgment of efficiency occurs at the system-wide level, not at the level of the individual trading strategy”.

To that point, I’d be inclined to think that the massive volatility we’ve seen in the stock market of late is an indication that it’s not getting any more efficient, and therefore that it’s entirely plausible that HFT is hurting efficiency. Zero Hedge (now with its own domain name) puts the case in its strongest form:

Long-term buy and hold investors have already departed the market, as they have realized the traditional methods of approaching stock valuation such as fundamental and technical analysis have gone out of the window and been replaced by such arcane concepts as quant factors.

I think that’s overstating things, but even if it’s only true at the margin, it’s still a negative development.

At heart, the debate comes down to liquidity: is HFT a good thing or a bad thing, from a liquidity perspective? Cowen thinks it’s a good thing:

High-frequency trading brings more liquidity into the market. Call it “low quality liquidity” if you wish, but it still looks like net liquidity to me.

I don’t think that case is proven, although again the term “liquidity” is vague enough that it’s important to be able to define terms here. I think the important sense of liquidity is not narrow bid-offer spreads, but rather the ease of doing big deals at the market price, and/or the ability to buy or sell stock without moving the market. In that sense, HFT hurts, rather than helps: every time anybody tries to buy anything, the predatory algos try to pick them off. If that makespeople more reluctant to trade (“if you don’t like it, you can trade yourself at much lower frequencies”, says Cowen) then that ultimately hurts price discovery and transparency.

My bottom line is that HFT is a black box which very few people understand, and that one thing we’ve learned over the course of the crisis is that if there’s a financial innovation which doesn’t make a lot of sense and which is hard to understand, there’s a good chance there’s systemic risk there. Is it possible that HFT is entirely benign and just provides liquidity to the market? Yes. But that seems improbable to me.