How the Speed of a Trade Got Down to Nearly the Speed of Light

2 Mar 2026 · 56 min · 23 chapters

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Odd Lots Podcast Episode Summary

Episode Title

How the Speed of a Trade Got Down to Nearly the Speed of Light

Hosts

  • Joe Weisenthal
  • Tracy Alloway

Guest

  • Donald Mackenzie, Professor of Sociology at the University of Edinburgh, Author of *Trading at the Speed of Light*

---

Episode Overview In this episode, Joe and Tracy explore the evolution of high-frequency trading (HFT) and its implications on financial markets. Donald Mackenzie provides insights into the technological advancements that have led to trading speeds approaching the speed of light, the market structure changes resulting from these advancements, and the sociological implications of this race for speed.

---

Key Concepts and Discussions

The Evolution of Trading Speed

  • Historical Context: The discussion begins with the realization that while an average person can execute a stock trade quickly, professional traders operate on a much faster level, executing thousands of trades per second.
  • Need for Speed: Trading firms historically aimed to minimize the physical distance to exchange computers to reduce latency, even drilling holes and laying cables for faster connections.

High-Frequency Trading (HFT)

  • Definition: HFT is characterized by high-speed electronic trading, often relying on algorithms and market structure exploitation rather than traditional trading strategies.
  • Market Structure Exploitation: The evolution of HFT has shifted the focus from algorithmic patterns to the underlying structure of the market, creating a feedback loop that drives trading behaviors.

The Role of Technology

  • Electronic Order Books: Trading has transitioned to electronic order books without direct human negotiation, allowing for rapid execution of trades.
  • Matching Engines: The speed of matching engines, notably at exchanges like Island, revolutionized trading by executing trades in milliseconds and eventually nanoseconds.

Sociological Insights

  • Research Methodology: Mackenzie emphasizes the importance of qualitative research, personal interviews, and historical perspectives to understand technological impacts on finance.
  • Cultural Shift: The aesthetic of trading firms has moved from traditional trading floors to tech-like environments, reflecting the profiles of employees who often have coding and technical backgrounds.

Implications of HFT

  • Liquidity Provision vs. Liquidity Taking: A distinction exists between firms that provide liquidity (market makers) and those that take liquidity, leading to moral and ethical debates about the impact of HFT on market stability and efficiency.
  • Impact on Market Efficiency: Mackenzie references research indicating that the efficiency of financial markets has not significantly improved over a long historical period, despite advances in technology.

Future Perspectives

  • AI and HFT: Mackenzie’s upcoming work focuses on the relationship between AI and trading, exploring how AI might influence future trading strategies and market dynamics.
  • Speed Race Limitations: The conversation concludes with a discussion about physical limitations of speed, the economic rationale behind investments in speed, and the diminishing returns in both HFT and AI contexts.

---

Key Takeaways

  • Arms Race for Speed: The competition among trading firms has become an arms race, with firms continuously seeking technological advantages to execute trades faster.
  • Sociological Impacts: The shift in trading methodologies raises important questions about the societal implications of prioritizing speed and technology in financial markets.
  • Economic Viability: Continuous investments in speed must be justified by returns, limiting the potential for infinite escalation in trading technology.

---

Conclusion This episode provides a comprehensive look at high-frequency trading's evolution, its socio-economic implications, and the broader trends in trading technology. Mackenzie's insights offer a unique blend of sociological analysis and technical understanding, highlighting the complex relationship between finance and technology.

For more insights, check out Donald Mackenzie's book, *Trading at the Speed of Light* and continue engaging with the Odd Lots community through their newsletter and Discord channel.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Chapters

Tap a time to open that second in VO

Understanding Market Processes

0:51 to 1:54

Discussion on how trades are executed and the complexities involved.

“Tracy, one of the things that I think we like to do on this podcast is sort of de-abstract the things that we take for granted in the world.”

The Evolution of High-Frequency Trading

1:54 to 3:56

Exploration of the transition from traditional trading to high-frequency trading.

“The other thing I've been realizing about trading, obviously, the big trend here is high-frequency trading, right?”

Introducing Donald McKenzie

3:56 to 5:00

Introduction of guest Donald McKenzie and his work on high-frequency trading.

“We're going to be speaking with Donald McKenzie.”

Sociology and Technology

5:00 to 7:12

Discussion on McKenzie's approach to studying technology in finance.

“Well, thank you very much for inviting me to do that.”

The Sociologist's Toolkit

7:12 to 8:10

Insights into how McKenzie finds and approaches subjects for his research.

“And that's the bit I've always enjoyed most.”

Cultural Change in Trading Firms

8:10 to 9:16

Discussion on the aesthetic shift in high-frequency trading offices.

“Yeah, well, it's always difficult and it's always very ad hoc.”

The Impact of Island Exchange

9:16 to 14:01

Exploration of the Island exchange and its significance in high-frequency trading.

“With the chessboard and the matcha on tap.”

High-Frequency Trading Origins

14:01 to 16:01

Learn how Ireland transformed trading speed and liquidity during the dot-com bubble.

“Ireland improved on that a thousandfold.”

Understanding Speed in Trading

16:37 to 18:08

Explore the significant speed of trading from milliseconds to nanoseconds.

“Subscribe today, wherever you get your podcasts.”

The Evolution of Trading Strategies

18:08 to 21:44

Discuss how trading strategies evolve with increasing speed and technology.

“Now, someone places an order to buy or sell a stock.”
Show all 23 chapters

Banks vs. Independent Trading Firms

21:44 to 24:18

Examine why banks struggled to compete with independent high-frequency trading firms.

“I have another cultural and I guess market structure question.”

Internal Competition Among Traders

24:18 to 26:51

Learn how competition within trading firms affects resource allocation.

“Tracy, we don't know anything about long waits for computers.”

The Genesis of the Speed War

26:51 to 28:00

Discover the key moments that highlighted the competition for trading speed and co-location.

“So actually, let's just, you know, on the subject of who is the closest or who gets to have their server located where.”

Evolution of Trading Technology

28:00 to 29:20

Learn how trading firms adapted to technological advancements in trading.

“Yeah, no, I'm sure it's not a coincidence.”

The Race for Speed in Trading

29:20 to 32:50

Discover the lengths traders go to in order to gain speed advantages.

“And that kind of thing was already in place by 2005.”

Understanding Market Making vs. Liquidity Taking

34:15 to 41:40

Examine the distinctions between market-making and liquidity-taking trading firms.

“competition to be faster than anyone else, there was a narrative around that time that this was a bad thing, right?”

Limits of Speed in Trading

41:40 to 42:00

Explore the theoretical limits of speed in trading and technology.

“that actually works pretty reasonably well, better than I would have expected it to work at the start of this research.”

The Speed Race in Trading

42:00 to 43:37

Explore the limits and ongoing competition in high-speed trading.

“I'm pretty sure people always say that you can't go faster than the speed of light.”

Economic Factors in Speed Investments

43:37 to 45:51

Understand the economic implications of investing in speed for trading firms.

“But the speed race is not and will never be done.”

Market Efficiency: A Contradiction

45:51 to 48:18

Discover the surprising findings on market efficiency over decades.

“So that is work by Thomas Philippon, or his French, so I pronounce it in the American way, it's Thomas Philippon.”

AI and Diminishing Returns

48:18 to 51:41

Examine the relationship between AI investment and diminishing returns.

“And what's the particular angle or what have you been discovering so far?”

Existential Dynamics in HFT and AI

51:41 to 53:12

Learn how the competitive landscape shapes the future of HFT and AI.

“Joe, I just want to state for the record, if you give me more money, I get more intelligent.”

The Race for Speed in Trading

56:00 to 56:33

Explore the ongoing pursuit of speed in high-frequency trading.

“But as you said, the race continues of various flavors.”
Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00Joe Weisenthal:Bloomberg Daybreak is your best way to get informed first thing in the morning right in your podcast feed. Hi, I'm Karen Moscow.

0:08Tracy Alloway:And I'm Nathan Hager. Each morning we're up early putting together the latest episode of Bloomberg Daybreak U.S. Edition. It's your daily 15-minute podcast on the latest in global news, politics, and international relations.

0:21Donald Mackenzie:Listen to the Bloomberg Daybreak U.S. Edition podcast each morning for the stories that matter with the context you need.

0:27Tracy Alloway:Find us on Apple, Spotify, or anywhere you listen.

0:35Donald Mackenzie:Bloomberg Audio Studios.

0:37Tracy Alloway:Podcasts. Radio. News.

0:50Tracy Alloway:Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal.

0:56Donald Mackenzie:And I'm Tracy Alloway.

0:57Tracy Alloway:Tracy, one of the things that I think we like to do on this podcast is sort of de-abstract the things that we take for granted in the world. There are various processes. We always say this when there's like a blow up or something like that, where it's like, oh, if we're having to pay attention to X or Y, there must be something going on. But we don't always have to wait for blow ups. But we live in this world where you click buttons and things happen and you have some intuition of what happens after the button is clicked. But you don't really have a great intuition of what happens between the button clicking and the thing happening.

1:33Donald Mackenzie:Absolutely. I actually don't know how the process of like inputting an equities trade actually works, like where it kind of shows up. So that's a big question. I suspect a lot of people, even people who are in markets, probably don't know like the entire sequence of events, partly because it's gotten more complicated over the years with like reg NMS. Do you remember that? Yeah. Stuff like that. The other thing I've been realizing about trading, obviously, the big trend here is high-frequency trading, right? And it's just getting faster and faster and faster. When we first started writing about HFT, I guess in the sort of like mid-2000s after the financial crisis, I remember thinking that it was all about the actual algorithm and finding like a really smart pattern in financial markets to exploit.

2:23Donald Mackenzie:But the more I learn about it and the more I read about it, I kind of realize it's not really that. It's about exploiting the market structure.

2:31Tracy Alloway:Yeah, yeah, totally. And there's so many, you know, we had that, I forget who we were talking to recently. Oh, it was that guy from Hudson River Trading. And, you know, there were the famous like wire wars where it's like, no, I want to be one inch closer to the main server, et cetera. It's like, God, this is like a good use of brain power. So like we're going to solve the market.

2:50Donald Mackenzie:One more nanosecond faster.

2:52Tracy Alloway:One nanosecond faster, et cetera. But, you know, the other thing, you know, sort of related to this, one of my longstanding questions is, you know, a jobs report will drop at 830 on a Friday and the market immediately moves. And I'm like, how did that happen? Because it didn't happen because someone was staring. They had their like fingers above the buy or the sell button, but also had something had to be programmed such that that data could instantly be ingested. And then some sort of like directional trade was made based on that. But I don't know how that happens. I don't know how that works.

3:23Donald Mackenzie:There's also, of course, the overall question of what all this electronic trading actually means for the market itself.

3:28Tracy Alloway:Yes.

3:29Donald Mackenzie:And people talk about things, you know, with the multi-strap funds, getting these feedback loops and maybe increasing volatility in the market and things like that. So we should discuss.

3:39Tracy Alloway:And now I just had one more thing. You said, what does it mean for the market itself? What does it mean for society itself that so much effort is being placed on, like, getting a meter closer to the server room or whatever? and why is this a good use of time and is this improved capital allocation, I'm really excited to say we do, in fact, have, I think, truly the perfect guest to talk about this, someone who has sort of an extraordinary body of life's work in a range of areas that is very distinct from almost any other academic or researcher that I can think of. We're going to be speaking with Donald McKenzie.

4:12Tracy Alloway:He's a professor of sociology at the University of Edinburgh in Scotland. I first came across his work, he wrote a fantastic book called An Engine, Not a Camera, which is sort of about finance and the original birth of quantitative finance and the use of advanced models and how these models didn't just reflect what was going on in the real world, but how the adoption of these models then created this feedback loop, the engine effect, such that it actually started to drive markets themselves. More recently, he wrote a book called Trading at the Speed of Light, all about high-frequency trading. He's also written recently a book about digital advertising.

4:48Tracy Alloway:And so truly a polymath in the world of thinking about this relationship between industry and the sort of technological substrates that drive them. Professor McKenzie, thank you so much for coming on OddLots.

5:01Joe Weisenthal:Well, thank you very much for inviting me to do that.

5:03Tracy Alloway:Absolutely. Why don't you start off by telling us the gestalt of your life's work? What is your core underlying interest, such that it's produced books in these various realms?

5:15Joe Weisenthal:Yeah. I mean, fundamentally, I'm a sociologist of technology. So I'm interested in the technical systems that affect or could affect all of us. So over time, first major project in that area was on nuclear missile guidance technology. Then I moved on to safety critical computing technology, then the work on financial models that you've just mentioned, then high frequency trading, then digital advertising, you know, because as well as driving us all insane by ads that we to see appearing on our screens. That's also, of course, the big funding source for much of the everyday digital world. And then most recently of all, I've started working on AI and large language models.

6:12Joe Weisenthal:So you can see the picture. They're all highly technical areas. One way or the other, they all affect all of us.

6:20Donald Mackenzie:The other reason we wanted to talk to you is because you come at everything from this sociological perspective. And I absolutely love it when like anthropologists and sociologists go to Wall Street and write about it. Why did you take that approach, especially with your high-frequency trading book?

6:37Joe Weisenthal:Yeah, well, I don't do kind of like quantitative social science. You know, I leave that, for example, as far as markets are concerned, I leave that to economists. What I do, I like talking to people. I like going, looking at stuff to the extent that you can look at it. I like tracing how things have developed through time. My work's often got something of a historical dimension to it. But the most fun bit isn't writing the books. The most fun bit is talking to people. And that's the bit I've always enjoyed most.

7:15Donald Mackenzie:One interesting thing in this book, Joe, I don't know if you noticed, but Donald writes down all his numbers of sources and who they are. So like, you know, people from the exchange, people from high frequency trading firms, what their seniority is, which is something I hadn't really seen before.

7:31Tracy Alloway:It is a really cool thing. By the way, as Donald says, the fun part is talking to people, not so much the writing of the book. As two people who talk to people every day and have never written a book, I feel already. Now, granted, we never actually went through the process of writing the book because the talking part is so much more fun. I don't want to ever take a pause from the talking. So I already feel like to some extent, Donald is a kindred spirit. It's like it's fun to talk to people, isn't it? Talk to us about how you find them. You know, it's like, OK, HFT is interesting to you. And like, you just want to have some conversations.

8:04Tracy Alloway:What is the sociologist's toolkit here for knowing who to talk to?

8:10Joe Weisenthal:Yeah, well, it's always difficult and it's always very ad hoc. There's always a lot of luck involved in it. And with a financial market topic, I will typically start reading the Financial Times, finding names in the Financial Times, approaching those people, and then maybe they pass me on to other people. But, you know, there's also, as I said, dumb luck involved, like a crucial moment in the work I did in high frequency trading was going to interview someone at the start of it. And framed on his wall was the front cover of an issue of Forbes with the headline of the article, Free Enterprise Comes to Wall Street.

8:55Joe Weisenthal:And I thought, oh, that sounds kind of interesting. And I checked that out and it was to do with a new electronic stock exchange called Island. And it turned out that Island, the story of Island was completely interwoven with the story of high frequency trading.

9:15Donald Mackenzie:Before we get into exactly what high-frequency trading is and how it fits into placing orders for equities or futures or bonds, I have a cultural question, which is whenever you go into an HFT firm's office, it always looks like a tech company. Yeah.

9:33Tracy Alloway:With the chessboard and the matcha on tap.

9:37Donald Mackenzie:And very modern. Why do you think they've taken that approach? How did that aesthetic become the norm? Yeah.

9:44Joe Weisenthal:That's a really good indicator of cultural change, because, of course, previous to that, the sort of dominant image we might have of a financial market would be the trading floor of the New York Stock Exchange, you know, folks in colored jackets. it's they're typically televised when even nowadays when you know so there's been a big drop in the market or something so the the cameras try to catch somebody who's looking kind of glum and and worried so we think of that as what finance is or we think about like the bud fox

10:21Tracy Alloway:in wall street of a bunch of guys and slick back hair sort of you know on the phone yeah yelling at each other, looking at a green screen. Sorry, I didn't mean to interrupt that. But those are, I suspect, the two things people imagine.

10:34Joe Weisenthal:Yeah, yeah. And, you know, there was a transition involved. By and large, the high-finquency trading firms hire people who know how to code, often, you know, with higher degrees in mathematical kinds of subjects. And even the people who refer to themselves as traders often have that kind of background. And, you know, I'm sure when the visitor isn't there, there'll be a fair bit of swearing at the screen when something goes wrong and that kind of stuff. But you're right that the normal experience of those trading rooms, they're quiet, they're orderly, and you could indeed mistake them for a Silicon Valley startup.

11:20Tracy Alloway:up yeah and you see people in jeans and i visit one and like i think i saw the ceo and he was just like wearing his like college t-shirt or uh something like that strong memory i've got because

11:32Joe Weisenthal:you know in the previous work they work for an engine not a camera and some follow-on stuff i would often go to investment banks and in investment banks you know i kind of had to wear a suit and a nice shirt and a tie and so on so when i started interviewing in high frequency trading, I turned up at one firm dressed like that. And the owner of the firm sort of snarled at me, you're overdressed.

11:58Tracy Alloway:Wow. You mentioned Ireland. Why don't you tell us that story? You know, I want to get more into the tech, et cetera. But you're like, OK, this turned out to be an exchange. What was distinct? What is Ireland? I've heard of it, but it's, again, one of these things that I've heard of it and then I moved on. What was distinct about this and why is it so interwoven into the history of HFT. How is it different from other exchanges that have existed for hundreds of years?

Read the full transcript

12:23Joe Weisenthal:Yeah, yeah. You know, I'm going to oversimplify, of course, because there were predecessors to Ireland, you know, were a little bit like it and so on, but that would take us too long to go into. I mean, fundamentally, trading on Ireland was organized around an electronic order book, which is a list of all the bids to buy or offers to sell the shares in question. And that electronic order book is managed by something called a matching engine. And as the name implies, that looks for a match. In other words, a bid to buy and an offer to sell at the same price. And when it finds that couple, it consummates the trade and the trade is done.

13:07Joe Weisenthal:So it's all done electronically. There's no direct human negotiation involved. You just enter your orders into the order book and the matching engine either executes them or fails to find a match. There were exchanges prior to Ireland that worked in that kind of way. But what was distinctive about Ireland is that its matching engine was blisteringly fast by the standards of the day, which was essentially the late 1990s. So the closest analogue was a system called Instanet. And it might take a couple of seconds, the matching engine, to find the match and execute the trade. And of course, for a human being sitting there, even if they're impatient, two seconds is not a very long time.

14:01Joe Weisenthal:Ireland improved on that a thousandfold. So it could execute trades in two milliseconds, two thousandths of a second. So that was the opening for high frequency trading that with exchange, I mean, strictly Ireland was not exchange. It was what was called an electronic communications network or ECN. But I'll call it an exchange for simplicity. If you've got an exchange like that and you've got an automated trading system, it's a marriage made in heaven. The two things, the exchange and the trading firm, fit each other very, very well. And amongst the consequences of that is that liquidity in Ireland, it traded NASDAQ stocks.

14:54Joe Weisenthal:This is the time of the dot-com bubble, of course, where there's a lot of trading of NASDAQ tech stocks. Ireland brought a lot of liquidity to that market. So that's, you get a kind of feedback loop where you get automated trading, bringing liquidity to exchanges that have the kind of technical features that make high frequency trading attractive and feasible. So the established exchanges started to have to change how they did things because otherwise they were going to lose out to the new exchanges. And that's basically the feedback loop that's created today's electronic markets.

16:00Donald Mackenzie:innovation and the future of business.

16:02Tracy Alloway:Every weekday, we bring you the top headlines from the world's biggest tech companies.

16:07Donald Mackenzie:From finance to defence, AI to entertainment, and from startups to the magnificent seven. We highlight the latest stories of the people and companies pushing the tech sector to new frontiers and the politics that shape global

16:19Tracy Alloway:tech markets.

16:20Donald Mackenzie:We do this all every weekday, then bring you the most important conversations and analysis in our podcast.

16:26Tracy Alloway:Search for Bloomberg Tech on YouTube, Apple, Spotify, or anywhere else you listen.

16:31Donald Mackenzie:Join us every afternoon on your commute home and stay ahead of the tech news cycle.

16:35Tracy Alloway:That's the Bloomberg Tech Podcast. I'm Caroline Hyde in New York.

16:38Donald Mackenzie:And I'm Ed Ludlow in San Francisco. Subscribe today, wherever you get your podcasts. Joe, whenever I hear terms like millisecond and nanosecond, I just, it's so hard to wrap my head around what that actually, it's faster than that, for sure.

16:53Joe Weisenthal:I can actually help there. I'm going to hold my fingers. I'm holding them 30 centimeters apart, or since you're in the US, I'll say one foot apart. At the speed of light in a vacuum, it takes a nanosecond to get from one finger to the other finger. And that's an indication of how fast automated trading, specifically high-frequency trading, has become. That when I started working on the topic in roughly 2011, people were still talking about milliseconds or thousands of a second. Two or three years later, it had become microseconds or millionths of a second. And by the time I was finishing the research, nanoseconds were starting to account.

17:47Joe Weisenthal:Unbelievable. light traveling that 30 centimeters, traveling that foot, that matter to high frequency trading by roughly 2018, 2019, 2020, around about then.

17:59Donald Mackenzie:That's very helpful. I do have questions about the physical realities of how fast we can actually go with all this stuff. But before we go any further, can you talk about the process of, let's just focus on the equity market for now. Now, someone places an order to buy or sell a stock. What actually happens in the ecosystem between traders and market makers and the exchange that makes that happen? And what does it mean to actually make that happen and execute the trade?

18:29Joe Weisenthal:So what happens is your order via the broker, your user, I mean, in the brokerage system is no longer a human being, via the brokerage system gets placed in the exchange's order book. And then one of two things happens. The first is if the matching engine can find an existing order in the order book that matches the price of your order, it executes the trade. And the trade then happens not quite instantly, but very, very, very fast. And you're done. That's it. It's over. If, on the other hand, there is no match as the order book stands, your order rests in the order book. And it stays there until either you cancel it or a matching order comes along and then it's executed at that point.

19:32Joe Weisenthal:So that's the basic process.

19:34Tracy Alloway:You know, so it occurs to me like gains of speed in trading have been happening forever, long before we were talking about, you know, anything electronic. I'm sure other technologies exist. Technological evolution is a long time thing. It is a pretty banal statement. I suppose. But what I'm curious about is the sense of which a change in degree becomes a change in kind, essentially. So that like when you go from one second to a thousandth of a second to a nanosecond, how does that change, say, like the types of strategies that can then be employed or the types of skills that might be required to be a successful trader in the nanosecond era versus the one second era?

20:18Tracy Alloway:Like talk to us about like that relationship.

20:20Joe Weisenthal:Yeah, yeah. Well, there's a wonderful book by the historian of technology, Jimena Canales, which is called A Tenth of a Second History. And the significance of the tenth of a second is that's the generally accepted lower threshold of the human perceptibility of time. You know, basically, we just can't mentally process time intervals that are less.

20:49Tracy Alloway:So, Tracy, don't feel bad about not being able to build an intuition for a nanosecond.

20:54Joe Weisenthal:Less than a tenth of a second. So what essentially happened is that we've moved from that kind of, you know, the tenth of a second or longer, from that kind of epoch into an epoch where human beings, I mean, they can still be in overall control of the system. but they can't actually execute the actual trading decisions fast enough not to be outrun by an algorithm. So we've moved from a kind of human-centered form of trading to a machine-centered form of trading. And the actual threshold of the change is probably around that tenth of a second amount.

21:44Donald Mackenzie:I have another cultural and I guess market structure question. But one thing that I always thought was interesting about high frequency trading was that the banks didn't really get into it, which, you know, there's one big reason why, which is the ban on prop trading after 2008. But even before then, they just never seemed to be able to compete with independent firms. Why did that happen?

22:08Joe Weisenthal:Yeah, I've asked people that. And there's, I think, complex sorts of reasons. And let it be said, some banks have been more successful than others. Banks are not always bad at this, though most banks are bad at it. One way of thinking about it is that typically, a bank will have an IT department that's separate from the other functions of the bank, like trading, like market making and so on. So if you're a trader, you got to persuade the IT people to give you a fast enough system, which involves them maybe writing some new software, buying some new kit. So you need to get higher level management sign off on it.

22:59Joe Weisenthal:And it all takes time. Whereas the high frequency trading firms are typically pretty small. 50 people is a decent size firm. 150 people is a reasonably big, high-frequency trading firm. Very often, those firms are owned and run by the people who founded them. So there is a boss or bosses, but other than that, it's a relatively flat organizational operational structure. If, for example, at least this was the case in the early days of high frequency trading, it's not quite as simple as this now, but in the early days of high frequency trading, you know, if some IT firm came out with a new, better, faster server, and you were a trader in a firm like that, and you know, okay, well, let's get this new server.

23:53Joe Weisenthal:You could just use your own personal credit card to buy the server, get it delivered to your office, and then get your engineers to take it out to whatever data center that they were trading in and get it installed straight away. And, you know, that could, maybe that would be a week or 10 days or something. Whereas in the bank, you'd be doing pretty well if you could achieve that within six months.

24:18Tracy Alloway:Yeah.

24:19Donald Mackenzie:That's amazing.

24:20Tracy Alloway:Tracy, we don't know anything about long waits for computers. to arrive.

24:26Donald Mackenzie:No, we surely don't. That's sarcasm, by the way. Actually, this reminds me of something that I wanted to ask, which is we know there's competition between firms, high-frequency traders for the fastest connections to exchanges and things like that. There's also competition, I imagine, within the firm itself, because setting aside the credit card anecdote, these can be expensive and there's also limited supply. You know, only so many servers can be co-located where they want to be. In your research, how did you actually find executives at HFT places? How did they actually allocate the fastest connections to which team?

25:07Joe Weisenthal:What I found was the high-frequency trading firms fell into two different camps, so to speak. In some of them, there were separate trading teams that didn't really communicate with each other and indeed by design didn't communicate with each other. In some cases, the office was actually laid out in such a way that somebody in one team was not very likely accidentally to overhear something said by someone in another term. And in those firms, yes, I mean, they are essentially in competition. And I think that in that kind of firm, then the results of each trading desk, the P &L, the profit and loss, the little trading teams that are doing best would get the available bandwidth on the microwave links that are crucial to high frequency trading and so on.

26:06Joe Weisenthal:And maybe they would get the fastest machines first and so on and so forth. The other kind of trading firm was and is operated as a unified entity. In some cases, even without individual profit and loss in individual P &L accounts for traders. And there's a lot of shared infrastructure in that kind of firm. And indeed, there's also shared infrastructure in the segregated kind of trading firm. Because if you're the boss of such a firm, there's obviously simple economies in not having completely separate IT systems for each trading team. But there's that kind of divide. Does the firm operate as a unified entity or does the firm operate as a sort of aggregate of competing trading teams?

27:00Tracy Alloway:So actually, let's just, you know, on the subject of who is the closest or who gets to have their server located where. Tell us a little bit more about the timeline. So Island emerges in the late 90s. When did it start to dawn on people in the trading industry that given this new physical reality, given the speed, we need to start thinking about who is going to have co-location? We need to start thinking about sort of like microwave radio line of sight. Where did that speed war? What was the genesis of it?

27:36Joe Weisenthal:Yeah, yeah. I mean, a kind of crucial date was 2005, where Ireland, which had already been bought by Instanet, was acquired by NASDAQ. And the New York Stock Exchange acquired another of the pioneering electronic trading exchanges. This one was called Archipelago.

27:58Tracy Alloway:Is that just a coincidence, that island and archipelago? That's kind of like...

28:02Joe Weisenthal:Yeah, no, I'm sure it's not a coincidence. And they technologically reorganized themselves in New York Stock Exchange and NASDAQ, technologically reorganized themselves around this new insurgent technological approach to trading, so to speak. And 2005, because of the acquisitions in that year, is a kind of noteworthy year. But even before 2005, people in trading firms started to become aware that you couldn't just do automated trading with the machine sitting in your office. For example, there's a Kansas City trading firm called Tradebot, whose owner, Dave Cummings, has written a really rather nice autobiography.

28:48Joe Weisenthal:And one of the things that Cummings came to realize is that trading an island while having your machines in Kansas City was placing him at a disadvantage. So firms like that started to move their machines either directly into the island computer room or they couldn't do that into the offices of another firm in the building to shorten the distance. And that kind of thing was already in place by 2005. Two things then happened. For a period, there was a kind of Wild West, so to speak, where there are lots of stories of high-frequency traders like drilling holes in walls so as to shorten the distance between their servers and the exchanges matching engines.

29:45Joe Weisenthal:That has by and large generally come to an end. And what happens now in the data centers of all the major exchanges is there is a rule about equal cable length. So if you happen to have, if a trading firm happens to have its servers physically close to the exchange matching engines, the fiber optic cable that connects them is coiled so that there's exactly the same cable length for each of the trading firms within that. data center. The other thing that started to happen is that getting the signal from one exchange's data center to another exchange's data center started to become a technological speed race.

30:35Joe Weisenthal:Back to 2005, by and large, it would just be sent by fiber optic cable, but the exact route was not under the control of the exchanges or of the trading farms. So there's a lot of sort of randomness. The crucial link here is actually between the Chicago Mercantile Exchanges data center. Then in downtown Chicago, it's now in the suburbs of Chicago. ago, the link between that and the data centers that trade shares in northern New Jersey. And there was a kind of triple evolution there. The first evolution was that the particular trading firm, and since it's no longer directly in business, I can actually name it Getco, managed to, as it were, stitch together existing fiber optic cables to get the fastest route on the Chicago, New Jersey link.

31:39Joe Weisenthal:And that actually in the business was known as the gold line, gold because of the money that you could make by having the fastest route. Then in 2010, if memory serves me right, a new firm, Spread networks actually dug an entire new cable from Chicago to northern New Jersey, you know, drilling, you know, sort of underneath car parks and, you know, just really trying to be as close to what a geographer would call the geodesic. In other words, the fastest route on the surface of the Earth from point A to point B. Then third phase, that was trumped by the arrival of microwave because light in a fiber optic cable, I mean, the core of fiber optic cable is essentially a specialized form of glass.

32:29Joe Weisenthal:And that glass slows the light down to only two thirds of its speed in a vacuum. Whereas if you can shoot your electromagnetic signal through the atmosphere, it's not exactly at the speed of light in the vacuum but it's very very close to the speed of light in the vacuum so that was the third phase when people moved from exclusive use of fibre optic cable to supplementing the fibre optic cable by microwave links between Chicago and northern New Jersey

33:02Tracy Alloway:that was incredible

33:19Donald Mackenzie:This is Scarlett Fu. And I'm Paul Sweeney, inviting you to join us for the Bloomberg Intelligence Podcast. Every day, we harness the power of Bloomberg Intelligence to bring you deep dives into the companies that are moving markets from publicly traded companies like Apple to those that are privately owned but known by everyone on earth like OpenAI. Now, I helped to build Bloomberg Intelligence to what it is today, Scarlett. And now our analysts are the best in the world, covering more than 2 ,000 global companies. That is your legacy, Paul. And we speak to those in-house experts every day. They are Bloomberg's go-to authorities on sectors, companies, and legal processes.

33:52Donald Mackenzie:And we do it all live each weekday, then bring you the best conversations in our daily podcast. So be sure to search for Bloomberg Intelligence on YouTube, Apple, Spotify, or anywhere else you listen. Listen in the afternoons on your way home from work to catch up on the market news you miss during the business day. That is the Bloomberg Intelligence Podcast. I'm Scarlett Fuhl. And I'm Paul Sweeney. Subscribe today, wherever you get your podcasts. I have another cultural slash market structure question, which is along with this intense competition to be faster than anyone else, there was a narrative around that time that this was a bad thing, right?

34:32Donald Mackenzie:And one of the interesting cultural things is you saw the high frequency trading firms kind of divide themselves into good guys and bad guys. So there were some that were saying, well, we make liquidity, right? We're good for the market. And then there were others. I mean, they wouldn't describe themselves this way, but they were accused of being liquidity takers in the market. How should we think about that particular tension?

34:57Joe Weisenthal:Yeah, no, that's a very fundamental thing because you're quite right. that there's a degree of differentiation between trading firms and indeed in the more segregated firms also within those firms you have different desks fall on others on different sides of that I mean the way to think about it is to remember that in all these exchanges trading is organized around an electronic order book as I said a list of the bids to buy and offers to sell the instrument in question. What a market making firm does is it places lots of orders into the order book at prices that can't immediately be executed.

35:45Joe Weisenthal:But those orders then populate the order book. So if somebody else comes along, an individual investor or maybe an asset management firm, somebody else comes along and needs to trade, they find an order book that's populated with lots of existing bids and offers that they can execute against. So the market making firm is providing liquidity. And most people reckon that that's a good thing. The other kind of firm, the one where, as you say, there tends to be more controversy, they don't do that. They don't constantly populate the order book. Their systems constantly monitor the order book. And then when they detect what they think is a profitable opportunity, they execute against the orders that are already in the order book.

36:49Joe Weisenthal:And that is called liquidity taking because, of course, the execution removes the order from the order book. And then in practice, a lot of this is actually going on between high frequency trading firms because most of the orders in the order book are placed by market making high frequency trading firms. And a lot of the executions against those are by the high frequency trading firms that specialize in taking liquidity. And this is the core of what gives the business its characteristic as an arms race in speed. So imagine for a moment that your algorithms are trading equities in one of the data centers in northern New Jersey and the relevant stock index future traded in Chicago changes in price or even there's a big shift in the order book for that stock index future.

37:55Joe Weisenthal:And let's say the price of the stock index future falls. That's very likely to lead within a tiny fraction of a second to falls in the price of the underlying shares being traded in New Jersey. And in that tiny little fraction of a second, in that intervening period, a lot of the market making firms orders in those order books become stale as people in the markets. So the making firms rush to try to cancel their stale orders and the taking firms race to execute against those stale orders. And that's a race that nowadays can literally be played out in nanoseconds.

38:46Tracy Alloway:That's interesting. I hadn't appreciated that dynamic at all, to be honest. You mentioned that these high frequency trading firms, they're all sort of like they're run and operated basically by the employees, the owner or something like that. What is it that sort of distinguishes a high frequency trading firm from, say, a hedge fund that would have limited partners and make distributions, et cetera? You know, you never hear about like, oh, I have an investment with Jane Street or something like that. I don't think that's the right thing. What is it about the nature of the business such that essentially they trade their own capital?

39:21Joe Weisenthal:Yeah, I guess that's just in a sense. It's one of those things that's historical evolution. I mean, in many cases, those high-frequency trading firms were initially set up by successful floor traders, particularly floor traders in the Chicago markets, the futures markets. And if you were successful in that, you could, you might not become a billionaire, but, you know, you could make a decent amount of money, tens of millions of dollars. and that was enough to enable you to start an initially small automated trading firm and you often didn't need any external capital to do it you just had to buy the necessary technical kit and hire technically savvy people and so on but you could start really quite small you could start a 10-person firm or something like that now the business has got a lot more expensive since then because in good part of the speed race that we've just been talking about.

40:28Joe Weisenthal:But by and large, those firms made profits. The owners reinvested the profits in the firm. So the capabilities of the firms grew to meet the growing demands on them. And then another thing that should perhaps be said is that those founders would have the majority of their net worth invested in the firm. And so their risk control was often pretty good. Risk management for those firms was not a separate bureaucratic department that the traders had to try to outwit, so to speak. The founder would quickly detect if you were trying to do something like that. Now, some automated trading firms are blown up nevertheless, but it's actually quite interesting how few of them have blown up.

41:27Joe Weisenthal:Yeah, because of, you know, for example, because of classical software bugs. And that's, I think, because the relatively small structure, the hands-on involvement of the founders, etc., etc., has created a kind of technical culture that actually works pretty reasonably well, better than I would have expected it to work at the start of this research. Hmm.

41:56Donald Mackenzie:So one thing I wanted to ask is this idea of physical limitations to how fast we can actually go. I'm pretty sure people always say that you can't go faster than the speed of light. There's probably some caveats there about like quantum entanglement and stuff like that. But surely we must be getting close to how fast things can actually go. What's your sense of how long the speed race can continue?

42:22Joe Weisenthal:Yeah, well, we can never get to zero, of course. I think Einstein is basically correct. You can't get faster than the speed of light in a vacuum. And similarly, a computer system, there's always going to be a non-zero processing time of the system. But you can get ever, ever closer to that. So in mathematics speak, zero is an asymptotic limit. That's to say you can always get closer to it, but you're never actually going to get right there. And I think that's the nature of the business. You know, we're still in the nanosecond regime. If I remember correctly, the next lowest time interval is the picosecond.

43:15Joe Weisenthal:I could imagine this continuing in a domain of picoseconds. So that's the way I would see it, that there's a hard limit, but we're never actually going to get to the hard limit. We're still going to race to get as close as possible.

43:30Tracy Alloway:But your understanding as of the time of your work is that the race is not over, that for these firms, And I'm sure they have many different projects underplayed, including various things with AI, which we haven't gotten to, and I guess we won't. But the speed race is not and will never be done.

43:47Joe Weisenthal:Yeah, I think that's correct. Now, of course, there is an economic process at work here, which is to say that the investments that you make in speed have to be recoupable from the trading profits that you make from your trading. And I think Tracy, I think, said at the very beginning, what essentially is going on here is that structural features of financial markets are being exploited, like the relationship between the stock index future and the underlying equities. the amount of money to be made by exploiting those kind of structural features is not trivial. It's been nicely measured by the Chicago economist Eric Sufert and colleagues.

44:45Joe Weisenthal:It's not trivial, but it's perhaps single digit billions of dollars. so you know suddenly deciding you're going to invest 50 billion dollars in the technology of speed would be a dumb thing to do because you wouldn't be able to recoup it so there is that economic factor that is i'm pretty certain slowing you know the speed race is still there but it's it's you know and things are getting faster but the rate at which they're getting faster. It's certainly not accelerating. And I think that economic factor probably explains it.

45:25Tracy Alloway:That makes sense.

45:26Donald Mackenzie:So we should talk about the impact of HFT on the overall market a little bit more. And one of the things that caught my eye in your book was you cite a previous study, I can't remember by who, but basically saying that the efficiency of financial markets has not improved between the 1880s and 2012, which is very counterintuitive.

45:48Tracy Alloway:It seems like impossible to imagine, but what does that mean?

45:51Donald Mackenzie:Yeah.

45:51Joe Weisenthal:So that is work by Thomas Philippon, or his French, so I pronounce it in the American way, it's Thomas Philippon. What he means by efficiency there is really rather different from what I've been talking about. What he means by efficiency is what he calls the unit cost of financial intermediation, which essentially is basically putting it crudely how much it costs to do the kind of thing that investors want to do, that asset managers want to do and so on. And, you know, it is a very striking finding that from the 1880s to, I think, his most recent data goes up to 2015, that there was no really clear cut tendency for that cost to decline, despite all the advances in information and communication technologies over those many decades.

46:51Joe Weisenthal:And the explanation, to put it simplistically and crudely, is the capture of those efficiency gains in the form of high pay in the financial sector, typically through the form of fees. So the fees that you pay for an index fund, say, for example, those have really gone down, but people have also been moving their money into private equity and the like hedge funds, which have much higher fees and so on. So those kind of effects seem to have sort of cancelled themselves out. In the 1940s, a professional in finance was basically paid roughly the same as somebody with equivalent educational qualifications in a different line of business.

47:45Joe Weisenthal:And then from the 1970s onwards, the gap has got bigger and bigger and bigger now. These days, of course, you can make a lot of money by being a technologist in AI, for example. But by and large, those sort of exceptions aside, finance is an extraordinarily well-paying professional, at least for those in the central roles in it. And that's essentially the explanation of that finding by Philippon.

48:17Donald Mackenzie:Well, this is a good opportunity to ask about AI because I suppose it's inevitable as a sociologist who examines the tech industry or looks at how tech is impacting things. Your next project must be AI, right? Yes, it is. And what's the particular angle or what have you been discovering so far?

48:37Joe Weisenthal:Yeah, it's very early days. But the thing I'm most interested in so far is the question of scaling and AI. Because, of course, it's no secret to anybody who reads a newspaper, subscribes to Bloomberg or whatever. I mean, the huge trillions of dollars are being thrown at AI infrastructure. And absolutely, there is a sort of logic there that's repeatedly stated that these systems are all built around neural networks. And the effectiveness of a neural network grows with the size of the network, the size of the training data, the number of parameters in the model, and so on. And there are well-known scaling laws.

49:29Joe Weisenthal:But, and this is the thing that interests me, is the but. There's a very nice little statement from Sam Altman in February of last year, that the intelligence of a system is roughly the log, the logarithm, in other words, of the resources devoted to training it, running it, to computation at inference time, and so on. Now, of course, what Altman meant was basically give the industry more money and you'll get more intelligence. And that's, of course, indeed, giving more money to the industry is exactly what's going on. But a logarithmic function, there's a bit of maths here. A logarithmic function, at least of the kind that Altman is referring to, is a diminishing returns function.

50:25Joe Weisenthal:You can draw its graph and it very clearly demonstrates diminishing returns. You can always get better and better, but each increment costs you more in terms of the resources deployed. And we're dealing here where the horizontal axis in the graph, so to speak, is denominated in trillions of dollars of financial input or hundreds of megatons of carbon dioxide emitted by the electricity generation needed to power the data center. So the question becomes, on a diminishing returns curve, how far do you go? When do you decide we really got to stop? Can you decide we really got to stop? Do you have kind of quasi-magical beliefs, so to speak, that at some point the diminishing returns, something qualitative will happen?

51:31Joe Weisenthal:that artificial general intelligence or super intelligence will suddenly appear. So that's the core of what I'm interested in right now. How far do you go along a diminishing returns curve?

51:45Donald Mackenzie:Joe, I just want to state for the record, if you give me more money, I get more intelligent. There's no diminishing returns. Just for the record.

51:52Tracy Alloway:You know, noted, first of all. And it's interesting, you know, hearing this in the context, And it suddenly makes so much sense how this fits into your work and this idea of like the arms race. Right. Because, yes, it's true. Like maybe there's only so much extra profit available for the firm. And maybe that pool of profit is shrinking and maybe it gets more and more costly to sort of exploit the remaining profit that's available. But on the other hand, you can't fall behind. You can't let – and this is – so it's true in HFT and it's clearly true in AI where, okay, like it spends more and more money to improve the model.

52:31Tracy Alloway:But you can't fall behind even if the economics look worse with each iteration. Anyway, Professor McKenzie, lovely conversation. I really enjoyed that. We really learned a lot. Really appreciate you coming on to OddLots. And yeah, thank you for joining us. Well, thank you both.

52:46Joe Weisenthal:Thank you both for inviting me, like I said. And, you know, thanks for a really great conversation.

52:51Tracy Alloway:That's fantastic.

52:52Donald Mackenzie:We'll have to have you back on when you publish your AI book. Absolutely.

53:07Tracy Alloway:Tracy, I love that conversation. Really interesting. I love like encountering people who are like actually understand the tech, actually can articulate what the tech is doing. Especially it's always impressive, someone with a sociology background, etc. To just sort of be like that comfortable. And I think that's like the through line of his work is like he gets it.

53:30Donald Mackenzie:Right. So in the book, there's lots of like field trips to data centers and looking at cables and things like that. But then also, as he stated, just talking to people and getting anecdotes. And there's funny stories about like the Battle of the Asterix and things like that. That's at the very end. But people should go and read it. The other thing that stood out to me from that conversation was towards the end when we discussed AI, you made the point that you get this similar dynamic between HFT and AI now where because everything is framed as existential, you just can't stop. Right. You always have to keep going.

54:06Tracy Alloway:Totally. Look, I mean, I suppose the high frequency trading is not sort of like existential in the broad sense, but it's existential at the firm level. That's what I was going to say. Yeah, yeah. At the firm level. So it's like and I hadn't really thought about, OK, you like have this like pool of theoretical profit, which is the gap between where the futures are trading in Chicago and where the stocks are trading in New York. That's fixed, right? That's not going to get that big. But again, like someone gets faster at exploiting that. And I had never really heard quite until your question and his answer, this sort of maker taker dynamic of, OK, I have these orders and now I'm quickly rushing to cancel them and you're quickly rushing to fulfill them.

54:51Tracy Alloway:and if you and I are both in the market, we can't slow, if you get faster, I must get faster because you're going to then snipe me every time or vice versa, et cetera. But, you know, they still make a lot of money, it seems like, unlike the AI firms. They make a ton of money. Yeah, exactly.

55:08Donald Mackenzie:That was a funny dynamic in HFT world, the like accusations of taker versus maker. And I always think of the, you know, the Spider-Man meme where they're all kind of pointing at each other. It felt very much like that. But it was really great to catch up on HFT again. This was sort of a blast from the past because you used to hear about it more and now it's become so normalized that people just don't talk about it that much.

55:29Tracy Alloway:Well, you know, we heard about it a lot and especially post 2008. Yeah. That was the ultimate finger pointing era, right? The Michael Lewis book. Because you just have like everyone had some, oh, it's the naked short sellers. It's the credit rating agency. agency. It's the, you know, the law that forces banks to be equitable and who they distribute mortgages to, et cetera. Like there was a million finger pointing, oh, maybe it's the HFT firms, maybe it's the short sellers, whatever. So, I mean, part of the reason we don't hear about it as much is because there hasn't been a crisis, et cetera. But as you said, the race continues of various flavors.

56:06Donald Mackenzie:You can never get to zero, but you can always get closer.

56:08Tracy Alloway:You know, it's like, you know, I always think about some lines, they go like straight up, you know? It's like, they ever like curve back around, etc. You get like negative space. Like, can we do even better than line go up? Like line go up and backwards? I guess Einstein would say no.

56:25Donald Mackenzie:If only we could have Einstein on as a guest to talk about trading. To talk about high frequency trading.

56:31Tracy Alloway:Shall we leave it there? That would be a perfect guest. Let's leave it there.

56:33Donald Mackenzie:All right. This has been another episode of the Odd Lots podcast. I'm Tracy Allaway. You can follow me at Tracy Allaway.

56:39Tracy Alloway:And I'm Joe Weisenthal. You can follow me at The Stalwart. Check out Donald McKenzie's book, Trading at the Speed of Light. And of course, follow our producers, Carmen Rodriguez at Carmen Armand, Dashiell Bennett at Dashbot, and Kale Brooks at Kale Brooks. And for more OddLots content, go to bloomberg.com slash oddlots. We have a daily newsletter and all of our episodes. And you can chat about all these topics 24-7 in our Discord, discord.gg slash oddlots.

57:05Donald Mackenzie:And if you enjoy Odd Lots, if you like it when we look back at the HFT boom and how it continues, I guess, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely ad-free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening.

57:58Donald Mackenzie:Hello, I'm Michelle Hussain, and for more than 20 years, I was at the BBC. military withdrawal from Afghanistan. But all the time I was delivering the headlines, I wanted to go further than the news of the day, to spend more time with the people shaping our world. And that's what I'm doing here on this podcast. Speaking to people from Nigel Farage,

58:21Joe Weisenthal:Russia needs to be taught a lesson, to tech journalist Kara Swisher.

58:26Donald Mackenzie:And the tech industry is running wild. You know, they've gotten what they wanted and they've seen a huge run-up in their stock prices. This will be a place where every weekend you can count on one essential conversation to help make sense of the world. So please join me, listen and subscribe to The Michelle Hussain Show from Bloomberg Weekend, wherever you get your podcasts.

58:51Tracy Alloway:You certainly ask interesting questions.

From the publisher

The average person can enter a stock trade on their computer, hit refresh, and the trade is done. As fast as that seems, there are professional traders moving even faster, executing thousands of trades per second. Over the years, the need for speed got so intense that competing firms would aim to get their own systems closer and closer to the exchange's computers, so as to minimize the length of the wires and get their trades in even faster. How did this happen? And how does this change the nature of trading itself? On this episode, we speak with Donald Mackenzie, a professor of sociology at the University of Edinburgh in Scotland. Professor Mackenzie has been studying the intersection of finance and tech for a long time, and in 2021 wrote the book, Trading at the Speed of Light. We discuss the history of finance technology and look at where the technological arms race is going next.

Subscribe to the Odd Lots Newsletter
Join the conversation: discord.gg/oddlots

See omnystudio.com/listener for privacy information.

More from Odd Lots

All 682 episodes
How the Speed of a Trade Got Down to Nearly the Speed of LightOdd Lots · 56 min
Listen in VO