In short
Podcast Notes: Richard Craib – Crowdsourcing Data Science for Returns at Numerai (Capital Allocators, EP.314)
Episode Overview In this episode of Capital Allocators, host Ted Seides interviews Richard Craib, the Founder and CEO of Numerai, a machine learning-based hedge fund that utilizes crowdsourcing to develop models predicting stock returns. The episode delves into Richard's background, the fundamentals of data science, and the innovative approach of Numerai in the investment industry.
Key Concepts and Themes
Introduction to Numerai
- Foundation: Numerai launched in 2015 with the aim of crowdsourcing models from data scientists globally to predict stock returns.
- Incentive Structure: Participants are incentivized through cryptocurrency, creating a unique ecosystem for model development.
- Centralized Portfolio Management: Numerai centralizes portfolio construction and risk management, contrasting with traditional hedge fund models.
Richard Craib's Background
- Early Interest in Investing: Richard's journey began at a young age, trading stocks and developing a fascination with market dynamics.
- Education Shift: Realizing the limitations of popular finance literature, he pursued quantitative mathematics and machine learning, leading to his first job as a quant.
Data Science and Machine Learning
- Quantitative Approach: Richard distinguishes between traditional investing strategies that form hypotheses versus machine learning, which seeks to uncover patterns in data without preconceived theories.
- Feature Utilization: Data scientists work with a vast dataset (over 1 million rows, 2,000 columns), where features are anonymized. The goal is to model stock returns based on these features.
The Numerai Model
- Crowdsourced Data Science: Numerai allows anyone to join, download datasets, and submit models to improve prediction accuracy.
- Staking Mechanism: Users can stake cryptocurrency on their models, aligning incentives with performance. Poor-performing models result in a loss of staked funds.
Community Engagement
- Data Scientist Community: The participants include some of the best data scientists globally, often from varied backgrounds, not limited to finance.
- Compensation Structure: Over $30 million has been paid to users since inception, making Numerai one of the highest-paying data science tournaments online.
Challenges and Innovations
- Market Dynamics: Numerai adapts to changes in model performance and market conditions, requiring fresh strategies and continuous innovation.
- Data Acquisition: The focus is primarily on acquiring long-term fundamental data that is consistently applicable across market conditions.
Future Prospects
- Research and Development: Richard sees potential in integrating large language models into Numerai's framework, aiming to enhance feature generation and prediction capabilities.
- Scaling Capacity: The firm is structured to handle significant asset sizes without losing effectiveness, focusing on high-capacity strategies.
Operational Insights
- Team Structure: As the firm grows, it has transitioned from a purely technical team to include experienced financial professionals for better operational management.
- Performance Attribution: The success of Numerai is measured through robust attribution models, showcasing skill rather than luck.
Key Quotes
- "The promise of a quant fund is that it can do well in all environments in a market neutral context."
- "Machine learning is so in our DNA."
Closing Insights
- Management Philosophy: Richard emphasizes the importance of interpersonal skills in managing teams and organizations.
- Personal Interests: Outside of work, Richard enjoys surfing and painting, appreciating the balance of physical activity and creative expression.
Conclusion This episode provides a comprehensive look at Numerai's innovative approach to investing through crowdsourcing and machine learning, highlighting Richard Craib's unique journey and the firm’s role in the evolving landscape of institutional investing. For more insights, visit the [Capital Allocators website](https://www.capitalallocators.com).
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Transcript
Automatic transcript. May contain errors.0:00Thank you.
0:31All opinions expressed by TED and podcast guests are solely their own opinions and do not reflect the opinion of capital allocators or their firms. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of capital allocators or podcast guests may maintain positions in securities discussed on this podcast. My guest on today's Sponsored Insight is Richard Crabe, the founder and CEO of Numerei, a machine learning-based hedge fund launched in 2015 that crowdsources models from data scientists around the world to predict stock returns, incentivizes participation through cryptocurrency, and centralizes portfolio construction and risk management.
1:14Our conversation covers Richard's background, some basics of data science, and the Numeri thesis. We discuss the firm's idea generation, incentive system, quantitative modeling, portfolio construction, and team. And we close with investors' reaction to the product and research and development on the horizon. I hope you enjoy the show. And if you do, this week, if you happen to be in a married or committed relationship, and one night you turn to your partner and say, Hey, babe, what do you think? And they turn back and say, Sorry, hon, not tonight. I have a headache. Why not turn a lemon into lemonade by responding, I have a better idea.
1:55Let's listen to the Capital Allocators podcast together. You can snuggle up and share a night of stimulating intellectual bonding. Thanks so much for spreading the word to your partner. Please enjoy my conversation with Richard Crabe. Richard, great to see you. You too, Ted. Why don't you take me back to your path to this particularly interesting approach that you're taking to investing? Well, to go way back, when I was eight years old, my dad, who worked as a portfolio manager, he gave me my first stocks. And I would follow them in the newspaper. And I remember rushing to my grandmother's room to try to see how the stocks were doing.
2:40And I was just fascinated by it. I was fascinated by the fact that these kind of numbers represented real companies in the world doing real things. And I was fascinated by things like, well, why can a stock move 20 % in a single day? Why isn't it 2 %? That's a lot. Why is it 20? And then also got fascinated with the idea that it's very hard to beat the market portfolio and that nearly all investors lose to it over a long period. And then when I was a teenager, I started trading stocks on the US markets, our high school actually ended at 3.15 p.m. South African time, which was 15 minutes before U.S.
3:22Open. So I could run to the library and trade stocks from the library computers in Cape Town. And this is just what I was interested in. If you looked at my library card, you would see that nearly every book I took out was about business or finance. So it's just been a lifelong long thing for me. How did that path take you into doing this professionally? Well, the big thing I realized after all this sort of childhood trading was I had this very big revelation when I was about 18. I was in London, I was traveling, and I remember going to a bookstore and buying two books on mathematics. And while reading these books, I started to have this revelation that my entire education about finance was actually a junk diet.
4:14I had been reading popular finance books, not quantitative finance books. I had been watching CNBC and the sort of soup du jour information that they give every single day wasn't exactly deep knowledge about how the markets work. And so that's when I decided, okay, well, I'm going to study mathematics. And so that's what I did. I studied pure mathematics and took it up and then started to get what I felt was a much deeper knowledge about how this whole world of finance works, studying mathematics and economics. By the time I graduated, I decided I'd heard about machine learning. This was a really interesting time for machine learning.
4:572012 was when I graduated and there were many breakthroughs. There was a self-driving car breakthroughs. There was just about to be the Google DeepMind acquisition. There were just many things happening that made it seem very interesting. And having a lot of deep knowledge about pure mathematics, it's very easy to pick up learning something new that's technical. And I decided to take a class in machine learning. And then that's what led me to get my first job as a quant working on a global equity strategy. What was that like? It was really lucky. I mean, the thing that was good about it was that the person I worked for actually worked for my dad.
5:38And he was just very nice to me. He kind of knew I was an entrepreneur. He knew I was going to be a little bit difficult to manage, maybe a little bit difficult to work with other people collegially. So he let me work by myself and build from scratch the global equity machine learning model using the data that the firm had. And it took a long time because there's a lot of domain knowledge in finance that's very relevant. There's a lot of jargon and junk in the junk diet part of learning, but there's also a lot of very important things to know about how trading actually works so that you can actually implement a strategy and make a back test translate into a live performance.
6:22So through working with him, I learned that and we ended up making a very good global equity fund that tracked the all country world index and ended up performing really quite well. So in that role, you're building a machine learning model from your elementary roots in self-taught finance. The inputs can come both from the fundamental side of the companies and then trading. I'm kind of curious, what were you putting into the machine and what was it trying to learn? Well, that's a really good point. So there were many funds that were trading with very short-term signals, a fund like Renaissance would classify, which I looked up to.
7:05But what was interesting was if you do short-term trading, your capacity is very low. So if you trade very frequently in very small stocks, you're moving the market a lot and it's very hard to have a strategy that scales. and working with my boss, he said, well, I could see that working, but only on$50 million. I want you to make something that works on a billion dollars because the firm had$15 billion and they wanted to put money to work. So he kept forcing me to say, no, make it work on a longer horizon. Make it work with fundamental data, not technical data. And so I started to develop something quite strange about there's a very different approach to tackling that problem versus tackling the short-term trading problem.
7:50But I think it's more valuable because it's higher capacity. How did you think about what the hypothesis was? Well, that isn't what we do. It was getting a lot of fundamental data and not having a view. It's true that even in quantitative finance, many people form some type of view. They say, well, I think this should be inefficient or these factors should behave in this way. And then they test it and then they say, oh, that worked or that didn't work. That's in some ways the opposite of machine learning, which is a path of saying, we have no idea what patterns are real or we don't have any theory.
8:30We don't have any hypothesis, but we do have a large data set. What can we learn in that data set that will generalize to the future? That is a very different game. And that's the story of machine learning. It's the idea that you can have a theory-free model, a model based only on the data set. So for someone like me, let's say non-quant, somewhat linear thinker, when it comes to forming hypotheses and testing them, is there a way of describing kind of a layman's version of how this works? The way it's set up the problem is that there are these things called features. And a feature could be a factor like value, or it could be something else like EV EBITDA or something sort of adjacent to value.
9:19And if you line up all these features, you can ask the question, do they have any correlation with returns historically? And a lot of them will have correlations with returns historically. But that doesn't mean you've solved the problem. That just means you've proved that these things change over time. And sometimes they're positively correlated and sometimes they're negatively correlated. With a machine learning model, you can say, let's take all these features, and there could be thousands of features, and let's model this target variable of return. If you can regress those features to return and do it in a way that's robust, then you'll have a model that can work out of sample.
10:02So you've done this first run in your career at building these models. What was the path from that to starting to think about Numeri? It was quite early on in this job that I started competing in data science competitions myself. There's a data science competition website called Kaggle. And they listed all these problems and people could go and try to solve the problems with machine learning. And so for anyone interested in machine learning at that time, this was definitely the place to go to try to test your skills on new data sets. What was really remarkable was that every single time a company put up a data set and they said, try to solve this problem.
10:45Like famously the Netflix prize, they said, to try to solve this movie recommendation problem. And as whenever it was done, the data scientists who approached the problem demolished the benchmark. I mean, the benchmark was at some level and the data scientists got way better. And that's an interesting thing that like even a great company like Netflix could have an internal model and have people on the outside quite quickly do a much better job. And that's what got me thinking about Numeri. Could you have a hedge fund that worked this way? Could you have a hedge fund where the data of the hedge fund was completely open and free?
11:25and then anybody could join and do data science on the data. And then you'd have the best hedge fund potentially because you would have the most talent being applied to your problem. You'd have more than any other hedge fund. And so that got me very excited in the early days of thinking about Numeri. How'd you go from that concept to turning it into the beginning of a business? I had a little bit of savings to basically get through the first what I thought would be 18 months. I had$250 ,000 and I thought, okay, I'm going to get through the first 18 months. And by then we won't need any more money.
12:07Turns out I spent that$250 ,000 in about three months and ran out of money pretty much the day of first round seeding the company with a venture investment from Howard Morgan and Bill Trenchard. Howard Morgan was a co-founder of Renaissance, and I just luckily met him a few weeks before running out of money. What do you think the spark they saw investing in an asset management business like yours? I think we were very unusual. I mean, we had this data science system, but all the data on Numurai is obfuscated. So you can't even see what the features mean. You can't even see what the rows mean. But for a machine learning person, you don't really need to know that stuff.
12:57So there was this weird obfuscated data piece to the company. And we had another piece, which was we were paying people in Bitcoin because that was easier. And a company that was involved in machine learning and Bitcoin and quantitative finance all in one in 2015 was definitely quite strange as an idea, but they did see something. They liked that I had this previous work experience and had built a fund before, and then they thought this could be big. Great. Well, let's walk through what this is. And so as a strategy, why don't we begin breaking this down with who are these people that are getting involved in creating what becomes the inputs for the model?
13:46The data scientists are typically some of the best data scientists in the world, frankly. I mean, one of the nice things about starting Numeri was that Kaggle already existed and already had Kaggle grandmasters, people who'd won multiple competitions in different fields. And so Kaggle had kind of networked the data science community already. And so when Numerize started, they all talked to each other about it. They all joined. And I would see people on Numerize leaderboard and be like, oh, I recognize that guy's name. He's a Kaggle user that I've competed against or something. But that's the thing.
14:26Data science is not quant. None of these people were quants. None of them had any background in quant. They're just the types of people who can solve abstract problems to do with data. And that's something people miss about Numeri. People are not coming to Numeri and working on making a trading strategy or something like that. They are modeling data. Just in a similar way, the people at Google Translate who are working on Google Translate do not know all the languages. They just model the data. And that's what's happening at Numeri. And just like Google Translate, they can be a lot better than a professional translator on certain languages.
15:10And Numeri can be better than a professional investor. So how does your process work for getting these people involved? What is it that they're doing? What they're doing is downloading a data set. And the data set is about, I think it's over a million rows long. So it's just millions of numbers between zero and one. You can download it, by the way. Anyone can. You can just go to a website and download it. But it's a million rows long, and it's 2 ,000 columns wide. So you can imagine opening that in Excel or something and looking at each of the features. But the feature names are hidden. You don't know that feature one means momentum or feature two means debt-to-equity ratio.
15:56And then you don't know what the rows mean. But row one could be Tesla. Row two could be Apple. and what you're doing is you're taking this huge data set and you're saying, can I build a model that regresses to this target variable or predicts this return out of sample? And if you can do that with even a 2 % or 3 % edge, that's very good. And you send your predictions from your model that you've trained on this data set. You send them to Numeri and then we trade them in our optimizer. Why are these people engaging in this data science contest? Well, for sure, they find data science fun. They could have a job as a data scientist somewhere.
16:43We have someone who's working at NASA Jet Propulsion Lab. And then on the side, he would train a Numeri model. And so there's a lot of different types of people in these fields where they have to pick up data science skills for their job. but then they want to practice. So that's a big motivator. And another big motivator is just competition. Becoming a master at something is very compelling. And finally, money. Numurai has created the highest paying data science tournament on the internet. We pay out more than Kaggle does. And we've paid out over$30 million since we started. So it's 30 Netflix prizes.
17:28worth of pet prizes. So that's definitely a piece, but we don't know. I mean, sometimes our users are anonymous and we don't even know why they're there or what their motivations are. How does the compensation aspect of it work? In quantitative finance, there's this term information coefficient, which is how good your predictions are at predicting returns. And it's really just the correlation of your predictions with the returns. If your predictions are correlated with returns, that means you have a good model. If your predictions aren't correlated, you have a bad model. So if you have a 2 % correlation on your model, then we pay you based on that score.
18:13But there is something unique about Numeri as well, which is the idea of staking. Numurai lets our data scientists stake their models, meaning they take some money in the form of cryptocurrency and they stake it on their models. And if their models perform well, their stake will grow by that 2 % I mentioned. And if their models perform badly, their stake will be destroyed. They'll have some of their stake burned. And that mechanism is something we pioneered back in 2017, long before most people had heard about staking. And that mechanism, it allows us to have our users have skin in the game. Why would you stake a model you didn't believe was going to work out of sample in live data?
19:06Because you're going to get burned. We're basically borrowing from another hedge fund I admire, which is Millennium. They get all these different strategies and they put them together and they make sure that every single manager is extremely incentivized to do well because they have skin in the game. And that's what we've done. So you mentioned paying out$30 million of compensation. What's the mechanism and the process that you pay that out? Yeah, so we actually used to pay people out in dollars, but we would find that we would have data scientists around the world that said, oh, we don't have PayPal.
19:48You can't send us dollars. Can you please just send us Bitcoin? And that's how we got into paying crypto in the first place. It was just a basic need to pay our data scientists. But since then, we started moving to paying in our own cryptocurrency, which we made in 2017. And so that$30 million, a lot of it's coming from our cryptocurrency, almost all of it. But it's quite liquid. In fact, our cryptocurrency is more liquid than some stocks. And so people will earn some of it. Maybe they'll earn$10 ,000 by staking on Numeri. And then they'll go to Coinbase and sell it into dollars. Or they'll keep staking it on their model.
20:33I'm curious about the persistence of the incentive scheme over time, given the necessity of having these data scientists building the models. What have you seen in terms of people's willingness to keep submitting models if they're talented and how that plays out over time? people do quit but often when they quit it might be because they think their model stopped working and that's another piece of numerae when these strategies decay what you want is for fresh new strategies to come into this system and numerae is constantly having new people sign up old people leave and people try completely new ideas completely new architectures for their models and that dynamism is what keeps it going.
21:21It's also why I talk about Numerize being a kind of AI version of Millennium. Millennium, they do not have all the same people they had 20 years ago working on Millennium. They have a totally dynamic system of PMs coming in and out, and that's how year over year they've produced incredible returns. So for the last couple of years, we've had a big rise and then a fall in the price of all cryptocurrencies. How has the volatility of the crypto ecosystem impacted the incentive system within Numeri? It impacts us a lot less than people imagine. There was a time in 2018 when NMR, our cryptocurrency, was down maybe more than 80%, maybe 90%.
22:08And in that year, our user base doubled. It wasn't like people quit because the cryptocurrency was down. They continued to stake and continue to participate. How have you gone about engaging with this community of data scientists? We have a Discord channel. We have what we call fireside chats, which is once a quarter where we have a chat with all of our community and answer the tough questions that they have. and then we have conferences. We had a big conference in San Francisco called Numicon, where we had probably the largest gathering of Kaggle grandmasters in real life ever. The number of bright, amazing people who came was staggering.
23:00And so that's the type of thing we do, but it's strange. I mean, it's a bit like who's a hedge fund customer, right? Or LPs. But then in some ways, you want to be like, well, wait a second, so are our data scientists. because they're kind of a customer of our website experience and data systems. But then they also kind of work for us because they're helping to build the strategy. That's why I think it's quite a cool company because it's like a modern thing. Like, how is it that you have a quant hedge fund that doesn't build any of their own models and has people that they don't even know building the models and they are getting paid in cryptocurrency?
23:36There's a lot of interesting things to think about. So I'd love to dive into the data set that goes into this research. What is the lump sum of the data that you start with? So we've bought a lot of data. I would call it about 75 % fundamental data. So we're trying to get data that applies for very long periods of time. Like we wouldn't want a feature that didn't go back all the way to 2003, for example. We want features that work in every single market, so available across all global equities. But yeah, it's primarily fundamental data. Things like PE ratio, price to book, EV EBITDA, analyst earnings expectations.
24:25And then there is some alternative data. There's some news data, for example. Is this stock being mentioned in the news type of things? How many different features are in the models? There are now almost 2 ,000. Three years ago, it was 40. We only had 40 features, and now it's 2 ,000. And over that same period, the number of data scientists submitting models has grown from 100 to 5 ,500. What goes into developing a new, say, good feature? Yeah, well, the first part is it shouldn't be correlated with existing features. That's one thing. But there are many other things besides. One thing we don't like our features to do is move fast, like have high turnover themselves.
25:20If we stuck a seven-day momentum feature into Numeri's dataset, the Numeri users would like it and their models would pick up on it. But coming to trade execution, we wouldn't want to trade that fast. And then we wouldn't actually make any money from that. So there's a sensitivity to churn. And then we also like to see features that don't have long periods of not working. Like they should have something to say about the variance of stocks all the time. What are the frequency of the predictions that come in for you to build into the model and portfolio? We get the predictions sent to us now every day, but we only trade quite slowly.
26:06So we'll only turn over the portfolio quite slowly, minimizing our market impact. And we end up holding positions for three or four months. And that's what people find quite unusual. They look at Numurai, like this is a quant fund. Oh, so you're going to be sub one week holding periods. But Numurai, we're somehow doing things on three to four months. And we're still very, very high sharp. So I think that's a nice part about it. Like even though we're playing in this competitive space, we've kind of done things a little bit differently. And that's what's paying off because I don't think people are working in the same region of space as we are.
26:44When you get all of these models submitted to you, How do you then turn the models into a portfolio of stocks? No one's submitting us trading strategies or even trade recommendations. No one's saying buy this stock, sell this stock. They're giving us a signal, which is just a long vector of predictions, 5 ,000 predictions on every stock in the universe. And we take those predictions. We compute the stake weighted average. So what does that mean? Well, if someone's staking a lot, then they really believe in their model. So we give them a higher weight. And if they're staking a little bit, they get a low weight.
27:27And we take the stake-weighted average of all these predictions. And that's how we compute what we call the meta model. But that's still not a portfolio. That's still just a signal on 5 ,000 stocks. And we take that signal. And then we kind of become a normal quant fund again. We say, let's take that signal and pass it through an optimizer. That signal might have a small skew towards a factor, and Numeri is factor neutral. So we will have the optimizer take out country risk, sector risk, factor risk. Before the optimizer runs, you have your signals. Does that then translate to rankings of stocks by attractiveness?
28:11That's exactly right. So by the time we're at the stage of having the Numeri meta model, the stock right at the top is kind of our favorite long. But we won't go long our favorite long unless we can hedge all of its risk characteristics because we run market neutral. So we need to find some kind of short that has the same characteristics as it. So much of when you think about these anonymized models and machine learning feels like a data mining exercise. and I'm curious how you get comfortable with that difference between in-sample and out-of-sample. Yeah, data mining is a very bad rap. I mean, it's exactly the thing you don't want to do.
28:53You really want your models to generalize, but really the point of machine learning is to not overfit the training data. I mean, would you call OpenAI's chat GPT data mining? You can tell it trained on data, but it's doing something very clever with how it modeled it and how it builds a model. And I think that's the exact same with Numeri. People are very good at this stuff these days. No one is looking for a patent that's not going to work. At some point, you can imagine that the models that come in could overlap with each other, have high correlation with each other. How do you integrate that into your process?
Read the full transcript
29:37That's a really good point. So one of the reasons Millennium works is because they can get strategies that are uncorrelated. And that's what boosts the sharp. They have so many different strategies that are all uncorrelated, it starts to become very hard to have a down year. Numurai, it's the exact same. We figured out maybe about a year ago how we were seeing people who'd say submit a model with 99 % correlation to some basic model that we'd released for free. So it's pretty clear that that data scientist didn't really even try to be different to the example script. But we started a new way of paying our users, aside from just their IC, their correlation with subsequent returns, we said, we're going to pay you for being uncorrelated effectively.
30:27So if you make a mediocre model that's very uncorrelated, you can do particularly well because of its lack of correlation is actually helping Numeri more than a good model that's correlated with the one we already have. And so that's a key piece of what's going on right now is people are working on strategies that are, wow, I never thought of that or I never thought of doing that and they end up being like 40-30 % correlated with the average model which is really good. How do they see what the average model is? We do tell them on the website here's your correlation with the meta model and so people are trying to be quite low the best users are quite low correlation and then we also give out the historical meta model signal because we're basically telling you, you have to try and make a model that improves this signal.
31:19Here is the historical signal and they attack that problem. And at the end of that, what does the balance sheet of the fund look like in terms of number of positions? It's about 500 stocks long and 500 stocks short. And a lot of the position sizes are kind of equal weight. Some that are higher than others, but it's basically, we never have more than 3 % of the fund in one name. How do you decide how much leverage to run on the strategy? It's a good question. That is in some ways the magic of quant. Jim Simons would make absolutely no money if it weren't for leverage. That's just how it works. But you want to leverage things that are safe.
32:07So what we try to do is make sure that even on very high leverage, our volatility is way lower than the, say, the S &P 500. When you're building these portfolios and you're so hedged to so many things, it's very hard for things to affect you. For example, in January, momentum, the factor, crashed and a lot of quants did badly. Numeri is neutral to momentum. So we basically didn't even notice this crash because we're hedged to that risk. And we do that with as many things as possible to make our track record more and more idiosyncratic and focused on alpha. I'm curious who's on your team and what they do to make this all happen.
32:53We have about 21 people now full-time. It used to be that the whole company was engineers who were working on the website, right? Like the API, serving data, the data systems. but then it's become over the last few years, very different. We've added a ton of financial experience and business builders. We want to grow a big business now. It took a long time to get the system right. And then about three years ago, we cracked that, but then we're like, okay, let's make sure we have all the business people in place. So for example, a hedge fund like ours, we're trading a lot, something like 37 markets that we trade.
33:34So we need amazing COO. We need amazing trade execution infrastructure and trade operations. So we have that function as well. And then we have the data team and the research team. And the data team is buying data, transforming data, making sure the data set keeps growing. And then the research team working on optimization. Since you launched six years ago, what are some of the adoptions that you made in how this approach has worked? The biggest one was cracking staking. I mean, we had a period where, yes, we were getting a lot of people submitting models, but we couldn't trust them. We couldn't trust that they would keep working out of sample.
34:18We didn't even know who the people were. But having staking was huge because suddenly, oh, wow, I'm going to stop submitting my bad models and only submit my best model because that's the one I'm putting money on. and that just cleared up everything and made Numeri start working. Before then, we really weren't working. Like the first couple of years, things were not working. We were growing our user base, but the rest of it had a long way to go. The other piece is optimization. Being factor neutral, but also neutralizing all kinds of other risks, I think has been very important for Numeri. we have a very, very good drawdown characteristics.
35:00The numerized fund hasn't historically gone down very much in periods of market stress. That's what people want from a market neutral quant fund. How do you think about what your competitive advantage is relative to the market for similar funds? Well, it's for sure a very competitive industry. In some ways, it's maybe the most competitive thing on the planet. But I do think there's a new technology that we're very good at in machine learning. And we have a new talent acquisition system by having this online open data science competition that no one else has. So I think it's very clear to me that for right now, Numurai has a very big edge on talent.
35:50We have people working on our data that aren't even hireable. Investors are so accustomed to be able to call it touch and feel to some extent what they're investing in. A lot of it sounds very black box-like from the outside. And I'm curious how people have gotten comfortable with the strategy given that necessary black box nature of it. Yeah, I think Numerae is in some ways the most black box you could be because we don't even build the models. Our data scientists do, and we don't even know what they built. Like they might have used a neural network or they might have done something else. We don't know.
36:32So we're definitely very black box. I think the way people get comfortable with it is actually quite simple. It's with attribution. Do attribution on a track record and tell me that there isn't skill in it. If you say, wait a second, you did well in March 2020 or other periods of market stress or the meme stock rally. And you said, well, it wasn't because they got lucky taking market risk. They never took any market risk. It wasn't because they're lucky taking sector risk. They never took any sector risk. They never took any factor risk. And you start saying, well, this is a track record that's indicative of skill.
37:14And so the quant allocators in the market are very sophisticated. A lot of the times I'm talking to people who have PhDs in physics or something that are running big pools of capital, and they get that. They know when a track record is reaching the point of being like, ah, there we go. This is now very statistically significant. So, it's odd though. I mean, if you like the idea of something like venture capital, it's like you can see and touch it. You can go to the websites of the portfolio companies that your venture capitalists invested in, go meet the founders, very different type of investment.
37:53But it's very important type of investment, especially in a market, perhaps where venture crypto macro economy is not going to produce the same things it did over the last 10, 20 years. Inevitably, any strategies go through ups and downs in their performance. How have you thought about getting your investors sufficiently comfortable with the unknown, such that if you do go through a tricky period, they'll be more likely to stay with you through it? That's a good point. Well, first of all, the magic of a quant fund or the promise of a quant fund is that it really can do quite well in all environments in a market neutral context.
38:34And they are a handful of funds that have cracked that like millennium, where for really 30 years, they've had one down year. So I think what's nice about this type of strategy is there's not going to be like five years where we're like, oh, I'm not sure the macro economy will turn into our favor again and we don't know what we're going to do. It'll never be like that for this type of fund. But still, even a five-month period of flat performance or down performance can worry investors. What's nice about Numeri, what I like to say is, all the signals responsible for making the bad performance are being burned.
39:14Our data science community is hurting more than you are. They're getting burned because they've submitted these signals that aren't generalizing, and they are going to work really hard to make sure their next signals do work. So we often can't say exactly why we're down, because we're neutral to so many things. Nothing's responsible. for our drawdowns except our alpha. But we can say we have by far the biggest, most sophisticated open hedge fund system in the world. And it's getting better and better every single week. How have you thought about the ability of this system to continue to scale and deliver with larger asset size?
39:58Yeah, that is the trick with quant funds. I mean, most of them just stopped working very quickly because they focused on a small asset amount. Numari, we were just super strict on this. And this was from my experience of working at a fund where I was trying to build a high capacity strategy. I basically told the team, we are never allowed to do a backtest of$50 million because even though we only had 10 at the time, we had to assume we're doing way more. And that discipline forces you into these higher spaces of capacity. The horizon is very, very long. And so that helps. But the other piece is also that more and more models helps too.
40:42You know, we can always grow the data set size. We can always grow our data science community. And machine learning as a whole right now is taking off like crazy. like the number of new research papers being released that apply to Numeri is very large. So I think capacity is kind of like in our DNA. We never wanted to build a small hedge fund. So we've never touched the small opportunities. We've always gone for the larger stocks, larger horizon. What are some of the other applications of this combination of, let's just say atypical inputs in the investment world outside of the investment world?
41:25I have often said I don't think there is application outside of the investment world, but I could be because that's been my whole life, as I told you, thinking about the problem of investing. One reason in particular, like you could say, what if there was a numerae for detecting cancer from MRIs? And there was all these data scientists working on solving that problem? Well, the problem is that problem is very easy to solve. With a single model, you can get very high accuracy with that type of image detection. That's sort of like a solved problem. And therefore, why would you need to do crowdsourcing?
42:02That's a solved problem. But stock market is never solved. It's a permanent race to have state-of-the-art systems and state-of-the-art data. So I think there's something unique about Numeri being the right domain. And then also the edge in finance really matters. Numeroid has good data scientists, we could probably build our own models and say we could get a 52 % edge. But to go from 52 to 52.5 is massive in terms of your Sharpe ratio returns, volatility. And so the fact is, it's just one of these industries where a tiny bit helps. And so you always want to be open to having new people join and help.
42:46What areas of research and development are you most excited about within the business? I think there are some times where the most hyped thing in the world should be completely ignored. If there's something very hyped out there, it's probably not worth even looking at. But I actually do think that right now, large language models, which is probably the most hyped thing in the world, is actually something I want to have Numeri become the best at. And I think we have a shot at it because machine learning is so in our DNA. So I think that's something I'm excited about, finding a way to make new features out of large language models to put into the Numeri dataset.
43:33said. And how do you drive that forward in the incentive system for this group of developers you may or may not know? Well, for that project, it might involve some internal work with our own internal data scientists, because we would be the ones building the feature to add to our data set as a new feature. But absolutely the case that other data scientists could do similar things and probably are already. So it's a very interesting time for machine learning. And I think people are going to be very surprised like the difference between having an AI hedge fund, a good one in your portfolio versus not.
44:15I know it sounds self-serving, but the difference will be very stark in the long run because there is going to be basically a wave, a third generation of hedge funds that do things differently to others. that I'm quite excited to see. All right, Richard, I want to ask you a couple of closing questions. What is your favorite hobby or activity outside of work and family? Surfing and painting. Not at the same time. I like surfing because it's like, somehow it's like exercise plus meditation, plus a cold plunge. It seems to be kind of like perfectly healthy. What type of investment, or in your case, maybe model, do you gravitate to like a moth to the flame?
45:00I think the really, really uncorrelated returns that are neutral to all known risks. I mean, the classic quant market neutral portfolio, that to me is by far the most impressive thing. If a PE fund makes 40 % in a single year, because they were actually holding positions that were two times market beta, and the market went up, that doesn't have anything close to the investment skill of a quant fund that's making money kind of out of nothing by taking no risk or as little as we can. So that's definitely what I'm going for. What's your biggest investment pet peeve? I think in the investment industry, there's a thing Warren Buffett says, you'd rather be a good promoter than a good investor because good promoters raise all the capital.
45:52I hope that doesn't stay true, but I think that can be true. And I've lived through in my career the last seven years, a very, very big bubble where investors were drawn to the very bubbly things that had the best promotion versus investment track record. Which two people have had the biggest impact on your professional life? I would say the first one would be Mark Beckenstrada, who was my first boss. just like he's just sort of somehow knew how to manage me and how to teach me and get the best out of me in my first job. And then I would say the second one would be Howard Morgan. I don't think Numerae would exist without him investing and just always investing.
46:36I often think I'm one day going to do something that he's going to stop liking us or something, but he's always very helpful and obviously super knowledgeable with his background at renaissance and in so many other hedge funds. What was the most challenging moment in your career? I would have to say 2018. The whole period was difficult. Cryptocurrency stuff wasn't working. The fund wasn't working. The team dynamics weren't working. The politics of the company wasn't working. And so it just felt like this is not a good spot. And I felt like, wow, I thought I was going to be like a good entrepreneur.
47:15I thought like this wouldn't happen to me. But I think it happens to all entrepreneurs that they're the pretty bad times where everything's going badly. But I had a friend come up to me and say, and I think he thought he was trying to be helpful. He was an investor and a friend. And he said, we could invest in something else. You could close the fund and we could invest in something else. We could work on something else together. And I was just, I just looked at him so angry. Like he was basically giving me permission to quit. and I didn't want to quit. In some ways, that moment was quite galvanizing.
47:47And then he looked at me again. He's like, oh, wow, you're just really never going to quit, are you? What teaching from your parents has most stayed with you? My mom used to tell me all the time as a kid, you can do anything you put your mind to. And it's a kind of a, in some ways, a very American statement. And I just think it stuck with me. I really believe that. And I don't know the amount of cynicism or growing up has changed that belief. So something she did right, getting me to believe that, I think. All right, Richard, last one. What life lesson have you learned that you wish you knew a lot earlier in life?
48:27People stuff is real. Like managing people, management is a real skill. and it's not all math, even though it's a lot math. Knowing that there's really good ways to run a company and be a manager and be kind and all of that, I think matters a great deal. Richard, thanks so much for sharing this really unique story. And I don't use the word unique lightly. So very interesting. And thanks so much for the time. Thank you, Ted. Thanks for listening to this sponsored insight. Sponsored episodes are paid opportunities for another 12 managers a year to appear on the podcast. If you're interested in telling your story in front of the largest audience of investors in the industry, please email us at team at capital allocators.com to apply for one of the slots.
49:29Thank you.
From the publisher
Learn More Follow Ted on Twitter at @tseides or LinkedIn Subscribe to the mailing list Access Transcript with Premium Membership
