E351: Why Most Family Offices Fail (And How a16z Fixed It)

0:00Many of these firms will charge you a relationship fee, some sort of flat fee based on your AUM. 50 bps, 60 bps, no matter what you do. So let me ask you, if you made the same money doing something quite easy and highly profitable, or something very difficult and questionably profitable, which one would you do? One of the big trends and alternatives is the rise of a taxable investor. Why is that such a recent phenomenon? And why is asset management not focused on the taxable investor? The core of the total assets in the U.S. markets is taxing. Let's say you and I are running our own asset management firm.

0:36And I told you, OK, we need a billion dollars to get to a reasonable level of profitability. And I'm offering you two paths to get there. One is find 10 institutional investors at 100 million each. Or two, let's find 1 ,000 people that will raise a$1 million check. And I don't even know where to go to find those. And if they do go through intermediaries like banks or IAs, they're fiercely guarding those clients. They're not going to sort of propose that those people invest in your fund. You came from Jordan Park where you were CIO. And Mark and Ben gave you this interesting thought experiment of how would you build a multifamily office from scratch?

1:10What were your first ones? So, Michelle, you're the CIO of A16Z Perennial. We're here at Andreessen Horowitz, built this massive venture capital firm. Why take the time and maybe the attention to build a family office? Well, several reasons. I think reason number one is a deep commitment to the founder community and the idea that this firm helps founders in every dimension of their startup. So why not extend that also to their personal financial world and therefore ostensibly let them focus more on their startup and less on these sort of mundane issues? Addition through subtraction? Addition through subtraction.

1:45I like that. I like that. That's good. So that's one element. Another one is just a community building element. I think these people like to interact and be together. And so building a community is valuable and who else to build a community than the largest venture community there is. And then third, they would look at to the LPs they have in their own big funds and they would get, you know, the big pension funds, the some wealth funds. They'd have these very professional teams and they'd witness that. And then they turn and witness their wealth team. And they'd realize that the wealth team really weren't investors by training, right?

2:14They were more sort of service providers. And what they wanted was investment acumen. And so they figured they couldn't be the only ones that had this problem. And if they fix that, it probably would be valuable for many people. The idea that wealth management isn't necessarily the best investment management, not very controversial to me or my listeners. Why build a multifamily office? Building a single family office is not as easy as you might think. So on the side of the principle, I think many of these principles, their dream is not to be the CEO of an asset management company, right? And manage people like me, right?

2:49So that's the first step. If you are really going to run a single family office properly, you're going to be the CEO of an asset management company. You're going to have to hire people, motivate them, give them a career path, et cetera, et cetera. And that's not something many of the founders or many people in general want to do, love to do. They'd rather focus on their startup or whatever they're doing or their next company. So I think first there's a desire element. And I think, in fact, that's a lot. I've seen a lot of single family offices start and then stop when the principal realized that their job is actually to be a manager.

3:18right and their whole sort of idea was i got wealthy and now i'm suddenly a manager like i wanted to detach myself from that not implicate myself even more so that's on the side of the of the principle and then on the side of the investment professional um why would i leave if i'm really a top-notch investor at a large asset manager making millions of dollars a year what's the appeal for me to go to a family office it has to be that i've got growth opportunities i can do more investing interesting investing um i can maybe build a team i can do things that are not there are more difficult to do in a more constrained environment.

3:49And often when they arrive at the family office, and I've seen, again, many of these stories, it's not at all clear how much discretion they have or authority they have. And it sometimes takes years and years for the principal to get to a place where they're comfortable giving some leash to this person. And so as a result, you see often the CIOs come and go. You see these revolving doors because both sides have misinterpretations of what the job is and what it requires. It's interesting because that also applies to endowments. So you might get a fresh pool of capital, you deploy it, and then you're in management mode for sometimes decades.

4:21That's right. So your fund is fully deployed. You're the CIO. You've got nothing to do. You're not in the latest trends anymore. You're not honing your skills. And so having a small static pool of assets, and by small, I mean a billion or less, call it, it becomes very difficult to keep a team. And I would argue that to do properly a multi-asset global kind of team with all the ancillary functions around taxes, you probably need about 20 people, I'd say at a minimum. And so if you're really going to do this, you know, equivalent to a real asset management firm. And so that requires quite a bit of assets and it's quite a bit of headwind on your returns.

4:56So you need to have enough of a pool to do that. I'm not saying you shouldn't do it. I'm just saying that it's, you know, only large pools of assets should really attempt to do this. Now you do see families that are smaller and that try to do this, but they'll do it in a specific asset class where they have expertise or knowledge. So maybe, you know, your smaller family, I don't mean really small, but you're a small billionaire family. Maybe you're good at PE for whatever reason you're operating business. So you hire a PE person and you do PE in your space. That's fine. But that's different than building global multi-asset portfolio.

5:24Right before we start recording, you said something that surprised me, which is asset management scales. So as you get more capital, you actually get efficiency of scale. I've been led to believe size is the enemy of returns. How do you square those two ideas? A lot of LPs investors don't appreciate the fact that scale benefits them too. They think that scale only benefits the GP, but I think that it benefits the LP too in several ways. One is, of course, the ability to sort of throw your weight around seed strategies, ask for concessions on terms, right? So there's that element. There's also the risk management monitoring and reporting element, which becomes very, very difficult when you're dealing with dozens of portfolios that have completely different allocations, right?

6:08And as a result of having that difficulty, it's very hard to maneuver and control something when you don't have a good read of what it is, right? So you don't really know what your risk exposure is. You don't really know what factors you're exposed to or whatever. And so as a result, it's very difficult to do course corrections or be opportunistic or whatever. So having scale allows you to build all those systems and infrastructure that then enables the flexibility to do that stuff. So I think it's valuable for both. I think it's a curve that maybe looks something like an upside down U. There's a point where you're so large, right, that your minimum check size becomes really big.

6:41And so now you're in highly competitive deals. And so your sort of margin gets eroded. But there's definitely a sort of a scale optimum, probably in the, I don't know, 15, $20 billion size. Expert calls have always been one of the most powerful ways to build conviction, but today, investors are asked to cover more companies, move faster, and do it with leaner teams. With AlphaSense AI-led expert calls, their TGIS call service team sources experts based on your research criteria and lets the AI interviewer get to work. The magic is in the AI interviewer, purpose-built and knowledgeable-based information to conduct high-quality context-rich conversations on your behalf, acting as a trusted extension of your team.

7:22Then they take it one step further. Your call transcripts flow natively into your AlphaSense experience and become querible, searchable, and comparable. So your primary insights plug directly into earnings props, digital work streams, and pitch books with zero tool switching. And with AlphaSense expert call services, the AI-led expert calls are just one option because we know the importance of a hybrid expert research approach. AI for coverage and efficiency. Humans for complexity and conviction. It's the institutional edge that scales research without scaling headcount. For hedge funds, that means validating thesis assumptions across dozens of experts before earnings instead of a handful.

7:59For private equity, it means faster pre-IOI scans and deeper commercial diligence. For investment banks and asset managers, it means pulling real operator perspectives straight into models and sector positioning without disconnected tools or manual handoffs. All of it lives inside the AlphaSense platform, trusted by 75 % of the world's top hedge funds, alongside filings, broker research, news, and more than 240 ,000 expert call transcripts, turning raw conversations into comparable, auditable insight. Take advantage of AlphaSense AI-led expert calls now. The first to see wins. The rest follow.

8:35Learn more at alpha-sense.com slash how I invest. I want to go back to this concept of wealth management versus investment management. Maybe double click and maybe give very specific ways how there's a principal agent problem. Is it just on the fee and selling products or is it fundamentally something on a portfolio basis? So many of these firms will charge you a relationship fee, some sort of flat fee based on your AUM, 50 pips, 60 pips, no matter what you do. Okay, so let me ask you, if you made the same money doing something quite easy and highly profitable, same revenue, right? Doing something quite easy and highly profitable or something very difficult and questionably profitable, which one would you pick?

9:16I would probably do the easy, but my brain is such I would figure out a way to systematize the easy and keep on going hard. But that then requires you to go hire a team of people who know how to do alternatives. We're back to that problem of how do you manage that team? How do you incent them? And so on. So you're going to have to have a scale and you're going to have to have an investment DNA in the org. So many wealth management firms don't have that. It's very hard to attract. Let's use a physician analogy, like you're trying to run a physician practice, but all the leaders don't know anything about medicine and don't care about it.

9:47Would you join that? Or would you join something that's run by other physicians who appreciate your art and your craft, right? So I think it's very difficult inside a well firm to even attract the right talent to start with, because you're going to be sort of a fish out of water. I kind of liken it a bit to these AI recruiting wars. There's probably a couple thousand, maybe 10 ,000 elite AI researchers in the world. And this is the pool of people. So they're being picked up. I think the same as investment management. And the reason I know that is when I meet with people, usually they talk about the same 50 to 100 people that are really good in the space.

10:17So there's a finite amount of resources and finite amount of talent. And if If you don't have that DNA, you don't have the resources, you don't have the incentives, you're not going to recruit them. And also, if you recruit one of them, they're going to leave. So you need a certain critical mass as well. You need critical mass. You need growth. You need ability to participate in the current trends and whatnot to keep these people engaged and interested. One day, maybe I'll replace all that. But right now, not yet, luckily. One of the big trends and alternatives is the rise of the taxable investor.

10:44Why is that such a recent phenomenon? And why is alternatives and why is asset management not focused on the taxable investors? historically. I was just looking at these statistics yesterday just to refresh myself, right? Something like a quarter of the total assets in the U.S. markets is taxable. So the vast majority is not taxable. That's one, right? So if you're running a firm, obviously first place to go is the bigger pool, right? The bigger TAM. Second thing that's important also is, let's say you and I are running our own asset management firm and I told you, okay, we need a billion dollars to get to a reasonable level of profitability.

11:17And I'm offering you two paths to get there. One is find 10 institutional investors at 100 million each. And by the way, just by doing a simple Google search, I can identify 300 institutional investors that could cut those checks. Or two, let's find 1 ,000 people that arise a$1 million check. And I don't even know where to go to find those. And if I do go through intermediaries like Banks for Eyes, they're fiercely guarding those clients. They're not going to sort of propose that those people invest in your fund, right? In fact, they will actively try to convince the clients not to do your fund, right?

11:45Because they're being paid based on how much AUM they're monitoring for you, right? in terms of relationship fee. So if I were to ask you that, I think everyone would answer, I'll go for 10 clients that I can easily identify and I only have 10 relationships to manage, right? So there's that element. And of course, now that there's been this sort of explosion in families and taxable individuals that are of institutional size, there's now, of course, a lot more interest in this as a new growth market because that other market is fairly saturated, right? So now all the endowments, pension funds, a lot of them, to our earlier conversation, have deployed.

12:18They have picked their 15 to 25 managers, and now they're only adding out the edges. And now the new capital coming on, you'd say the taxable investor. Also, there's obviously the sovereigns coming on, but taxable investors are one of those net new investors. They're the marginal dollar coming into the market. That's right. You came from Jordan Park where you were CIO, and Mark and Ben gave you this interesting thought experiment of how would you build a multifamily office from scratch? What were your first principles? The first principle was to get rid of all these principal Asian misalignments that are out there.

12:47Second principle was, to my point earlier about wealth management firms, we want to be investor-centric because what we're doing is giving you advice on your portfolio. And so, to use a medical analogy again, if you walk into a building and it says hospital on the door, you kind of expect there to be physicians inside, right? Same thing here. If it says wealth management, of which asset management is part of that, you would expect asset management people to be inside of there. So many of the people on the team have had direct investing experience, have been professional investors before being allocators and portfolio constructors, right?

13:19And what do I mean by professional investor? At some point, they were paid on the P &L of their investments. That was all that was driving their compensation. And that discipline teaches you a lot of things around how to invest. It also gives you the flexibility to implement investments either through third-party management or directly. If you don't have the direct experience, you're limiting yourself to only doing the fund-to-fund approach, right? And that's okay, but it's an expensive way to do things. On a look-through basis, a typical endowment foundation portfolio, you're probably talking 400 basis points of fee drag.

13:51So anything you can do direct, you're going to reduce your fee, and ostensibly you can even do it more tax efficiently. It's important to state the opposite of what you did. So if you bring in investment managers, doesn't that sound obvious when everybody do that? Well, the industry practice is you bring somebody with a book of business, with relationships, and you focus on who has the most clients, not who is the best investors. Those are kind of the two archetypes. And this one has really dominated the last 40 years. How do you look at a private and public portfolio for an individual? And how do you weight those?

14:19So that's another thing where Perennial is very focused on being customized. And so if I'm trying to, so I just said a minute ago, scalability can benefit the LP, but there's some instances where scalability is solely for the benefit of the GP. One of those is running playbooks. So I look at you and I put you in some bucket. I'll say, you know what, David, you're like a middle risk kind of guy. So here's the standard profile I'm going to deploy for everyone just like you. and I'm not going to take the time to understand your tax situation, your personal trusts, your personal preferences, nothing like that.

14:52I'll just build kind of a standard portfolio for you, right? So we don't do that. We'll sit down and try to understand. And there's an element of psychology in this, trying to understand your risk profile, your preference, your goals. Like, do you want to have some sort of large philanthropic gift later in life? And so on. And the psychology also takes root in the portfolio and in the returns. The strategy is only as good as its implementation. It's only as good as implementation. And if it doesn't meet the goal of the client, and there's an interesting analogy there to do with endowments and foundations, then it's useless.

15:21Early in your career, you spent time at McKenna, which was a spinoff from Stanford's endowment. What did you learn at McKenna? So before McKenna, I was in a hedge fund. And before that, I was serving private equity clients. So I was much more of a direct investor. And then McKenna was my first sort of foray into being an allocator. And I was what they call there the strategist, which was sort of the number two to the CIO. This was the team that spun out of Stanford, right? And so I got to see what a top-notch endowment does, what professional high-end manager selection does, and all the good things around how a portfolio should be managed, monitored, risk management, all those kinds of things.

15:56And then from there, then I went more to the family office world. And, you know, the difference is quite stark in terms of the asset management quality, but also in terms of the attention to tax and individual goals, right? where the endowment foundation world is a very specific goal in mind, whereas individuals have much more varying goals. You said you got to see what excellence looks like at McKenna and what excellent asset portfolio and construction is. Double click on that. What does it mean to be elite when it comes to portfolio construction? I wouldn't necessarily say it's portfolio construction as much as it is having access to the top managers.

16:34If you look across all the top endowments, You know, I could rattle off a dozen managers that every single top endowment and foundation is in, right? So I think there's broad agreement. Consensus. Consensus. Groupthink, if you don't want to be around what a sort of standard portfolio should look like. And then people sort of tweak on the edges around the endowment foundation world. And it's reflected in the returns. If you look at the returns, they're all clustered. Three, five, 10 years, eight, nine, 10%. They're all pretty tightly clustered. Ex ante, if you didn't know what I just said about, you know, kind of this sort of consensus, 10 different portfolios, you would expect quite a bit of dispersion.

17:13And in fact, they're quite tightly clustered, right? And between that 8 % and 10 % roughly spread across them, right? If you assume that the returns across the different endowments are sort of more or less uniformly distributed, that means every 50-bit piece across that 2 % is like a quartile, right? So 8 to 8.5. The bottom quartile, 8.5 to 9, you're now at the middle. This is over 10-year periods. Over 10-year periods. And the other interesting thing, right, for endowments is the compensation structure is a sort of self-reflective compensation structure because you're paid based on your performance relative to other endowments.

17:44And since they're all super tightly clustered, effectively what you're doing is you're telling one endowment manager, you will get a bigger bonus if you manage to move your return. Because 50 bps is your stream median and top quarter, something like that. I'm making a lot of assumptions here. And so what does that mean in terms of portfolio difference across the United States? It's a small tweak, right? You're not going to have this massively different asset allocation. That's why when you look at their asset allocations, they're not that different. You don't have someone with 50 % venture, or it's pretty rare if you do, right?

18:16And I would argue, again, for individuals, it's a different story. For endowments, right, what's the goal? The goal is to meet your payout, 4, 5, 6%, hopefully not compromise the corpus, right? So maybe you keep the dollar amount steady or maybe even keep the real value of it steady. So if you assume like a 3 % inflation, you need to get an 8 % or 9 % return on the portfolio. Because you're distributing 5%. Distributing 5%, you got 3 % of inflation. Lo and behold, they're generating right that. So it's not when people say, oh, endowments care about returns. They care about risk-managed returns, right?

18:48It's more of a risk management exercise. It's not a maximized return. And the reason they're not doing 50 % venture and trying to be top quartiles is because there's also downside risk. You lose your job. You lose your job. They're trying to optimize just enough to get their bonus, but not necessarily enough to lose their job. And of course, all this is subconscious. Subconscious? Okay. Okay. Before I get in trouble, Dell PCs with Intel inside are built for the moments that matter. For the moments you plan and the ones you don't. Built for the busy days that turn into all-night study sessions.

19:17The moment you're working from a cafe and realize every outlet's taken. The times you're deep in your flow and the absolute last thing you need is an auto-update throwing off your momentum. That's why Dell builds tech that adapts to the way you actually work. Built with long-lasting batteries so you're not scrambling for the closest outlet and built an intelligence that makes updates around your schedule not in the middle of it. They don't build tech for tech's sake. They build it for you. Find technology built for the way you work at dell.com slash Dell PCs. Built for you. Dell PCs with Intel inside are built for the moments that matter.

19:56for the moments you plan and the ones you don't. Built for the busy days that turn into all night study sessions. The moment you're working from a cafe and realize every outlet's taken. The times you're deep in your flow and the absolute last thing you need is an auto update throwing off your momentum. That's why Dell builds tech that adapts to the way you actually work. Built with long lasting batteries so you're not scrambling for the closest outlet and built an intelligence that makes updates around your schedule not in the middle of it. They don't build tech for tech's sake. They build it for you.

20:28Find technology built for the way you work at dell.com slash dellpcs. Built for you.

20:37Support for today's episode comes from Square, the all-in-one way for business owners to take payments, book appointments, manage staff, and keep everything running in one place. Whether you're selling lattes, cutting hair, running a boutique, or managing a service business, Square helps you run your business without running yourself into the ground. I was actually thinking about this the other day when I stopped by a local cafe here. They use Square and everything just works. Checkout is fast, receipts are instant, and sometimes I even get loyalty rewards automatically. There's something about businesses that use Square, they just feel more put together.

21:10The experience is smoother for them and it's smoother for me as a customer. Square makes it easy to sell wherever your customers are, in store, online, on your phone, or even at pop-ups, and everything stays synced in real time. You can track sales, manage inventory, book appointments, and see reports instantly, whether you're in your shop or on the go. And when you make a sale, you don't have to wait days to get paid. Square gives you fast access to your earnings through Square checking. They also have built-in tools like loyalty and marketing. Your best customers keep coming back. And right now, you can get up to$200 off Square hardware when you sign up at square.com slash go slash how I invest.