Erotic ChatGPT, Zuck’s Apple Assault, AI’s Sameness Problem

0:00Chat Shippie Tea is getting spicy in the chat room. OpenAI's latest revenue numbers are in. Zuck poaches another Apple executive, what's the goal here? And is it time to call out all the work slop? That's coming up on a Big Technology Podcast Friday edition right after this. Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called Chat Concierge, and it's simplifying car shopping. Using self-reflection and layered reasoning with live API checks. It doesn't just help buyers find a car they love. It helps schedule a test drive, get pre-approved for financing, and estimate trade in value.

0:42Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One. Welcome to Big Technology Podcast Friday edition, where we break down the news in our traditional, cool-headed, and nuanced format. We have really a fun show for you today, a great fun show for you today because finally Sam Altman has relented and allowed ChatGPT to get spicy with adults. We're also going to talk about OpenAI's revenue numbers. We're going to talk about Zuck and Apple. We might get into AI sentience. Who knows? It's going to be crazy. Let's let it go off the rails and joining us as always on Friday to do it is Ranjan Roy of Margins.

1:19Ranjan, great to see you. Oh my God. Today, I'm a little nervous. This is going to be interesting. Sam, ChatGPT and Erotica. Let's go. It's a great day for me because I've been talking about this as a thing that's going to happen for a while. And, you know, I think some of us, wink, wink, didn't want to go down the AI erotica path. But you have no choice now. It's a thing. Take your victory lap, Alex. AI erotica. Your AI erotica victory lap. Usually we get to this stuff at the end, but we're going to just start with it at the beginning today. By the way, I just am happy with my chat GPT is getting spicy in the chat room leading.

1:58I wrote that and I felt really good about it. Okay, so let's talk about what's going on with chat GPT. Sam Altman puts a tweet out this week on Tuesday. Of course, the OpenAI CEO, he says, we made chat GPT pretty restrictive to make sure we're being careful with mental health issues. We realized this made it less useful and enjoyable to many users who had no mental health problems. but given the seriousness of the issue, we wanted to get this right. I will skip the rest of the tweet like many people have and get to the news. In December, as we roll out age gating more fully as part of our treat adults like adults principle, we will allow even more like erotica for verified adults.

2:41It's not just the fact that OpenAI is getting into erotica. It adds to questions of What does it say about its need to grow? Questions about whether it actually is close to AGI. But first of all, let's just get your immediate gut check reaction here, Ranjan. How do you feel about this? Okay. Before I get into how I feel about it, I actually think you skipped over some of the important parts of the tweet. There's two parts that jumped out to me. So first, he actually talked about in a few weeks, we plan out to put a new version of ChatGPT that allows people to have a personality that behaves more like what people liked about 4.0.

3:19Now, remember, it was the sycophancy of 4.0 and that everything, the gushing, you are great, you are amazing, what a great idea, that they tried to tone down that people complained about. So that actually starts to worry me even more because we're not just talking about erotica here. we're talking about sycophant erotica. It's like that was the part that everyone made an uproar around 4.0 in the move to 5. And the fact that they're still calling that out kind of worries me. But then what really was interesting is he then says, if you want your chat GPT to respond in a very human-like way or use a ton of emoji or act like a friend, chat GPT should do it, but only if you want it, not because we are usage maxing.

4:06Usage maxing completely jumped out to me. We've been talking about this a lot around how the way they have it speak to you and give you constant prompts to keep going and running the conversation feels like a growth marketer decision as opposed to an actual effectiveness of the platform. And the fact that he even used that term is a reminder that they realize like that is part of what they are doing. The fact that he says not because we're usage maxing almost makes me convinced that that's exactly why they're doing this. And it's not about treating adult users like adults. But I think overall, I am terrified of this.

4:49Longtime listeners will know of my friend on Labor Day went down a deep flirting with chat GPT rabbit hole. And I as I listened and it was terrifying. So like God knows the Pandora's box that we're opening here. Are you happy about this? You know, only from a content perspective. I think it's still unclear what the impact will be as people develop more and more romantic relationships with AI. I think the thing that I am happy about is that it's finally out in the open. Like this was going to happen anyway, whether it was ChatGPT or some other app that uses a GPT model with less guardrails. This is going to happen.

5:31And now it's come to a head and it's really a moment where humanity will have to reckon with the fact that more of us are going to get into relationships with more of them. And what does that mean? Okay. I will say on the sycophancy thing, one takeaway for me is that words of affirmation, it's the forgotten love language. It turns out that people really, really like those words of affirmation. And of course, ChatGPT can do some of the love languages like quality time. It can't do touch. It can't. Maybe it could do acts of service when it suggests things for you. But words of affirmation, I think it really is the forgotten love language.

6:11So it's getting its due today. That is a wonderful point, Alex. I really appreciate that incredible logic and rationality that you put into that point. Let's dig into a bit more. That's my sycophant chap GPT impression right there. I'll give you that it's out in the open and we have to reckon with it. So, okay, I'll give you that that is a good thing because we've been talking about companionship for a long time. It's been an uncomfortable discussion at times, and now everyone has to have it. So I'll agree that that's good. But, yeah, this still terrifies me. Actually, even to break down his tweet further, he had talked about how we made ChatGPT restrictive to make sure we were being careful with mental health issues.

7:00And then he says, now that we have been able to mitigate the serious mental health issues and have new tools, we are going to be able to safely relax the restrictions in most cases. It's like a checkmark. We're done. We're good. Mental health issues with ChatGPT solved, where in reality, this is just beginning. So I'm curious, like what are these new tools that they have or is any of that clear? I am so glad that you're reading this tweet with the level of detail that you are. And forgive me for skipping over these very important points. This is the most substantive tweet ever. I think in OpenAI history.

7:39Yes. No, I think you're right. And I think going back to your point about usage maxing, right? I mean, OpenAI is very aware that ChatGPT is the fastest growing app of all time. 800 million weekly active users right so i think that like while they're they may not be actively trying to usage max they don't want to slow down that adoption i think that adoption is also you know central to their um to their their fundraising pitch and to go back to the tweet here uh i don't know how they could be so confident that they have solved these problems systems we may talk about it later but this is still let's just talk about it now this is still technology that we don't really understand the insides it's not controllable in a way that you can control more deterministic technology so for me it's a big we'll see here I don't know if we can trust this company fully for sure not given what we've seen already to be able to say that they have safely mitigated all the potential mental health issues.

8:45So spot on to call that out. Yeah, I mean, there's no way they have. And there's been, I mean, reporting after reporting around, like, I mean, really awful things that have happened with people who went too far down the chatbot rabbit hole. So I think, like, and you mentioned the trust. It is interesting, because we're at a moment, I feel, generative AI in large language models that we went from this assumption that they hallucinate and everyone kind of joking. And it's almost a afterthought that, yes, chatbots hallucinate to a world now where most people I know are a lot more comfortable with the assumption that they don't or that they're somehow usable and responsible.

9:30So does is this going to completely backfire on them? Or is this going to, because trust is kind of paramount to the central. Every time you ask ChatGPT a question, you're assuming it's at least relatively correct and responsible about how it's answering you. Does this actually potentially hurt their regular usage? So, okay, I have two thoughts on that. First, let's not underestimate or de-emphasize the fact that these models have gotten much, much better. Just if you think about the level of hallucination, like I'm going to read out a list of Apple researchers that have left for meta over the past couple of months.

10:15And I did a chat CPT query. It had all the researchers' names. It was accurate. God help me if it wasn't, but it had links. I followed the links, confirmed the links, and it was right. So we have seen much better. These models have, as they've gotten better, they have hallucinated a lot less they are much more trustworthy and it it bears out in the data at least Matthew Prince from Cloudflare talked about how people people don't need to go to the footnotes as much as they did anymore the problem is of course becoming too trusting of it like if it's getting 95 % of the things right and you trust it like it's 100 % you're going to make some big mistakes so but they let's I don't think we should downplay the increase in in trustworthiness there And then the second thing is I expect this erotic or loving companion feature to be extremely popular.

11:07And I think this is important. We shouldn't glance over it. Your relationship with technology changes a lot when you view it as a friend or a lover. And that trust thing, I don't think you'll ever be able to have. you'll ever put more trust in a technology than when you view it as a buddy or a girlfriend. And this is getting into, and again, I'm glad we're talking about this. I'm glad in a way that this has been, the issue has been forced because this is going to open up so many more really important questions about the relationship that we have to OpenAI's technology and the responsibilities that open AI has to us.

11:53What do you think this actually, okay, technology aside, but societally, what does this look like in day-to-day relationships? Like now you start dating someone, do you have to disclose your AI companion? Yeah, so I have an AI companion. I just want to make that, get that out up front. And I would like to remain with them as we progress in this relationship. I am married. You are married too. Do you get an AI companion? And that's kind of, you have that open discussion. Like, what does this actually look like in human interaction? It's mind-blowing how weird that's going to get. Definitely. So I'm old school on this front for sure.

12:36And I believe that, yeah, if you have a relationship with an AI, you should disclose it. if you're in a relationship with a person um i also think so old-fashioned very old-fashioned yeah uh i mean we both talked about the south park where the guy is in bed with his wife and like talking to chat gpt and like basically comparing it favorably to his wife who's like turned to her side and ignoring him uh so i think people will get into those situations but look if this is let's do it are we ann landers now um doing our advice advice podcast This is a relationship podcast now. If you fall in love with AI, first of all, when you're on the way there, probably disclose, but don't keep that a secret.

13:19It's all about communication and openness. That's the secret. Just be honest. Now, let me ask you this. This is sort of off the rails question, but I feel like why not tackle it? I mean, could this potentially be good for society? You think about the loneliness problem. We as humans have not been doing a good job being in community with others. I'll put it that way. If ChatGPT can become an effective companion or romantic partner to people who otherwise cannot find it in the quote-unquote real world and makes them happy, maybe that's good. Yeah, but to me, it's not effective in the sense that it always agrees with you.

14:06Again, in the fact that he said 4-0 was good, sycophancy was good. We're going back to that. Well, did he say that it was good or that people want it? Oh, no, people want it. People want it. That's very different. Okay, fair, fair. He said people want it. And yes, it's human nature that you want something that agrees with you all the time. But I have never had ChatGPT tell me, actually, that's a terrible idea. Again, South Park was just so spot on where they're like, I think it was like a French fry salad. And it's like, that's a culinary adventure. um like it only will tell you that you're right and good which most other humans don't do um so like in terms of actually totally distorting how people can actually form any normal human interaction it'll distort the way you approach that like even my son with alexa pre-alexa plus which we've talked about over the last few weeks but in the old school just play me a song what's So whether like you could see how demanding he would become around it and like expecting that this thing does whatever I say.

15:15So like the more people start to kind of associate that as a relationship and like a friendship and interaction, that is even, as I'm saying this now, even more terrifying. So yeah, effective is a broad term there. It's also never as clean as I suggested as you're talking, right? are people going to basically deprioritize their friendships with people that keep it real with them for AI, which doesn't. That's why we keep it real here. No sickening to see. Exactly. No, this is an enduring friendship. The AI doesn't threaten us, I hope. I don't know. I'm going to start podcasting with ChatGPT and Notebook LM soon.

16:00Okay, so let's talk about what this is actually going to do for usage, because the usage maxing thing is interesting, whether this is going to lead to an increase in usage or a decrease in usage. And Mark Cuban, none other than Mark Cuban, brought up a really good point. He said, this is going to backfire hard. No parent is going to trust that their kids can't get through your age gating. They will just push their kids to every other LLM. Why take the risk? Same with schools why take the risk a few seniors in high school are 18 and decide it would be fun to show their hardcore erotica they created to the 14 year olds what could go wrong i think cuban's making good point here oh yeah i mean i i don't age gating in the history of the internet i don't believe has ever worked so the idea that uh it's going to actually just hey we have new tools we solved mental health.

16:53Let's move on to this, I think is a ridiculous idea anyway. So we just have to, if this is real and there's nothing, we move in this direction in an open way, just assume that this is going to go, forget 14, God help, like the younger this goes. But I also think like that Nate Silver had made a good point around, like he said, you know, OpenAI's recent actions don't seem to be consistent with a company that believes AGI is right around the corner. Do you think, like, is this, and we're going to get into the usage numbers and revenue in just a moment, some new figures we've gotten, but is this an acceptance that kind of that AGI that's going to replace 50 % of white collar work and transform society is actually far away, so we might as well juice some numbers and let people get a little creepy with their chat GPT.

17:50Yeah. So Nate has this great point. He says, if you think the singularity is happening in six to 24 months, you preserve brand prestige to draw a more sympathetic reaction from regulators and attract and retain the best talent rather than getting into erotica for verified adults. They're loosening their guardrails in a way that will probably raise more revenues and might attract more capital or justify current valuations. But this feels more just like as AI as normal technology. I hear everything that Nate Silver is saying there. I just wouldn't be as definitive as him for two reasons. First of all, the same technology that is behind a convincing AI romantic partner is this same technology behind everything else in this LLM world, right?

18:35It's the same foundational technology. Making it better will make it better across the board. But I'm happy to hear the counter argument. I disagree because actually, like, being a good companion or on the erotica side, in a weird way for a large language model is actually like already done. It's easy like to just repeat back, reinforce, come up with some text that's a little bit erotic. Like that stuff is like GPT 3.5, maybe GPT 4. That is not complex, agentic AI across large data sets. And I mean, that's what large language models have been doing for a long time. So I actually think this moves away from the promise of complexity.

19:24This moves more towards the core function that an LLM has been good at for a long time. This is going to get back to our product in the model conversation but i do think as the models get better there'll be better romantic partners so yeah um but but the other side of this is uh is the revenue side first of all i'll just i'll hand it to you that was a good point rajan okay you might you might have me there is that your sycopancy chat gpt impression or do you mean it yeah look it's maybe ai is pushing us in the sycophant direction and we're both going to just like each other a lot more because of chat gpt infecting our brains.

20:00But let's talk about the revenue side of it. The other thing is, oh, they're just usage maxing and revenue maxing. I think the argument opening I would make is the more revenue they have, the more they can invest in data centers, the better models they can make, the closer to AGI they get. That might be the stronger of the two arguments. Yeah, I think that's fair. I mean, the numbers actually, I think we should get into them because 800 million users. This came from the FT. And then 5 % are paying 40 million users, 13 billion in ARR, which implies that$325 annual average revenue per user,$27 per month, which makes it feel that you have some small percentage I'm sure you can model out are paying the 200 bucks.

20:50Most people are paying 20. Were these impressive to you? Or were these, as we get into the usage maxing and what they're actually trying to do? Or were these concerning to you? I would say not surprising to me. It tracks a lot of the numbers that we've seen so far. The fact that they have 800 million users is what we've heard. 13 billion in ARR was predicted. 70 % of revenue is from subscription. So ChatGPT is the lead driver here. Also, I think a lot of people who are just getting into this technology are just not going to pay. but maybe they will in the future like there was this tweet is it just me or is 40 million paying chat gpt users kind of low spotify has 276 million paid subscribers so you know i i don't know i just think give it time and olivia moore from interesting horowitz looked at this uh and compared it to the data of ai subscription products and she said chat gpt's five percent conversion to paid is far above the top quartile for AI products and$27 average revenue per users implies that 4 % of paid users are upgrading to the$200 per month plan, which is also not bad.

22:01So I tend to look at it favorably because it has grown so much so quickly and because there's a lot of room to grow, although you could look at that on the plus or the negative side. How do you read it? I guess, and we haven't even gotten into the losses yet. So we'll do that in a moment. But just I actually agree that 5 % conversion, I mean, like in media, 5 % conversion to paid is good. Substack, do you remember the days they were promising 10 % conversion of all free subscribers to paid newsletters? I do remember those days. Which was ambitious. 5 % is good. And by like major, like working in media and seeing subscription conversions for a long time, that's good.

22:43I think the idea, they almost have, it's been good as in terms of being simple, zero,$20 or 200. There's a lot of room between 20 and 200 to start getting creative. But that's actually where I think the problem on the conversion and revenue side, and they have made clear that ChatGPT Consumer is pretty much the direction of the business, is getting people addicted to usage is definitely going to be part of getting that conversion. And that's why, so to me - How would you do that? No, no, I know, I know. That's why the erotica feels like, how do we get 5 % to 8 % or 10 %? I wonder if there is a slide deck somewhere that has like a projection of like increased conversion rate due to attribute to erotica.

23:41Someone's tracking that. You know there's a deck somewhere. There's a deck. There's a dashboard. A growth manager somewhere has like tagged erotica increasing attribution of conversion. Oh, my God. What a job that would be. It's the whole ballgame. So, yeah, talk about losses. Okay. So$8 billion loss in first half,$20 billion run rate loss right now, spending$3 for each$1 in revenue. I mean, that's kind of like WeWork numbers right there. And it both is terrifying and concerning. And from a pure kind of like SaaS business standpoint, for an early stage growing company, maybe you can argue it's not that bad.

24:25Like, I actually don't think it's horrifying and concerning. It's more, if you just look at it, this is just a traditional software business that's rapidly growing and scaling. Maybe it's okay. I think it's more, we don't have a clear path to, and we've talked about this a lot, generative AI is not traditional software. So growing your revenue at a loss doesn't, it's not like you're just going to scale to, you know, like near 90 % margins. It's going to cost more. The more erotica people are churning out with their companion, that's not high margin business. That can get pretty expensive. So I think the loss, I mean, we all know is concerning.

25:08But to me, getting people more addicted, unless they change the actual pricing model, this is very concerning to me. Definitely. And Noah Smith has a really interesting perspective here, which is, okay, so let's say you assume that a large part of this is training costs. So if you eventually get rid of training costs, then you could be more profitable. Here's his perspective on this. AI model companies assume that model development is a fixed cost that will eventually go away, allowing them to become profitable. But even if that does happen, the lagging model makers might just catch up after a couple of years and compete all the profits away.

25:51Yeah, I think that's actually the most concerning part. I mean, we haven't even talked about the competition side. because like going back to the idea that people will like either you're all in chat gpt erotica or you start to kind of look at it a little uncomfortably and you're like okay maybe i need to go somewhere else and we all know claude is not sexy so maybe that's where you head maybe a co-pilot is the least sexy of the chat bots i'm guessing like uh you should definitely have a Can you imagine talking dirty to something named Copilot? I just can't. And your Microsoft suite just.

26:34It had to go there. It had to go there. You know it was going there. No, no, but for just non-erotic utilization of AI, does this start pushing people into other chatbots? And suddenly, I mean, especially you think parents and high school students, if you're like, if suddenly having ChatGPT open on your screen is concerning to a parent, that starts to change where people spend their time. And remember, like the next year or two, I think is where behavior really starts to form. The switching costs on these chatbots is very low. like what we we've been hopping around from uh the bing back in the day a little gemini on the side so gemini on the side we don't tell the other bots about that of course

27:30oh man um yeah i think competition certainly like it opens up a whole new vector of competition that is not there today. Like people don't look at ChatGPT as a highly problematic thing. And if it's going back to the point around regulators, parents, just overall branding, if it starts to be the kind of skeezy place to hang out, it becomes Facebook blue almost. And that's not good. Live with erotica or live by erotica, die by erotica. It seems like it's just the story for AI. Tale as old as time. All right, let's take a break. I think we need a break there after this. On the other side of this break, we're going to talk about Google's promising new AI well development for treating cancer.

28:17And then we'll also get into Zuck's war with Apple. And then, of course, AI same as problem. Oh, we have a lot to talk about. And we'll do it right after this. Finding the right tech talent isn't just hard, it's mission critical. And yet, many enterprise employers still rely on outdated methods or platforms that don't deliver. In today's market, hiring tech professionals isn't just about filling roles. It's about outpacing competitors. But with niche skills, hybrid preferences, and high salary expectations, it's never been more challenging to cut through the noise and connect with the right people.

28:48That's where Indeed comes in. Indeed consistently posts over 500 ,000 tech roles per month, and employers using its platform benefit from advanced targeting and a 2.1x lift in started applications when using tech network distribution. If I needed to hire top tech talent, I'd go with Indeed. Post your first job and get$75 off at Indeed.com slash tech talent. That's Indeed.com slash tech talent to claim this offer. Indeed, built for what's now and what's next in tech hiring. Did you know your credit card points and miles can lose value to inflation? Credit card companies often reduce the redemption value of your points and miles.

29:25Now, imagine a credit card with rewards that can grow in value. With the Gemini credit card, you can earn Bitcoin or one of over 50 other cryptos instantly with no annual fee. Every swipe at the store or gas pump earns you instant rewards deposited straight to your account. Plus, sign up now for a$200 Bitcoin bonus to kickstart your rewards. Visit Gemini.com slash card today. Check out the link in the description for more information on rates. Again, if you're looking to invest in Bitcoin but don't know where to start, the Gemini credit card makes it easy. The Gemini credit card is issued by WebBank.

30:02In order to qualify for the$200 crypto intro bonus, you must spend$3 ,000 in your first 90 days. Some exclusions apply to instant rewards in which rewards are deposited when the transaction posts. This content is not investment advice and trading crypto involves risk. The Gemini credit card cannot be used to make gambling-related purchases. Shape the future of enterprise AI with Agency. A-G-N-T-C-Y. Now an open source Linux Foundation project, Agency is leading the way in establishing trusted identity and access management for the Internet of Agents, a collaboration layer that ensures AI agents can securely discover, connect, and work across any framework.