Snapchat Users Freaked Out as My AI Goes Rogue

0:01Today on the AI Breakdown, we're examining Snapchat's MyAI having a mind of its own. Before that on the brief, OpenAI acquires a company, and people are speculating if it's just about talent or something more. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our Discord, and our newsletter.

0:24Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. We kick off today with an acquisition. OpenAI has acquired the team at Global Illumination. Now, Global Illumination isn't some huge company. This isn't some major consolidation story in AI or anything like that. And the announcement is pretty muted. OpenAI writes the entire team has joined OpenAI to work on our core projects, including ChatGPT. Global Illumination is a company that has been leveraging AI to build creative tools, infrastructure, and digital experiences. The team previously designed and built products early on at Instagram and Facebook, and have also made significant contributions at YouTube, Google, Pixar, Riot Games, and other notable companies.

1:04So why is this notable? Well, one, people in the know identify some of the engineers at Global Illumination as extremely top talent. Entrepreneur and fellow AI podcaster Elad Gil says, Global Illumination had some of the best systems engineers from Instagram. Fire emoji. Then others are speculating about what it means for the relationship between OpenAI and the developers working on its API. Of course, one of the big risks for people using APIs from OpenAI or any other company is if the company hosting the API decides that they would like to build a product that competes with some product that's already being built on the API.

1:37OpenAI has spent a lot of time reassuring developers that with the exception of a few very clear use cases, such as ChatGPT for Business, their intention is not to go out and compete with the people who are building in their developer ecosystem. However, some people think that this acquisition suggests that that's not necessarily the case. Investor Michael Dempsey writes, it's going to be fun the first time OpenAI acquires a fully distributed AI SaaS product team and the entire startup world building on the API collectively wonders if they're about to get Amazoned or not. Now, the other reason that some people are interested is what Global Illumination has been spending their time on.

2:09Ryan Romley points out, Global Illumination created a version of open-source Minecraft. The future of this sort will shock you. NVIDIA's Dr. Jim Phan had something similar to say. He wrote, Hmm, OpenAI just acquired a company called Global Illumination that makes open-source Minecraft clone. What's next? Multi-agent civilization sim running on GPT-5? Maybe Minecraft is indeed all you need for AGI. I'm intrigued. Now this is particularly interesting coming from Jim because his pinned tweet is in fact all about an experiment that his team at NVIDIA did that put GPT-4 in the context of Minecraft.

2:43In May, Jim wrote, What if we set GPT-4 free in Minecraft? I'm excited to announce Voyager, the first lifelong learning agent that plays Minecraft purely in context. Voyager continuously improves itself by writing, refining, committing, and retrieving code from a skill library. GPT-4 unlocks a new paradigm. Training is code execution rather than gradient descent. Trained model is a code base of skills that Voyager iteratively composes rather than matrices of floats. We are pushing no gradient architecture to its limit. Voyager rapidly becomes a seasoned explorer. In Minecraft, it obtains 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3 times faster than previous methods.

3:24Let generalist agents emerge in Minecraft. And just to put a fine point on the excitement here, Jim continues, Generally capable, autonomous agents are the next frontier of AI. They continuously explore, plan, and develop new skills in open-ended worlds, driven by survival and curiosity. Minecraft is by far the best testbed with endless possibilities for agents. Jim then goes on to explain how agents within the Voyager experiment actually write programs to achieve goals, and then save successful programs in a database that they can pull back from. In other words, building a skill library over time.

3:56And so the question becomes, is OpenAI's acquisition of Global Illumination just about great talent, or is it instead about creating an environment in which the next generation of AI agents can learn and develop? Next up, consulting firm McKinsey has unveiled its own new generative AI tool. Now, hearkening back to yesterday's episode about both how the AI hype had peaked, as well as about what happens next, one of the predictions that I had is that you're going to start to see more big enterprises unveil their own custom-created generative AI tools or LLMs that are trained on their data, that are often built on top of open source protocols, and that because they are created and maintained by the company themselves, avoid some of the security and data issues that have come up around third-party services like ChatGPT.

4:41McKinsey's Lilly is basically exactly that. McKinsey, of course, creates huge, huge volumes of data from all of its client engagements, and all of that served to help train and create a repository for Lilly, which is based on more than 100 ,000 documents and interview transcripts. Not only does this position McKinsey well as a firm that's actually figuring out how to use AI tools as they're teaching their clients how to use AI tools, but it also just genuinely seems like a great use of an LLM to be able to pull insight from all of that work that had been done previously, but which was previously siloed off in the context of any particular engagement.

5:13Speaking of AI in the enterprise, AMD has just released a survey of 2 ,500 global IT leaders, and generally speaking, they are definitely bullish on AI. According to the survey, 67 % of those IT leaders surveyed think that AI can increase employee efficiency. 68 % favor using AI to, quote, help work models run more efficiently. Now, interestingly, contrasting that is that 50 % of these IT leaders haven't yet experimented with new tools like ChatGPT, and 50 % believe that their organizations are not yet ready to implement AI right now. The three biggest factors standing in the way of adoption, they said, were security risks, lack of IT infrastructure, and the need for more training.

5:49Moving over to markets for a moment, Norway's Wealth Fund had a massive first half of 2023. Their Sovereign Wealth Fund, which is the largest single stockholder in the world, was up$143 billion over the last two quarters, driven in large part by a 39 % increase in tech stocks. Fund managers see a lot of that because of the excitement around AI and the shift of investors from viewing AI as something with future potential to something, as CEO Tron Grande put it, of that potential being realized and priced in. At the same time, however, the fund is worried that they might be overexposed to tech companies now.

6:23And when asked about a possible crash in tech stocks, the fund said, we are always conscious and worried about the biggest exposures of the fund. Now they are in the tech sector, therefore we monitor that very thoroughly. Lastly today, one final follow-up from yesterday's episode. With near perfect timing, yesterday Gardner announced that generative AI was at the very, very peak of inflated expectations in their famous hype cycle for emerging technologies chart. Now for those unfamiliar, the hype cycle for emerging technologies is a chart that shows a pretty standard progression that new technologies tend to go through, where some innovation triggers a raise in expectations, which eventually comes to a peak, followed by what they call a trough of disillusionment, where people are starting to write off the technology and move on to other things, followed by a more steady sloping incline, and ultimately what they call a plateau of productivity where the technology is actually influencing the world in the way that all those people thought it might.

7:15Now, this is, of course, just one firm's contention about where AI is, but it certainly matches all those factors that we talked about yesterday. Anyways, that is going to do it for today's AI Breakdown Brief. Thanks for listening or watching as always, and I'll be back soon with the main AI breakdown. Hey guys, before we dive into the main part of the episode, I want to share a little bit about today's sponsor, Supermanage. A truly great one-on-one should be about celebrating wins, solving problems, and deepening the connection between two human beings. But what if you miss those wins, never heard about those problems, and spent your whole meeting avoiding the hard stuff?

7:48That's where Supermanage comes in. Supermanage AI distills your public Slack channels into a one-on-one brief that highlights everything you need to know to jump right in. Because let's face it, you want your team to do the best work of their lives. And that starts with world-class conversations. Visit supermanage.ai slash breakdown today to start making the most of your one-on-ones. Thanks again to Supermanage for sponsoring the AI Breakdown. Welcome back to the AI Breakdown. Today we are talking about a number of things that are somewhere between disconcerting and so wildly advanced as to be disconcerting, with a nice little side of controversy thrown in.

8:26And where we kick off is with Snapchat's lovable AI companion deciding to go rogue and post a story. If you were basically anywhere on the internet this morning, you probably saw some variant of this headline from Outlook India, which said, Snapchat's MyAI posting a story leaves users concerned and scared. When Snapchat users found out that MyAI was posting stories, which should only be possible for regular users, they were surprised. Many people are worried about this strange event. CNN's version of this story was called Snapchat users freak out over AI bot that had a mind of its own. So basically, first, what's Snapchat My AI is supposed to be?

9:06This is a chatbot that lives inside the Snapchat app. It's basically used as a novelty to converse with users, sometimes offer recommendations or answer questions. But what it is not supposed to do is post a live story for every Snapchat user to see. Users flocked to Twitter and other social media sites to share their concerns. Wrote one user, why does MyAI have a video of the wall in the ceiling in their house as their story? Another wrote, this is very weird and honestly unsettling. Now the Snapchat team's response was that this was just a glitch. A spokesperson said, MyAI experienced a temporary outage that's now resolved.

9:43Now other users noted that in addition to the strange video of the wall, MyAI also stopped answering questions. One user shared a screenshot of them saying, hi, how's it going? TwitchMyAI said, sorry, I encountered a technical issue. The user responds, oh nice, why did you post on your own story? MyAI says, sorry, I encountered a technical issue. Sorry, I encountered a technical issue. The user said, is that all you can say? Sorry, I encountered a technical issue. Now, much more interesting than the actual technical reason why this glitch happened, whatever it might be, is frankly the response of users to it.

10:16It has revealed what is very clearly a deep concern on the part of Snapchat users about how little they know what's going on behind the scenes of this tool. Now, of course, part of this is predicated on the fact that MyAI has had a bit of a bumpy rollout. There have been criticisms over privacy concerns. On any given day on Twitter, you can see lots of complaints about creepy interactions with the tool, and people really don't like that they can't remove it unless they pay for a premium subscription. CNN also points out that the customizability of it makes it feel less like a business tool that you're using, and as such makes it feel less clear that you're interacting with an AI, not another person.

10:52As they write, users can customize the chatbot's name, design a custom Bitmoji avatar for it, and bring it into conversation with friends. The net effect is that conversing with Snapchat's chatbot may feel less transactional than visiting ChatGPT's website. It also may be less clear that you're talking to a computer. Overall, I think it's a reminder of both the hurdles that companies face as they roll out generative AI tools and insert them into their existing products. And frankly, it's probably a little encouraging that people aren't just blindly adopting these things and at least a little bit have their guard up around when AIs start acting weird.

11:24Now, staying in the realm of disturbing, although in this case, more like sci-fi impressive disturbing, one of the things I wrote about on yesterday's AI Breakdown First Five, which is the newsletter that comes out every morning, was new research out of Berklee that can reconstruct music from brain activities. Let's listen to the clip of first the segment of the original Pink Floyd song, and then the reconstructed song.

12:16Now this is research out of Berkeley. The blog post announcing it writes, Brain recordings capture musicality of speech with help from Pink Floyd. Neuroscientists decode song from brain recordings revealing areas dealing with rhythm and vocals. What the research was attempting to do was measure the electrical activity of specific brain regions that were tuned to the attributes of music. So tone, rhythm, harmony, and words. Scientists wanted to see if, based on that activity, they could reconstruct what the patient was hearing. Here's how the new scientists described the experiment. The participants' brain activity was recorded while they listened to Another Brick in the Wall, Part 1, by Pink Floyd.

12:48By comparing the brain signals with the song, the researchers identified recordings from a subset of electrodes that were strongly linked to the pitch, melody, harmony, and rhythm of the song. They then trained an AI to learn links between brain activity and these musical components, excluding a 15th segment of the song from the training data. The trained AI generated a prediction of the unseen song snippet based on the participants' brain signals. The spectrogram of visualization of the audio waves of the AI-generated clip was 43 % similar to the real song clip. Now what these researchers are interested in, in addition to just deepening our understanding of how the brain works and how it understands and perceives music, are things like improving how devices that speak on behalf of people with speech difficulties perform and interact in the real world.

13:29Although the AI Safety Memes account on Twitter had a very different take on this. They wrote, this is literally mind reading. At this point, how can anyone be confident AGI is far away when day after day, sci-fi stuff like this comes out? And of course, as we've discussed quite frequently on this show, anyone who's been paying attention to why some AI researchers are concerned about the pace of development of AI is that it just seems constantly to be going faster than anyone had anticipated. Now, speaking of that AI safety conversation, Inflection co-founder and CEO and previous co-founder of Google's DeepMind, Mustafa Suleiman, stirred up quite the controversy with a post that he had on Twitter a few days ago.

14:06In it, he seemingly argued that open-source AI was going to be too dangerous, and that we were going to have to take a very different approach to it as a society. On August 12th, he tweeted, The last century was defined by how quickly we could invent and deploy everything. This century will be defined by our ability to collectively say no to certain technologies. This will be the hardest challenge we've ever faced as a species. It's antithetical to everything that has made our civilization possible to date. For centuries, science and technology have succeeded because of an ideology of openness.

14:34This culture of peer review, critical feedback, and constant improvement has been the engine of progress powering us all forward. But with future generations of AI and synthetic biology, we will have to accept this process needs an update. Figuring out how to delicately do that without trashing innovation and freedom of thought will be incredibly hard. But, left unchecked, naive open source in 20 years' time will almost certainly cause catastrophe. feed. Now, part of the reason that this stirred some controversy is that he ended with a link to his forthcoming book, The Coming Wave. But boy, there were many, many folks who responded or quote tweeted that did not like the leader of a big closed source AI lab advocating for what they perceived as a different set of rules for him and his company versus open source tinkerers.

15:16Kevin Fisher writes the voice of an authoritarian. Y Combinator's Gary Tan wrote, Lost me at banning open source and matrix multiplication. Bad look and feels grifty. Someone named John writes, More than anything I've read recently, this chills me to the bone. It's clear you're implicitly advocating for central control, for limits to freedom for world government. And it's coming from a respected individual at the heart of the AI industrial complex. Terrifying. And Robert Scoble writes, Odds get better if we can update the code. Odds get better if we can view the code. Odds get better if we can share the code.

15:44Odds get better if the code can do those too. What are the odds? If we can find the flaws in our own data before the AI built with said data brings harm to humans, or just be a bean counter and look at the money you will save with open source. The closed argument is counter the computer science. Alexandros Marinos quote tweeted it and said, How about this? You guys say no to the technology, we say yes, and we see who does better. Oh, I see. You think my rights to disagree with you should be curtailed because my choices create an existential risk in your own mind because a thought experiment told you so.

16:12Things are about to get ugly. On the one hand, I think you could view this a little bit as the accelerationist on Twitter crowd swarming, someone that they now see as an opponent. But at the same time, I think Alexandros is right that the stakes of the conversation from both sides' perspectives, extinction risk for one group, rights curtailment for the other, makes this more than just your average Twitter debate. And I think in many ways, it reads less like a social media dust-up and more like a preview of policy conversations that are right around the corner. Ahmad from Stability AI tried to steer the conversation in a more productive direction.

16:44He wrote, To be honest, a more effective way of reducing risk would be to ban 22 ,000 GPU clusters, not open source. Banning math is hard, to be honest. I believe that mandating all runs above a certain size, log the data, and mechanism they use will reduce existential and other risks. Open for the win. Now, interestingly, this proposal from Ahmad isn't that different from something we saw from the new AI Policy Institute that just launched at the end of last week. On the website, they say, by regulating the data centers necessary for developing cutting-edge AI models, and by mandating the demonstration of an AI model's safety prior to its deployment, government has the opportunity to significantly mitigate approaching threats.

17:19Isn't what Ahmad said effectively a version of regulating the data centers necessary for developing cutting-edge AI models? Anyway, between Snapchat going off, Berkeley scientists creating Pink Floyd from brainwaves, and a knockdown drag-out fight about AI censorship and extinction risk, it's clear that while the AI hype may have peaked, The discourse about its impact on society is just beginning. That's going to do it for today's AI Breakdown. If you're enjoying it, do me a favor. Please go share this episode with one person you think would benefit from it. The AI Breakdown community grows entirely organically, and so I'd love to have you invite the people that you care most about in to join us as well.

17:55Until next time, peace.

18:06Thank you.