Zoom CTO Xuedong "XD" Huang on How AI Revolutionizes Productivity - Ep. 235

0:10Hello, and welcome to the NVIDIA AI Podcast. I'm your host, Noah Kravitz. Zoom became a household name in 2020 as it rose to prominence as the go-to video conference platform during the COVID pandemic. Since then, the company has not only been refining their video technology, but also helping us all rethink the way we approach work in the era of digital communications and AI. At Zoomtopia this past October, Zoom took the wraps off of a number of new AI-first products and initiatives, all in service of the company's mission to deliver an AI-first work platform for human connection. Here to discuss everything from Zoom's approach to federated AI and AI agents to the future of how we all live and work with technology is Dr.

0:54XD Huang. XD is Zoom's Chief Technology Officer and has a prolific background in artificial intelligence, coming to Zoom from Microsoft, where he founded the Speech Technology Group in 1993 and most recently served as Azure AI CTO and Technical Fellow. XD is an IEEE and ACM Fellow and an elected member of the National Academy of Engineering and the American Academy of Arts and Sciences. And most importantly, he's with us right now, so let's get to it. X.D. Wong, welcome and thank you so much for joining the NVIDIA AI podcast. Thank you. I'm glad to be here. So we're recording this on the Friday immediately following Zoomtopia, where Zoom basically announced to the world that you're going all in on AI.

1:37We want to hear all about the new stuff, of course. But first, maybe we can set the scene a little for the audience. Can you tell us kind of broadly about Zoom's approach to AI and AI in the workplace? Yes, I think this is really the most exciting time. I started working on AI since I was a graduate student. This has been over 40 years. Right. Now, GenFD AI really, really transformed how activity is going to be. Yeah. So Zoom is in the forefront. We are providing amazing media conferencing for the whole world. It's a household name that everyone understands what Zoom is about. Right. It's a verb even at this point, right?

2:20Yeah, it's a verb. That's right. So we're now facing even more exciting opportunity in front of us. So meeting is one of the most important business functions, but we want to expand that capability for people to work happily on the zone platform. Right. So zone workplace is going to take advantage of general AI capability. We believe that general AI is going to really provide an exciting new opportunity. I reflected my own journey where I started writing my first master thesis in Beijing's Tsinghua University. The first paper I was using typewriter. I loved expensive liquid paper. In China, in Beijing's Tsinghua University, that was 1983, 82.

3:08I remember liquid paper was expensive. It's a luxury. Right, yeah. I mistyped any letter. I have to really use it for people. Right. I go there and correct that. And when I wrote my book, Spoken Language Processing, it was my colleague in Microsoft. I was fortunate. We had Microsoft Word. But even at that time, Word cannot accommodate a hundred pages document. Too big? Too big. So we have to separate the file for each chapter. But Microsoft Word did a wonderful job. It's hard for me to imagine without Microsoft Word, we have to use a typewriter to find that how hard it's going to be because we have a lot of graphs, math, and references.

3:56Right. Time, task. Quickly, one of my colleagues in Microsoft, he spoke with the systems of GPT-4, Chad 2D. It's just amazing. That productivity really pushed everything to a new level. So you can see that reflection journey. To stop you for a quick second, if you can think back to when you were writing on the typewriter, could you have imagined where we'd be now? You've been in this field for a long time, so perhaps you could. But I'm just curious, you know, if 30 years ago, where we're sitting today, your colleague using GPT-4 to help write a book, is that something you thought about back then?

4:36Yes. Yeah, I selected a speech recognition as my thesis. And the biggest challenge was there at that time. I had only IBM PCX. And I don't know if you know what that means. IBM PCX. Yeah, no, I remember. Plus a few Apple II computers. Yeah, I had a 2E growing up. Yeah, that was all we had. Right. And I actually told myself, if I could let the computer understand spoken language, I could retire. Right. 40 years passed. I'm more excited than ever. So I'm not retiring. The frontier is push now. Yes. It's not for a speech recognition. In Microsoft, we were the first to reach the human parity on the most difficult speech test switchboard in 2016.

5:24Most people didn't believe we could have done that. Yes, we did. Now, chat LGBT really redefined and opened up the imagination for the whole world. Right. I think OpenEd did a great job to really redefine the new frontier. We shared the story. Why Zoom? Yes. going to grab this opportunity to redefine productivity. Yeah. Every era of computing created productivity leader. Microsoft revolutionized desktop computing. Office, unquestionably, is the productivity leader for desktop computing. That's why I shared with you when I wrote the book Spoken Energy Processing with my colleague at Microsoft. We love Microsoft Word.

6:08When the work came together, Google took advantage of it. They enhanced productivity to support multiple people working on the same docking. As we know, Google Docs and the Sheet slides, they all really supported the cross-team collaboration. Right, the collaboration. That took notice from HomeSaveral. Right. One viewer is an incremental info, in my opinion. How so? Now, in general, We are all in the same leveling field, whether it's Microsoft, Google, or Zoom. Of course, Zoom has a unique advantage in the most important business function to connect people on the meeting. We are the leader. But just having that meeting capability would be insufficient.

6:56If you think about the work, we have probably a few key functions, right? One is to consume information. You need to learn. This is intrinsic human value to read, to satisfy our own curiosity. GenoD.I. can help you to read 500 or 800-page books, like what my colleague and I published. 800 pages in one page. So magically, GenoD.I. can redo this and create an amazing amount of learning for almost everyone. We just do not consume information. That's one of the important functions. We also need to communicate, influence, bring people along. So generally, I can help you to compose the draft because we understand your need.

7:44So those two most important human fundamental capabilities to read, to write, to speak, are good to write and to speak on the same. Right. Consume information to communicate. Yeah. They are going to be really, really fundamentally helped. So, magically, we can take that capability, redesign productivity, not just build those capabilities to the existing software. That's an opportunity zone really, really possess. Right. The approach for us to work on productivity suite and the approach we are addressing AI, there are three key ways I want to highlight. I talk about this in Zootopia. Let me just explain this in detail.

8:29The first thing I want to really highlight is our federated AI stack. So we integrate the best form leading AI companies, Amphlopic, OpenAI, Meta, Jet Client, et cetera, plus leading open source opportunities. We augment this with web search leaders, leader of the new complexity. So we federate all of them together. in addition to our own proprietary small language model we're training, we're developing. That is already reaching amazing capability. We appreciate that the small language model, because of the Stirling Law, need to really work together with this amazing cloud-based large language models.

9:17So we have this unique approach to combine them together seamlessly behind to support the productivity of each individual's needs. What does the small language model do in the stack? How does it differ from what you're tasking the large language models with? We are training the small language model like everyone else to train large language model. It's just a harder task. Okay. In addition to that, we're also incorporating each individual's unique context. So we can really make that personalized. In addition to consume this massive amount of tokens. Right. Always somewhere. So if I'm using Zoom's AI features, can I give permission?

10:03Zoom can basically just ingest all of my conversations, all the meetings I have, the voice conversations, the documents, the chats, all of that, and use that all as context for the generative AI going forward. This is the feature that is coming together. through our custom AI company. Oh, got it. AI company 2.0 is horizontal, generic, not personalized. Okay. It's a custom AI company that we will introduce later this next year. Next year, okay. We will actually incorporate the ability for anyone to customize and to personalize. Right. This is actually a very powerful opportunity for the small language model running on the devices to really augment what the blockchain models cannot offer because you don't understand your personal needs, your learning pattern, your writing pattern, etc.

10:59So I just want to really first highlight our federated AI stack is unique, very unique in the industry. Unlike many other productivity companies, they use only one. Right. And so for the audience who might not be familiar, federated AI, a federated stack, does that essentially just mean that the system can choose which LLM to prompt depending on the situation? What does federated mean the way you're using it? There are multiple ways to federate. The way to federate the larger language models and the small language model is a new frontier. The way we are federating this is different from federated learning.

11:38That's trying to really combine multiple models together to form this powerful capability that can preserve the client that we see. What we're doing is we can choose based on different workloads. Because their company took a load is almost like a super agent. That is trying to understand with different modalities, different memory, span, etc. So we can choose what model is the best for different tasks. We can also combine different models together and we can reflect like a chain of thought. We think and we perform the same task based on what we have learned from a small language model, for example.

12:20So if a small language model can perform the task very well, then we stop there. It's sufficient. So it's a very sophisticated system that can actually orchestrate multiple models together. This is being developed and being pushed by Zoom AI talents. So this is a very unique approach that set us apart from almost anyone else. Yeah. You use the word agent And as we're recording this, AI agents are, there's a lot of buzz. I'm hearing a lot of buzz around the word agent and the concept of agentic AI, which isn't new. But as it's come to the fore lately, can you talk a little bit about what that means, what the idea of an AI agent is kind of broadly, but then specifically to how Zoom is using it?

13:04And I want to come to that later. You want to come to that later. Okay. Nick, why we approach AI differently with three key ways. that are different from traditional approach. So the first one, if you think about traditional collectivity suite, most of the companies are using one model, either OpenAI or Gemini, to augment what they do. They build AI capability to the existing software. So on the backend, they're mostly using one very good generation model, either Gemini or OpenAI. So approach is different. we massage OpenAI Anthropic, Gemini, Meta, and our own small-end model together to offer unmatched performance.

13:53So that's the number one I want to really highlight. Of course, we also integrate our partner, Perplexity, for the amazing web search result. For the web search. So whether it's web question or work question, and in the future, personal question, we can massage them together. That's what we are pushing to differentiate Zoom AI through our federated approach. That's number one. Okay. Number two, our user experience is AI first. This is what I call AUI. We're all familiar with screen-optimized graphic use interface as defined by Xerox many, many, many years ago. Yes. You can go back. Right, right.

14:34It's populated by Mac and Windows. So both Office, Google Docs are examples of taking advantage of graphic user interface. So that's well understood. And ChatGPT redefined conversational user interface. They reached 100 million users amazingly fast. Right, faster than anyone. So what Zoom is doing is developing a UI that will seamlessly combine GUI and CUI together. What that means is in Zoom workplace, AI company to follow would be a persistent panel on the right. And the traditional graphic user interface services, whether it's scheduling a meeting or having a meeting with someone. Right. Calendar, your meeting.

15:25Is on the left. Okay. Information flows seamlessly between those two in the AUI. So we are trying to take advantage of both conversational use interface and the screen optimized use interface seamlessly. The visual eye emission is one where the technology intuitively adapts to your own needs. That's getting more personal. That's what we are coming with custom AI companies. Sure. When you say adapts, do you mean that the user interface changes or that it can create a conversational window sort of in context when you mean it? Or could AI potentially just redesign the UI on the fly to match what you're doing?

16:08How do you envision that? AI has the vision. We read it and with that, what information you want to consume, how you want to consume it. Okay. That's what the UI is going to do. It's not just chat interface as chat JPP defined today. It's not just graph user interface. Like Zoom meeting is defined today. Right. Also combined seamlessly in the multimodal environment. Right. We learn that based on your individual needs. Okay. Right now, we're trying to combine those two categories into one. Green, traditional, it's so massive, right? Most of the world's services and applications, they are green, optimized.

16:50Yeah. ChatGVT, conversational user interface, is a new category. And we just cannot have that to be the only one. We're bringing those two together with information flowing across those two categories seamlessly. Right. and trying to understand the user's needs and adapt on the front. Okay. That is what the AUI is going to be. Got it. I want to really call this word AUI. So this is on record. This is the first time I'm telling you in detail what the future of user interface is going to be. World premiere. I love it. Yeah. So that is the principle of the approach Zoom is taking. Right. Embracing AI natively.

17:31That's what we call AI first. You joined Zoom about a year and a half ago, a little less. Yes. When you joined, did you, I know Zoom has had, you know, AI functionality, AI companion version one, and you know, you can use third party apps for transcription and et cetera, et cetera. That's been around for a little while. But when you joined, is this sort of, you came in and thought, okay, let's rebuild this from the ground up, AI centric. Was that sort of already happening when you joined? Just kind of wondering, you know, as you stepped into the role, sort of what was envisioned and how much you've shaped things since then.

18:10So, Eric Yuan, Zoom CEO, got this great vision. So, Zoom has invested in AI before. Yes. Yeah. Since I came, I worked with Eric and the leadership team together. We defined AI first. Okay. Great. Before I came, it was just adding AI, both AI, like almost every other company. Right. Yeah. We have transformed that and got the consensus and pushing AI-first platform. So what does AI-first mean? The three things. The first is the AI technology backend. Both the edge, small language model, and build on the shoulders of great AI companies out there. whether it's OpenAI or Anthropical or Meta or other open source companies like Mishra, et cetera.

19:01Just that there are a lot of them. It would be a mistake not to take advantage of all of them. Of course, right. So it's like we form a committee working to support our workloads. That's always better than just using one single model trying to really perform the same task. Right, two reins are better than one. Yeah. So you see how inclusive we are. On the models, we're trying to combine all of them together. And the user interface, unlike some companies that chat interface is the only way, or graphics user interface is the only way. I'm going to add a button here and there, right? We are combining those two category user interface classes into one that will adapt to your own needs with information flow between those two categories.

19:48That's the second important advance. I want to use AUI as the phrase to summarize this design principle. So the third thing I want to talk about is what is the work productivity suite? That's in the general AI era. I would say it's all about creating a true system of action. We exist, but we have tasks to do. We take action, right? Of course, you can say you want to entertain people, but that's not productivity. There are a lot of entertainment software. AI will do that. So when we say we are AI first work platform, this is about AI companion is designed to understand your workflow. Can learn from your pattern.

20:38Everyone got different workflow. Everyone got a different selection of services. And we use AI to anticipate your personal needs, emphasize that. And they can take action on your behalf with your permission or with your code participating to make a better decision more than what you can just do by yourself. Those are really the soul and the spirit of AI-first productivity. That's very different from just to replace liquid paper with word processing. Right. Or just to support three people co-editing the same document. Right. Or just about formatting this document with nice fonts. It's about those three things.

21:25It's about learning from your own pattern, anticipating your own personal needs, and take action on your behalf. Whether it's tracking tasks or managing action items, it's always one step ahead of you. Ensuring that productivity flows seamlessly, effortlessly throughout Zoom's whole ecosystem in workplace and a certain part of the solution. Right. So if the AI companion, you know, understands my workflow and then can suggest to me actions to take either now or going forward, is it a case of like imagining the AI would say to me, hey, you should do these things in this order? Or will it actually, you know, call up an additional tool to help me help facilitate getting these things done?

22:16Like how, how does that or how do you envision that working? Just innovation, AI company can proactively inform you. You are not answering the question right in the meeting. Only you can see that, right? Right. Just imagine how powerful that is going to be. Right, we're in real time and I start to give the wrong answer on it. Because in real time, AI company is always augmenting your ability to influence others, make others like you better. Right. So this is just what I want to call another phrase. So I'll talk about the Frederick AI stack. unique. Yes. I talk about the user interface that's AUI.

22:54The AUI. This is about action-oriented task flow. Action-oriented task flow. Okay. This will flow to every corner for the whole lifecycle of what you need to do because it's almost like you have a very expensive executive assistant. Focus on the most important task you need to pay attention to for the lifecycle of the whole project until you get that project done beautifully in a time-sensitive manner. And in a way, you delight your co-worker, your family members, for better human connection. This is the goal of Zoom's AI-first work platform. It's an action-oriented information flow. If there's something you don't need to take action, we can still accumulate those tasks to confuse you.

23:42That's okay. And you can decide. And you do not want to actually track those actions. We learn that pattern from you. Yeah. And we improve our ability to track. But if you select the, oh, air companion told me this actually is important. You check that on that. Air companion will work harder. Keep your eyes open. So a week later, if you receive a piece of email that is relevant to the task you're tracking, air companion will work 24 by 7 to update what you need to do and give your tips and the device what you have to do better to accomplish that task. This is what I talk about, action or rendering information flow.

24:23Right. And so is the AI companion an example of an AI agent working on your behalf? Absolutely. AI companion to the know already brought agent-like capabilities like in a meeting. We're not just actually use speech recognition to understand what is being talked about. If you present your slide, and you're complaining to Pono today, understand what is presented in the slide or what you wrote on the paper that you shared your screen. Multimodal. The capability multimodal. Or if you're sharing your points in the side panel with chat, we take that into account as well. That's amazing. It's almost like an agent really participating in the meeting as you do.

25:06Then we present meeting recap. In that meeting recap, the most powerful way for us is identify next steps you need to pay attention to or your colleague need to pay attention to. Then next step is unique. We offer a major quality. We worked on this so far in the past year and a half to improve next steps, to reduce isolation, to assign the right task to the right person. We are roughly right now probably 80 % accurate. Okay. So we're not done. It's not perfect. But 80 % is really impressive. I was going to say, In my experiences with LLMs and hallucinations and accuracy, 80 % sounds pretty good.

25:47Yeah. So we do not stop here, right? So let's say you have a meeting you discuss. You are complaining to identify the task. And that task will show up in the upcoming task panel. And a week later, through the whole lifecycle, that's something you want to track. You want to track. You'll receive a piece of email from Zoom. And create an update on your behalf. If you want to read a report about an actual item or status report for your colleague, that information flow into Zoom docs. We'll draft the status report on your behalf. If you're pretty happy with a few changes without doing anything, use an equal paper or Microsoft Word to format everything.

26:34And you want to say, hey, change this into the form that I can present in that next status update meeting. Done. one command. Yes. Right. And it's not as beautiful as, you know, what a PowerPoint can do with beautiful picture. But the key points is very much like when I was in the College of Maryland, when we presented information with black and white, the T-Roth, right? And printed on the really transparency paper and just used really projector to talk about the key points. Was them actually effective? It still works, yeah. Information is slow. with that beautiful color background animation. So that's the point I want to take.

27:18Zoom docs alone is actually performing most of the function because of the general AI. With AI companion, you can instruct, you can summarize in the form of status report, or you can publish this as a blog that's more ready to be consumed for the public, or in the simple form of slides that you can communicate your points in your next meeting. You do not need the last generation productivity suite, as we know. So that is actually high level three key points. Frederick AI-Style, AI-First Use Interface, AUI, and Action-Oriented Information Flow for productivity. That is really the landmark. How AI-First Productivity Suite will differentiate from web-centric productivity suite or desktop-centric.

28:05Of course, both web-centric and desktop-centric can add a capability. That's bolded. That's not... Bolded, sure. Yeah. What Zoom is defining is native. Right. Back-action-item, information flow into every corner of the productivity stream. I made it myself clear. Only three things. Our guest today is XD Wong. XD is the CTO of Zoom, a position he's held for going on a year and a half now. Before that, he was with Microsoft for quite some time. And really, it's just a continuation of an illustrious career that started way back, as XD was talking about, in the days of typewriters and liquid paper, as we both know.

28:48XD, I want to switch gears here. We have a few minutes left to talk. I want to kind of look at things from the public's standpoint, and specifically from business users and the types of customers who Zoom has been working with for a while now. Now, when Zoom is talking to business customers about AI, about adopting Zoom's products and all these wonderful things you're building, but broadly just about, you know, using generative AI and making the investment and spending the time to upskill workers and sort of figure out how are we using these things. How do you help your customers think about both adopting AI and also how to measure return on investment?

29:27There's a lot of conversation we've had on the podcast and just generally in the world about being kind of, for exciting as the past couple of years have been, still being in the early days of figuring out what can Gen AI do? How do we use it? How do we rethink things like productivity suites from the ground up with AI? So when you're talking to customer companies, how do you educate them about getting started and measuring performance? There are a few things absolutely our customers love Zoom. First, Zoom workplace as a whole offers ease of use. That's just, you know, unmatched. Whether it's eating or dogs or chat.