"How I Went from Non-Technical to Building an Andrew Tate LLM Chatbot in 3 Months"

0:00Today on the AI Breakdown, the first in what I hope will be a series of interviews on AI learning journeys. Today, I'm joined by Emmett Hom, who taught himself to code over the last three months and has just released not only a couple of celebrity chatbots, but also better tools to help others build something similar. The AI Breakdown is a daily podcast and video show about the most important news and discussions in AI. If you're enjoying, please like, subscribe, and share. And here is my interview with Emmett. A couple weeks ago, I noticed this kid, Emmett, who was posting about a chatbot that he had created for Andrew Tate.

0:35Obviously a very controversial figure, but a kind of cool, interesting use case for training a chatbot on someone's public media and then turning it into something that other people could play with. But the most notable thing about Emmett was that he said that only a few months earlier, he was completely non-technical. He hadn't actually known how to code at all. So he went from not coding, being non-technical, to building an Andrew Tate and later a Chamath Palihapitiya chatbot in just about three or four months. So I thought that journey was really interesting. I wanted to get into how he actually built that chatbot and what other people can learn.

1:09So this interview is about both what he built and how he learned to do it. All right, Emmett, welcome to the AI Breakdown. How are you doing, sir? I'm doing well. Thanks very much. I was super excited to do this. I think it'll be really fun. As we were just talking about, you have been building in public, learning in public, and that's really what this new interview series is all about. So I think before we get into broad strokes, what you've been building, just who are you? What's your story? How did you find yourself coming into this space and where you are now? For sure. So that's the big question.

1:40But a little bit about me. I'm 21 years old. I grew up in the US. I was at Harvard for a year studying history and Russian. And then COVID happened, big pivot, took a year off, really went down the entrepreneurship private hole, started my first company, which was an ed tech company. So we were like a mix between a startup accelerator and a college admissions company. Worked on that for a little more than a year. And we really like our secret sauce was we had a really good TikTok account with a really good sales top of funnel. And so eventually that company was acquired by a larger ed tech company, primarily for that weed acquisition strategy.

2:13And then I was back at school, first semester, jaded, been interested in crypto since about 2017. And this was back in like end of 2021. So really went down the Dow rabbit hole and thought that my time would be best spent building Dow tooling. And so I ended up finding a co-founder, leaving Harvard again, and raising and founding this company called Dow HQ, which is like a Dow marketplace and aggregator for these online companies. We worked on that for a little more than a year and then ended up finding an acqui-hire deal and winding that down. And so for me, it was after that, there was like a point of thinking, okay, like, what do I want to do?

2:46I was previously fully non-technical. So in charge of everything from raising, organizing, just like putting people in the right place and getting us where we need to be. It felt like I was strong at that, but really felt that the one major thing that was holding me back or that would make me an even better founder in the future was just having these base technical skills. And so really just started diving in, chatting with my developer friends, some of our former employees who were really smart and seeing, okay, how can I learn just from the basics, like HTML, CSS, JavaScript, like very much from square one.

3:16And luckily, ChatGPT came out at about the exact time that I started going down this journey. So roughly three, four months ago. And so that's been incredible. And so I feel like I'm able to build 10 times faster than my technical understanding lets me build. And so it's a bit of a cat and mouse of I build way ahead of my technical understanding, and then I have to catch up my technical understanding, and I make another leap. And overall, it's been a lot of fun and a lot of opportunities have come just from really leaning into it, not being afraid to fail, trying to build as much as possible, ship in public, and then just really listen to all the smart people online when they criticize my ideas and give feedback on how I can improve.

3:52Super interesting. First of all, history major into ed tech, into crypto. We could have a very long conversation because it's basically a shared path. But so I guess when it comes to AI, had you made the decision to start dabbling with it when you decided to be non-technical or did it just happen to coincide with you decided to make this decision to learn and then it's the world's greatest learning tool there waiting for you? I think more so the latter. So I started when ChatGPT came out, I was still working on DAO HQ. And so I was just using it for a lot of work tasks. And then saw that our developers were using it religiously, just to debug to build things really quickly.

4:31And so I found that very compelling, because I often have lots of ideas and things I want to build. But the main bottleneck is finding someone to then build the thing, or like using a no code tool, which can often be quite limited. And so I just started playing around with it to build websites and then set up a list of incremental projects that I wanted to build. The first just being like a very bare bones website and just see what I can build with it and my technical understanding and just push myself to build like more and more complex things over time. So I want to talk about some of the things you've built.

5:00The one that I think first captured people's attention was the Andrew Tate project or the Andrew Tate bot, but maybe that's wrong. Was there something before that? Yeah, there were some things before that, but I think that's the first one that really took off. You wrote a thread about sort of the pieces of what it looked like to actually build that. Maybe we can do a video version of that almost, or at least walking through it. Yeah, absolutely. I just remembered the one project that did take off before that was I built like an AI chef. So it was very simple prompt engineering, but you put in your ingredients and then it just output something you could make.

5:31So that was like the first thing. But yeah, I built, it's called Top GPT. And I'll go ahead and share my screen. Let's go to the entire screen. There we go. All right. This is TopGPT, simple chat interface. Obviously, we have Andrew Tate, plan words, TopGPT. This was just a shower idea that came to me. I was just thinking that the name was too funny not to do. Very controversial. So high viral potential. So for me, I was like, okay, good opportunity to improve my chatbot infrastructure on the backend and then also like high viral potential. And so for TopGPT, you ask me questions and it really just mimics the style and voice of Andrew Tate.

6:06And so the process by which you do that is called fine tuning. And so fine tuning involves, and we'll go into my code right here, involves a process of scraping data. So I scraped primarily from YouTube, cleaning that data. And then for the OpenAI fine tuning API endpoint, what you have to do is put it in this format called prompt completion pair. And so I'm going to dive into that in a minute. And that also relates to another project that I launched that hopefully lets people fine tune these chatbots much faster. So this is actually, a few of these are examples from the latest chatbot that I released last night, but the principle is the same.

6:43So the first things first, I made Python scripts to quickly scrape YouTube. And I used ChatGPT to write a lot of this and then just debugged, worked with developer friends to like really work out the kinks. And so after I got these scripts, I had things like this. So this is the Chamath Lex Freeman interview transcript. transcript. And so I have this huge transcript and right now I need to figure out what to do to make it useful. So what I also did is I wrote a series of scripts that will take this transcript. The first thing it will do is it's going to pass this entire thing into essentially chatGPT using the chatGPT 3.5 turbo API or the four API.

7:24It's going to clean it. And one of the really exciting things I think is applicable to a lot of people is now a lot of data cleaning tasks that are slightly too complicated for that code to do, so for Python to do, but are tedious enough that you wouldn't want to do it and you really wouldn't want to give it to an intern are perfect for ChatGPT to do programmatically through the API. So an example is when I'm cleaning this, not only do I want grammar to be fixed, remove words, but I also want it to identify who the speakers are and identify context, which is more of like a general intelligence thing that you can't really code into Python, say, hey, infer who's speaking.

8:00But chatgbt is quite good at that. And so long story short, I get all these transcripts from a bunch of different videos. In this case, it was Chamath. For Andrew Tate, it was just Andrew Tate videos. And then I pass that through. I clean it. So I get these clean transcripts. And then the next script that I pass it through is my prompt completion parascript. So what this does is it chunks the text. It passes it to the, and here's the 3.5 Turbo API. And so the prompt is basically, hey, I have all this data, put it in JSON-L format. Here's an example, write about 10 or 15 questions. And so this is currently, previously was the big bottleneck to fine-tuning a model, is often you have a big piece of text, but there's not a lot of natural questions in that conversation, maybe just a few.

8:48But in order to train it, you need a constant, like a very big document of question response. And the prompt completion script just generates a bunch of synthetic questions. And then eventually you get something like this in Chamath.jsonl, where you have this big document where you can see prompt. So it gives you the question and then the completion. And so the completion here is an actual response from, in this case, a Chamath interview transcript. And then most of the questions here are fully made up by OpenAI. So let me just ask you quickly. So is it working backwards from answers basically to create the questions?

9:27Yeah, that's correct. I posted a version of this on GitHub, which the people have starred. It's called TuneAI or Auto Fine Tune. And so this is that whole process I was walking through. So this is open source. Anyone can use it. You might even tinker with it a little bit, but when it works, you can run it actually in one line, just from your command terminal. and the idea here is like when I was building top gbt I had to manually just copy and paste into chat gbt took quite a long time because the quality of your data directly influences the quality of your model so that's really the big bottleneck and so here if you string all these scripts together then you can essentially just put in as much as many youtube videos as you want as many podcast transcripts as you want clean them and then you know create the synthetic data, so the prompt completion pairs.

10:14And all you have to do is run on OpenAI. And so it's pretty cost efficient as well. So like for top GPT, I think this was 150 prompt completion pairs. Cost about a dollar to fine tune. And then Bestie AI for Chamath, I think it was about 550 prompt completion pairs. That was about three and a half dollars. So overall, works pretty well. another repo that I posted. This is also part of that same repo is infinite GPT. This is essentially just a Python script that chunks. So if you have a really long transcript or any long piece of data, you need to translate or just pass into chat GPT and using the API, this just chunks it.

10:59So it lets you like keep passing it in over and over without passing the token limit. So that's a little bit of a show into what I've been working on and how you can quickly fine tune a chatbot. super interesting one of the things that seems to be really interesting right now is you have on the one hand the sort of arms race among the big players trying to constantly one up each other and create throw more power at this more compute at it like anthropic today just announced their 100k clod model right but at the same time you have just this absolute legion of folks like yourself who are like tinkering building experimenting just like rapidly iterating on what you can do with these things.

11:42It feels, Cambrian Explosion is a super overused term, but it really does feel like this moment where everything that can be tried is being tried almost. Yeah, I couldn't agree more. I'd say it's pretty stark, both in terms of how much people are learning from each other. So I'm not sure if you've, a lot of people, if you go to GitHub, would highly recommend it, just looking at the trending projects. There's a lot of people who are pushing the limits of what you can do. with the limited infrastructure that we have right now. So an example of that is like baby AGI or auto GPT, which like, while some of their use cases are quite buggy, it's still very impressive that people were able to build that just because those are capabilities.

12:20So in this case, those are both two AI agent projects. Those are both capabilities that OpenAI has not let the public have yet, but people just figured out how to string it together in a very clever way. And so I think a lot of innovation is going to come from that. The second, which I find for me has been incredibly be useful and at times like very funny is using these tools to learn more and then building tools to help the tool help me more. So it's a very circular thing. So for instance, earlier this week, I realized I should be doing way more with Langchain. So Langchain is a framework for building with large language models.

12:53It's building a website in HTML versus building a website in React makes it much, much easier and faster. And so I realized, okay, I'm wasting a lot of time. I should really be building a LangChain. The problem is that ChatGPT, just as like a tutor for teaching me how to build, does not have access to LangChain because LangChain is a new development. And so what I had to do is trying to like piece together the tools, like the PDF chatbot tools, download some of those repos, give it the LangChain documentation. So essentially I'll use some of other people's work, a little bit of my own work and build like a personal ChatGPT that knows LangChain.

13:26And then it can teach me LangChain. So I was like, okay, I have this God-like thing. but I need to give it this info. So let me build a tool to give that information, give it back to it. And now it has the information to then again, teach me and help me build forward. So it's like very circular where we're learning from the technology and then we're building tools to help the technology improve so that we can improve. And just like this back and forth, I think is gonna result in a lot of exponential growth and interesting projects in the next few months. Yeah, it feels like the ability to stretch farther.

13:55Like one of the things you said right at the beginning of the conversation was you're effectively building things you have absolutely no business being able to build. And that allows you to fail in more spectacular ways that allow you to learn like way more quickly than you would. Yeah, absolutely. And I think if anyone's thinking about building an AI project or just has an interesting use case maybe for their work or just for fun, there's really almost no barrier to entry. Like the place that I start is just asking Chad GPT, hey, I have this idea. Here's what I'm thinking. How can I build this?

14:28And then ask it some questions. and that's often a very good place for me to start. And it says, okay, for this section, you're going to need a Python script. And for this, you're going to need a, maybe like a Vue app or a React app, and then you're going to need to build a database. And then you can just have it walk you through each of those steps. And while it can't do it for you yet on its own, it's incredibly helpful. And so I think some mixture of using ChatGPT, asking a lot of questions, being willing to just ask stupid questions and really start from square zero is super helpful. And then definitely helps to be on Twitter or have some developer friends because there definitely are limitations.

14:59And so I often will find bugs that ChatGPT has no idea what to do. And so in that case, it's definitely good to go to the adult developers who have actually done proper training and very grateful for their help. I do think that people underestimate how valuable ChatGPT and just these tools in general are from the standpoint of brainstorming for people who are non-technical. What would your three best or biggest tips be if they aspire to call it two to three months from now, be actively building interesting applications in this space? Yeah, so I would say the first is watch a YouTube video or read an article about how to use the command terminal.

15:39That's like the one, and ChaiGPD can help you with that. But if you use the command terminal on your computer, then basically all you have to do is download like a coding environment, so like VS Code, which is free. And then essentially you can just clone other people's repos on GitHub. So like learning how to do that. and you're not really doing any coding. You're just using other people's code. And by playing around with that, you can like tinker it. There's a lot of things you can build. You can use templates. That's essentially all no code. So I think that would be number one. Number two, just like use ChatGPT religiously.

16:08It's incredibly useful. It's like getting better at prompting. And so like the more specific you prompt, the better. So typically I'll have quite a long prompt. I'm like giving it an identity, kind of like priming it a little bit for what I need. And then I would say, let's see, on the third thing is it definitely helps to have some technical understanding and so for me i realized that what chat gpt and github copilot eliminate is the need to know like a very specific syntax like when i was in high school i took one web development class and we had to write on paper how to build like a like the structure for a website and so you're no longer going to have to necessarily know like all of the html tags like all of the python and javascript syntax but what is very helpful is knowing the components in the structure.

16:52And so for that, like YouTube is great. I think there's a channel called Free Coding Camp or something along those lines. I watch some of those videos, having ChatGPT explain things to you and generally just reading articles or documentation because components you need, then you can have much more effective prompts as opposed to saying, hey, ChatGPT, do this magic thing for me. That will get you like a kind of a mediocre result versus, hey, ChatGPT, I need you to build a Python script that takes this much data, chunks it for every X tokens and then outputs this file. So it's like slightly more specific, but it's much more high level.

17:26So you don't really have to get nitty gritty with the specific syntax, but just knowing like the structure of that language, like how it communicates, like generally how computers work. And you can do that in the afternoon, just like watching some YouTube videos or reading. What AI trends are you most excited about or interested in right now? And maybe more locally, what are you excited to build towards in the next few months, the next Yeah, whatever the interval might be. Yeah, I'm very excited about unsurprisingly about chatbots in the process of either building personal assistance, building a second brain.

17:59I think there's just a ton of traditional business opportunities, whether it's just like traditional businesses that have a lot of proprietary information that either their employees need to be more effective. So you can do like simple Q &A chatbots on a database of proprietary info. I've had a lot of people reach out to me who are working in trading or in finance, who are thinking about building proprietary language models to then take the massive amount of data they have and help them with their existing strategies or maybe test their thinking on existing strategies. So really exploring chatbots, the best methods for fine tuning, like fine tuning, vector embedding, prompt engineering.

18:36What are all the different tools that you can use to get different outcomes? And so if you've been watching some of the stuff I've been building, each iteration of these chatbots that I've been launching, I hope they go viral so I like I am choosing people who are fun, who are maybe controversial or get a lot of clicks. But underneath is I'm really trying to improve the infrastructure for how to very quickly and effectively build a chatbot around any use case you want. And so that's a little bit of a hint of what I'm working on. And we'll be eventually building that out as more of a software product So people can go in and won't give too much info away, but very quickly and efficiently build any chatbot you want based on any niche, any person, many different use cases.

19:16So currently playing with that. And then I think another area that I'm very excited just to explore in, but I'm curious if you've seen any good examples of this. But I think monetization of AI applications, with the exception of the base layer, so OpenAI and maybe Pinecone, so just like charging for APIs, is very untested. And so I'm curious to see which monetization models work in this environment, because it's often hard to charge people. If you're paying OpenAI$20 a month for essentially godlike powers, it's not very compelling to pay someone else$20 a month for a very specific use case, unless the pain point is really high.

19:53And so I'm just curious to see which monetization models pop up, whether it's traditional SaaS, whether it's a few messages for free, and then there's a paid tier, whether it's something more creative. I'm sure that people will be testing these out in the next few months. And that's something that I'm going to be sinking a bit of time into is figuring out which monetization strategies can work for some of my projects. Super interesting. Yeah, I have a whole bunch of thoughts on that. But Emmett, really awesome to have you joined and share your kind of learning journey. Excited to see what you build.

20:21Where can people find you if they want to keep track? Yeah. So you can find me at Twitter. So every week I share what I'm building and what I'm learning. And that's just at ehalm. So E-H-A-L-M underscore. Awesome. All right, Emmett. Great to talk to you and look forward to the next time. Awesome. Thanks, Daniel.

20:49Thank you.