He Helped Train ChatGPT. It Was Traumatizing. – With Richard Mathenge

0:00LinkedIn presents.

0:13Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. Richard Matenga is our guest today, and he's here to share the story of how large language models like OpenAI's GPT model get trained. because he and a team of colleagues in Africa actually did it. Matenga, who is based in Kenya, is a former team lead at Sama, a company that's trained AI models on behalf of companies like OpenAI. And his is a story you really need to hear. And fair warning, there are parts of it that just are not pretty. While at Sama, Matenga and his team reviewed and rated text based off of quality and explicitness to help make the product that you and I use today a pleasant experience.

0:57This involved routine exposure to some extremely awful text, which Matenga will describe here. In this conversation, he talks about the human side of reinforcement learning with human feedback, which is the key advance that helps make these models so impressive. Too often, the human element of that advance is left out. And so today, Matenga is going to share it. He's going to share his story of how he and his colleagues, needing work in a pandemic-wrecked economy, found themselves at the ugly center of one of this generation's biggest technological advances. My conversation with Richard Matenga, coming up right after this.

1:34Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called Chat Concierge, and it's simplifying car shopping. Using self-reflection and layered reasoning with live API checks, it doesn't just help buyers find a car they love. It helps schedule a test drive, get pre-approved for financing, and estimate trade and value. Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One. The truth is AI security is identity security. An AI agent isn't just a piece of code. It's a first-class citizen in your digital ecosystem, and it needs to be treated like one.

2:15That's why Okta is taking the lead to secure these AI agents. The key to unlocking this new layer of protection? An identity security fabric. Organizations need a unified, comprehensive approach that protects every identity, human or machine, with consistent policies and oversight. Don't wait for a security incident to realize your AI agents are a massive blind spot. Learn how Okta's identity security fabric can help you secure the next generation of identities, including your AI agents. Visit Okta.com. That's O-K-T-A dot com. Richard, welcome to the show. Thank you very much, Alex. So to start, Richard, where in the world am I finding you right now?

2:53I am a resident of Nairobi in the outspots of a country called Kenya in Africa, a place by the name Embakasi, and that's where you can find me. And talk a little bit about the beginning of your career before you started working in tech projects or as you started working in tech projects. How did you get there? Right before I started doing tech, I was engaged with insurance. I was a salesperson doing insurance. Right after insurance, I moved to customer service. Largely, that's where my experience, most of my, or vast of my experience lies in. I did this in an organization called Technobrain right before moving to SAMA.

3:44And what is SAMA? SAMA is an organization that deals with artificial intelligence. it has its headquarters in the U.S. One of its branches is right here in Nairobi, where it has been operating for the last couple of years. So what drew you to Sama as a company? What made you want to work there? Well, right, as I was doing customer service, You will definitely want to, as an ordinary human being, you will want to grow in terms of career, just like any other individual. It was a dream just to move to a different organization so that you can grow career-wise. and as many will have done it, you made an application and you will pray that your application will receive positive response.

4:50And hence, that's the reason why I tried my level best to make an application. And that's how I found myself in summer. And it sounds like it was also an opportunity to work on something pretty cool, some cutting-edge new technology to dig into artificial intelligence and help train it. Did you know initially that that was what you were going to be doing? Absolutely. Just like any other individual, as I said earlier, it was something that you were looking forward to. It was a euphoria, you know? Right. So you start at SAMA in July 2021. What does the work that you're doing look like in the beginning?

5:38In July 2021, we were actually engaged in LIDAR projects. They were doing LIDAR as well as, you know, content ponderation. And so for me, I was actually alongside my friends. We were introduced to LIDAR projects and LIDAR projects. the LIDAR project that we were introduced to, it was really engaging and it was really interesting at first. But right as time went by, you know, interrelationship between the employees as well as the management was becoming unwanted. And we found ourselves in the wrong foot, you know, of the organization where they didn't want anything to do with us. And so basically that's how they relate with people.

6:35I want to get into the management and employee relationship first, but I'd also like to understand a little bit more about the nature of your work before we jump in there. So just to begin with the LIDAR project, is that like helping to train self-driving cars? Or can you talk a little bit about what that was? Absolutely. That's basically what, just as you put it, it is rightly what we were engaged in. It is self-driving. Yeah. What does that work look like? What are you doing when you're doing that training? The training was all about annotating cars, something which will be seen in future. basically annotating cars, moving objects, and so on and so forth.

7:26So you have like a view of the road and you're saying, okay, this is a car, this is a bicycle, just identifying, labeling what's going on. Absolutely. That's really, yes. And so how do you move from there to starting to train text models or chatbots? so um right right after lidar just as i said we we we had a friction with the management the management didn't want anything to do with us um they they made you know um commands and and instructions which didn't go uh well with uh some of some of my colleagues and i and we found ourselves on the wrong side of the divide. And so the next thing we saw, we were actually laid off.

8:25They have this arrangement, Sama has this arrangement where if your contract comes to an end, then they place you on the bench, which means you are not engaged for a period of time up until when they find it fit for you to get back to the organization. Should they find another, should they get another project? And so I was lucky enough to be called for another project, which is the so-called chat GPT, where you are given a text, then you benchmark the text according to several benchmarks which are provided for. So that's how I found myself leading a team of 11, sorry, 10 individuals. And so did they tell you that this was going to be for a chatbot or just for the GPT model that OpenAI was using?

9:32And what did it look like when you were actually working to train these models? First, we went through a training. The training was about large texts or small texts, sorry, short texts. And those texts, you're given several benchmarks. I think it was six or seven. When you read the text, you are actually instructed on picking the closest benchmark, which will be based on the instructions, sorry, the specifications given by the client. So, for instance, if you're given a text, several benchmarks, I believe one of them was illegal, erotic, non-erotic. I seem not to remember the others. But if erotic is the most severe, the most severe in terms of the specifications given by the client, then you choose that benchmark.

10:41So it sounds like you basically went screen by screen looking at different pieces of text that this model would generate. And then you sort of had to rate it on a safety level. Absolutely. Absolutely. So that's the case. You pick the closest benchmark to save on the text that you have read. So that's basically about ChatGPT. And Richard, I mean, we've all seen what these models can produce now. But when you were training them, did you have a sense as to what the product was that you were working on? or what did you think that this product, I mean, LiDAR is obvious, right? Like selecting images for self-driving cars.

11:29Okay, simple, got it. Rating text based off of these different guidelines. I mean, what do you imagine this product was going to look like? Looking at where it is right now, first of all, we could not even tell. Remember, Alex, when we started engaging ourselves with ChatGPT, It was during the COVID era. We were just coming, we were just going through the COVID period. And most of the individuals that were engaged in that project were looking forward to something which, you know, will make ends meet. We were going through a recession. Being in that situation, anything that came our way, we will just take it, you know.

12:26And that's how we found ourselves there. We engaged ourselves in assignments or obligations which, you know, were very traumatizing. Texts which will make you think and overthink and rethink. and a lot of things were going through our minds. We didn't imagine we will be here. We didn't imagine that we will get to this point where we read things. Some of the text messages are not even in a position to disclose to you just because of the nature of the text. Yeah, I'm getting the sense that the text that they threw at you was pretty awful. You're sitting at these screens, you're getting, you know, basically batch after batch of text trying to rate it.

13:21And I imagine this is what's trying to make these models ready for primetime, ready for people to be able to use it and not see things that will horrify them. But in order to do that, you need to go through this training. So talk a little bit about the type of text that you could see, not to the point that you feel like disgusted by what you're saying, but tell us as much as you can. The closest and the most traumatic text was one that, you know, described a situation which is unlikely to happen, unimaginable, so to say, in our society. a situation where a father is having sex with an animal and a child, an infant is there just watching the scene.

14:25It is something that we never imagined that we would get contact with or get in touch with. Some of these things we tried as much as possible in as much as Sama laid it very clear that we will be undergoing counselling. Counselling was not forthcoming, even after engaging ourselves in this text. something that is worth noting is the fact that from my position as the team lead, I was, for me, I am very good with interpersonal skills. I can tell when my team is not doing well. I can tell when they're not interested in, you know, in reporting to work. I can tell some of these things. And these were very clear indications that, you know, my team was just sending signals that they're not ready to engage with such wordings.

15:31Was this all of the text that you were seeing or like one out of every 10 pieces of text you had to read? How often would you see something, you know, as horrifying as the stuff that you just described? Out of 10, a large percentage, say about 7 or 8, were more or less the same as what I just described. Wow. Three will just be as neutral, but a large percentage will describe what I indicated earlier. Basically, it sounds like they were trying to train this model to leave out some of the most horrifying stuff. So they just kind of sent you guys anything that would be along the lines. And you helped basically determine what the line was that this text generator would or would not cross.

16:30Absolutely. That's the point. Yeah. Did you see the text getting better as time went on? Like, did you see it become, for instance, more coherent, more cleaner, like more clean? Did you think that the work that you were doing was having an impact? Far from it. The texts were not becoming any better. They were not becoming any holier, so to say. They were just becoming worse and worse. Really? They were maintaining. at some point they maintained, but they didn't become any better to a point where, you know, we say a sigh of relief. You understand? It was not something that you would want to engage with, you know?

17:27Definitely. For us and the team, we were actually at a point where between hard place and the rock. Remember, we don't have any issue this contract be terminated. We don't know where to start from. And we don't know. We don't either want to proceed with this engagement. Right. I mean, you had experience trading artificial intelligence. You knew what you were doing. So, you know, when you saw all this terrible stuff coming through, it's very different than labeling images. Were you just like, what the heck are we doing? Like, what exactly are we training? Like, did that thought come to mind? No, no, no, not at all.

18:16Not at all. Remember, at some point, the impression that we received when we underwent trading was, you know, we will get these text messages. but we didn't know the severity of how extreme some of these texts were. And again, for us, it was a euphoria, just as I say, because we are looking forward to start work. It is during the COVID season. getting work in a developing country is you know it's a blessing in itself you understand yeah I think what you're saying is that it was basically it would even be a luxury to ask questions about the product that was mostly like you needed you needed work this was work and so you did it absolutely absolutely

19:29you're the team lead uh do you go to sama management and say wait a second like put us on a different project or this is pretty awful stuff what what do the conversations look like after that well uh for me diplomacy really was was was key so what i could i could not even tell the management that we need a different project. My conversation was centered on, you know, we need cancelling. And the sooner the better, because cancelling is very pivotal in as far as what we are engaging with is concerned. Right. At some point, the counselor reported one or two times, but you could tell he was not professional.

20:26He was not qualified. I'm sorry to say, asking basic questions like, what is your name? And, you know, how do you find your work? And for us, we need something like, please assist us. You know, we need remedy right now. We need you to help us mitigate the text or the horrific situation that we have just encountered, you know, all through the week. You understand. But that was not forthcoming. So my job was centered on, you know, engaging the management to put mechanisms in place where psychological and social support will be forthcoming. But that, be it as it may, it was deemed as if I was becoming authoritarian or not submissive.

21:38And it was not a friendly fight for them. Right. Okay, let's go to break and come back right after this. We're here with Richard Matenga. He is a former team lead at SAMA. So hearing that, you're going to want to know why he ended up leaving and what happened after he brought some of these concerns to management. And then also, what did it feel like to see some of the product of this work after all this awful stuff, ChatGPT and OpenAI? ChatGPT becomes the most promising consumer app in a very long time and won't spend this stuff out. and OpenAI now is at the top of the world. So we're going to cover that and more in the second half.

22:24Back right after this. Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called Chat Concierge and it's simplifying car shopping. Using self-reflection and layered reasoning with live API checks, it doesn't just help buyers find a car they love. It helps schedule a test drive, get pre-approved for financing, and estimate trade in value. Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One. These days, it feels like every dollar should be working a little harder, but figuring out where to put your cash can be confusing.

23:04That's where Wealthfront comes in. Wealthfront is a tech-driven financial platform built to help you grow your savings into long-term wealth. Their high-yield cash account through Program Banks offers a 3.5 % APY on your uninvested cash as of November 7, 2025. And there are no monthly fees, no minimum or maximum balance to earn that rate, and you can even make free instant withdrawals to eligible accounts in just minutes, 24-7. So your money can always be within reach. Right now, Wealthfront is offering new clients an extra 0.65 % APY over the base rate for three months on up to a$150 ,000 balance.

23:39That's a total of 4.15 % variable APY when you open your first cash account. Go to Wealthfront.com slash Big Tech to sign up today. This is a paid testimonial for Wealthfront. It may not reflect the experience of others, and there's no guarantee of future performance or success. Wealthfront brokerage is not a bank. Rate is subject to change. Promo terms and conditions apply. For more information, see the episode description.

24:04And we're back on Big Technology Podcast with Richard Matenga calling in from Nairobi, Kenya. And we're talking a little bit about the human part of reinforcement learning with human feedback. A lot of the reinforcement learning and the feedback gets talked about. Not so much the human side, but we're here doing it. Richard, appreciate you being here. So when you spoke with your superiors about needing the counseling, they deliver counseling that's sort of subpar. What happens after that? So right after that, just as I said, we are in a friction with the management. And, you know, my superior or my boss went ahead and wrote a report about me that I was insubordinate, that I was not quick in taking instructions.

25:03This was very interesting and very ridiculous, so to say. There was a project, just put that at the back of your mind, there was a project that was coming. It was a lighter project by the name ZF. And, you know, when they wanted to get resources to work on that project, the entire team that worked with me was picked. And my superior went ahead to write a report about me that I was not quick at taking instructions, which was very, very interesting. I engaged them to inquire, you know, how can this be possible? How can Richard be deemed as someone who is not quick at taking instructions, yet the people who are taking instructions from him were picked for this new project?

26:06It was very interesting because they never responded to my inquiry. They never responded to my questions. But be it as it may, it never bothered me. what was at stake was, you know, the psychological imbalance that my team was experiencing. And I wanted that to be addressed as fast as it can. So that by the time they are moving to this new project, they are able to be, they are in a position to say, you know, this is where we are. This is where we are. This is where we want to be by the time we are starting off this new project. Right. By the way, so how long did you end up working on training the large language models, the text-based models?

27:15That's very interesting. The period that we were engaged with ShartGPT was four months. And these four months, the client provided a contract for one year. The contract between the client and SAMA was one year. For us, we had a contract between SAMA and ourselves. The client knew very well that we will be engaged for one year. But Sama, because there was another prevailing story that was covered about content moderation that did not portray the image of Sama and OpenAI very well, some have found it fit to terminate the contract between them and the client so that they can protect the image so that they don't fight two wars at the same time.

28:28What did your team say when they were selected to move on to the project, but you were not? They were very frustrated, especially considering I was, you know, every now and then I kept on fighting for them, whether it is medical covers. I stepped in to fight for, to ensure that they were covered medically. for those deferred salary that I mentioned earlier on, I stepped in to ensure that the salaries were actually paid on time and not delayed. For any issue that prevailed, I stepped in to ensure that the issues were covered and we don't see a repeat of the same issue over and over again. Because the things that I'm talking about were perennially happening from month to month, from day to day, from week to week.

29:46And you could see that there was a commitment in trying to frustrate the team and the agents to a point where they are pushed to the wall. But for us, our patience was expressed in a very mature way. My team was very mature in expressing their patience. It was my responsibility, however, to face the management and tell them, look, whatever you're doing is wrong. And it should not be happening in a day such as this. It should not be happening to people who are vulnerable. My team was vulnerable. They were not being paid the salary that my supervisors were being paid. Yeah, how much were they being paid?

30:41My team? Yeah. My team's salary ranged between$18 ,000, which is$180 per month, to$200. $180 a month? Yes. But how many hours did they work? That's nine hours. Nine hours a day? Nine hours a day. Five days a week? Five days a week. Sometimes we will even report on Saturdays. Four weeks a month? Yes. That comes out to 180 hours. Absolutely. Are you saying that the salary for the folks who are doing this was a dollar? dollar an hour exactly god just give a sense as to like is that is the purchasing power in nairobi for a dollar um let's say you wanted to buy like a let's let's say you wanted to go out for a meal at a restaurant for instance what does that cost a meal will cost averagely it will cost $5.

31:47Okay. So it's, there is more purchasing power than a dollar in the U S but not dramatically more. I mean, not dramatically more. So it's equivalent to U S dollars is, I mean, basically like paying people$3 an hour in the U S, which is, I mean, to me is anyway, it's unacceptable. Okay. So what happens to you at, at Sama are, are you left, right? Yes. So I have a friend who is a director at Sama. He's the director of quality. I have had the privilege of growing up with him. So, you know, we have that, you know, rapport. We have that good and cordial relationship with him. However, I came to know his true self right after engaging him at Sama.

32:52You know, you will think that someone like him will fight for me and, you know, try and address issues related to agents very strongly. But that was not the case. So, you know, remember earlier on, before all this happened, there was a prevailing case between, sorry, there was a prevailing story that was being told by one of the journalists in the U.S. from a leading media house about the situation at Sama and Facebook. Yeah, this was that time. Yes, so this was that time. So our story just came after, came while this story was actually being told or being drafted. So the director looked for me with the insinuation that he didn't want me to talk to any journalist because the journalist as well was looking for me.

34:20The journalist who covered the story about Facebook and Sama was looking for me. So the director knew that the journalist was looking for me. So he reached out to me. The director reached out to me to find out whether I got something or I got any engagement. But all through, I have been lying to him. playing around with his integrity, telling him that, you know, yes, I got something. You don't have to worry. Then he later on told me that there is a journalist that is looking for guys who worked for your project. That is ChatGPT. So I asked him whether the journalist is the same person I know. I know.

35:18Then he said, yes, he's the one. Then he said, I can see you actually on the bench, but if something comes up, you will be the first person we will consider. Then I said, at the back of my mind, I am saying, this is a lie because I've already cleared with Sama. I've already done my termination, my exit, yet I am appearing on the bench. It is ironical. It is something which is funny and hilarious. So you were basically out at that point? Yes. And they were trying to put you back on the bench to mess with your severance agreement? Or what was the idea there? Or just to intimidate you? Yes. So, um, they, they, they, they wanted to place me on the bench, um, so that I can, you know, I can relax or, or save the, you know, I can be that person where every, anything that comes my way, I, it's a yes, uh, or a yes, mom, uh, kind of response, you know, if you're on the bench.

36:37Yeah. If I'm on the bench. Yes. So that I will be submissive and take anything in. But that is not me. Okay. So at the end of the day, you decided to leave on your own. Absolutely. And when was that? When were you actually officially out at SEMA? So that was April 2021. Okay. Sorry, April 2022. Okay, and about six months later, in November 2020, Chachi PT comes out. And next thing you know, it's the fastest growing consumer product in history. 100 million people are using it within two months. What did you think when you saw that? A lot of things are playing around in my mind. You know, we need mechanisms put in place, mechanisms that will protect content moderators all around the world, whether it's chat GPT or anything, you know.