The Privacy Nightmare Hiding Inside Every AI Chat

28:51When you have something like Lumo, even on the client side, we also encrypt the index. All the indexes are encrypted too. So even if someone came onto your system and started looking for something, it would all be encrypted because they can't decrypt it without your key. So that part makes debugging a bit harder, which means that we build things into the tool which allow people to debug. So there are hidden functions that we can tell people to control, shift, and P, for example, or bring up a debug view. You can inspect the index and see how we process your documents. and also for example for large files you know if you have a large file with let's say 300 000 tokens so a token is roughly like four or five characters depending on how you want to compute it it's not an exact science but let's say it's around four or five characters if you have a book for example and upload that book you cannot just throw the whole book into the context right it's it's going to saturate the context and you the lm will just drop information it will probably drop your question it will drop maybe half the book or more and it will also be slower because it will require more computation to ingest all those tokens to do the inference in the end so what we do is basically if you upload a large document or a large book or whatever else, one of your books hopefully not copyrighted material at least you're on this, okay you upload it into Lumo and basically what we do is we chunk it all up into sections automatically for you.

30:28So whenever you start querying that for something, we find the right chunk in the book and we insert the right information into the context, not the whole book. So this way we can really deal with larger files and people can do more, I would say, with less. That's the idea.

30:48Eamonn Maguire:It's pretty cool. I mean, the index, the foundational index was built by another team in Proton called our foundation team. And that index will also be used across other products. But Lemo's use case, for example, is a little bit different from other products because most people, when they type, for example, or search for something inside mail, they might type one word, right? Or two words, or the sender or something, and they search. That's it. and in LUMO it's a bit different because you type you're prompting you're asking a question and you have to be able to extract out the information you need and then find the relevant information inside the document but yeah that part wasn't supported at the start by the foundational index so we also built things on top of it to make this work and now those things also go into that index as well.

31:46So it's very sort of cyclical in terms of development, but also we learn a lot as we go along. But in the end, it works pretty well. I think I use it now. I didn't use projects before, but now I use projects for almost everything.

32:03Eamonn Maguire:Yeah, I'm the same. Like when I'm using any AI tool, I have to use projects to organize, you know, the information, the context, you know, the system instructions even. It's just a much simpler way to work. could we maybe take a step back and just cover very briefly so what all can you do with Limo? We talked about projects I think you also have a CLI potentially that you just launched yeah just give us maybe a broad overview of everything that you can do with Limo We don't have a CLI yet actually we do there is a Sorry, that's past CLI? Is that? It's a past CLI, yeah past CLI is really cool by the way they did a really good job Could you explain that because I guess I didn't know what it was the the past cla is more of a way of of controlling uh getting access to your passwords on the the terminal i saw it so if people are connecting to remote servers for example they don't have to like go and find their password somewhere and come back again they're able to inject it there or they can also use it for for secrets for building projects and so on building other projects just not generic projects um right you know building your code base usually you've got secrets that you need, like environment files, you can basically use a pass.cla to inject that information into the environment variables so you don't have any secrets lying around.

33:23And it's really, really cool. That was awesome.

33:27Eamonn Maguire:Yeah, that's one of the intimidating things because I've been teaching myself programming with using AI over the last couple of years and the environment management, the environment variable management is always the most intimidating part to me. Yeah. it's an unsolved problem up until maybe now let's say it's not unsolved, there's many solutions there's Bao and stuff like this for secret management we have Vault and stuff too inside Proton but yeah, the past CLI came out a few weeks ago and a lot of us are now using that for secret management that's really cool. I'm going to try it but Lumo in general is, I mean our features so far been sort of focused on the core things, you know, real-time information lookup.

34:14So we can go off to the web. We can ask financial questions about stocks or whatever. We go off to a different service and pull that information. Basically, the primary purpose is to provide a tool which is asking or answering your questions. The connectivity to drive has been there since the start. we sort of we understand that context plays a big part in ensuring that responses are more accurate. That comes from real-time lookups, from things like search APIs but it also comes from the files that you already have and the information that is only known by you for example, which projects you're working on what's your CV so you don't have to keep repeating the same information over and over again these things are kind of important to really ensure that the results that are given are relevant and projects is kind of an evolution of that to provide even more information on top so instead of having to upload files in every chat you just have a standard a collection of things that it now has in memory that you can use to to answer questions better and get better results also we have things like personalization you've got the whole favorites thing, you've got search across all of your chats and so on, all the standard things but I think in the start, in ghost mode too of course but at the start really what we were building is something which is providing the bread and butter requirements of people they need to be able to use these types of tools to answer certain kinds of questions, to do most, 90 % of the work that people are doing day-to-day with things like ChatTBT and Cloud and so on.

36:08But then put that into

36:12a privacy-focused tool. And now what we're building on top of it is all the rest of the features, the vision capabilities and so on, are going to be coming in the next, hopefully within the next month or two. That's awesome. Yeah, and that's like image ingestion, video ingestion, image generation.

36:35Eamonn Maguire:So that means they'll also be able to understand images and videos when people share them. Yeah. Yeah. And, you know, these things are kind of, I don't think image generation particularly is used a lot. but for example one of the things that happens like last year for example was if you look at some of the trends the viral trends that were happening with OpenAI you've got the doll box thing box yourself doll I mean these trends they served a purpose they were actually harvesting your information so to be able to create these dolls in a box type representations you were uploading pictures of yourself you're uploading context so about what you're interested in what you like what you did for a job all these things were being uploaded and yeah we actually ran a story on how that's like a privacy nightmare giving it all that information yeah people did it anyway because it's viral right then and it's fun and it's cute and adorable and it's like you know you get an ego you get to show off a little bit yeah and at the end I mean if everyone has it does it really matter I don't know I never got into that trend but yeah I typically am not very viral I would guess but then if you look at also like Studio Ghibli trend that was another interesting one people are uploading pictures of themselves with their families for example so again a different data set that could be taken because all of those pictures weren't necessarily available to companies like OpenAI and Cloud because they were maybe on Facebook for example or in google um they all of a sudden you had millions and millions and millions of people uploading their personal pictures to ghibli-fy them and then you've got all the copyrighted stuff around like actually the recreating the style of ghibli itself which is also questionable right and then you've got other trends like you know uploading uh landmarks and then asking can you guess this landmark and then people are basically they've gotten really good at this like you've seen where like you can show a picture of a boat on a dock and maybe within you know one or two guesses it can figure out exactly where you are yeah and you know the the way they did this is because they provided that viral trend uh quiz type thing where you uploaded a picture it was you basically say um chat gpt will respond i think you're in paris and then you respond no i'm actually here and now it has labeled data for free but then people are using that as their trend um so i think they they were pretty smart i think they they did very good viral trends and they got a lot of data at the same time of doing it but people haven't realized well many people some people realized many people didn't realize the privacy implications of doing this and yeah one big part of our our mission is also to raise awareness of of what can be done with the information that you give.

39:41It's not benign information. It's not like you should care that people have this data because they're not just constructing one data point on what you're doing there. You're uploading many pictures. You're uploading, you're talking about your life problems. You're talking about your work problems. And all these things can be used against you. Not necessarily now, but also later.

40:06Eamonn Maguire:I wanted to ask that. What do you think is the biggest threat? I mean, you know, is it that, you know, at some point the data gets used maliciously by the companies? Is it that now this is like a huge target for hackers? What do you think is the biggest threat of having this data out there? Well, I mean, there's the fundamental picture around privacy and the right to have ideas, I would say, or the right to have thoughts or the right to be able to discuss things.

40:40Imagine someone legalizes or illegalizes, de-legalizes, makes illegal. Makes illegal. say something which is kind of benign today and then the US government or someone else goes and subpoenas ChatGPT for all the records of people who are talking about that thing. Then they go and arrest everyone. Would you be happy with that? Probably not. If someone is homosexual, for example, and they hadn't come out and they're talking about it because they're a therapist or whatever, as a chat UBT is a therapist or some other system and then someone asks for all the people who are talking about these types of thoughts and then goes and arrests them, would you be happy with that?

41:28Or would you be happy if someone, like basically I did everything that you talked about, like about your problems at work or about how you didn't like someone or your political ideologies, which maybe it might change over time. So the temporality of information and the permanent record or the right to be forgotten in some cases is kind of removed if all your privacy is also removed. And, you know, people are using tools like ChatGPT for really serious things. It's not just saying, oh, I would like a cat. What color should it be or something? they're writing that they have some mental health issue.

42:14Yeah.

42:15Eamonn Maguire:Health data. People are talking about how they upload tons of health data to the AI. Yeah. And these, I mean, they might be able to create something that looks relatively sensible because they're kind of tuned to create things that look plausible. But at the end, it's not something that you should ever use for an actual medical decision. There's many cases now where they do benchmarks, marks where these tools like chat gpt or clod are basically just hallucinating so much stuff about personal information that's being uploaded and it's not true like if you go if you type in any of your symptoms into google um basically 90 of the time cancer will be probably one of the things that comes up as a as a possible uh thing right because they have so many symptoms um so relying on people I think there is a place for for helping the health system because the health systems across the world are overloaded but I also think we should be fearful against something which is harming the health system by creating lots of false positives you know if people get scared and then they all go to the doctor because everyone's turned into a hypochondriac that doesn't help anybody so right it overloads the system yeah and all these false positives and every single environment is overloading systems.

43:41So it's a sort of a grandiose idea where you think that you're building something which is going to be a help, but then it's actually turning out to be a hindrance. So maybe with medical professionals, people who are trained using the system to make them faster, maybe. But how much faster is still something that people don't really know, right? They don't know how fast they're going to be by using these types of tools.

44:07Eamonn Maguire:Well, from a technical perspective, I mean, is there a way to make it accurate enough eventually in the future? Like where a user could have, like, let's say a private chat with Lumo Health, if you ever launched that or one of these, and they could get accurate information from it. There's always a probability of getting accurate information. The question is going to be, is it accurate most of the time? Yeah. and none of the tools are accurate enough at this point in time. A lot of these tools, their objective function at the end of the day is to create interesting content, and that need to create, and interesting usually means more novel.

44:53Now, novel is typically created by going off script slightly, which is why you have this temperature parameters and models. so basically you know if you're generating a token you'll basically have token one and a bunch of lines back which is basically going to be your attention to other things inside your context and then from that you're going to predict the next token right and then the next token has a you have this temperature function or temperature parameter which is basically saying should I pick always the first token so it's very predictable or do I pick a token which is a bit further down the list, a bit less plausible?

45:38And by having a higher temperature, you end up creating more novel content. And more novel content is very good for creative purposes, but it's not necessarily so good for risk-to-life type purposes. So you want something which is predictable. You want something which is repeatable. And that's kind of the science goal. But these models are not particularly predictable. and they're not repeatable. You know, you can run these models multiple times and you'll get different tokens unless you really basically reduce the temperature to zero. And that's the thing that these models are, you could theoretically create a variation of these models, which is not very innovative.

46:23That's just pulling in the information and outmitting the same thing each time. But it probably wouldn't gain very much interest from the population because it would be super boring. Yeah. So that's, you know, the hallucinations and randomness and so on, that's what kind of makes them cool because it's not disgenerating, you know, like your autocomplete from before on your phone, which is always just for the previous word, what's the next word, which is kind of more obvious. This is like from the previous collection of words and plus all the things that those words might be related to, what is the next one?

47:02And the more variable you are on that, the more interesting it can be. So I do think that there is ways of doing it. I mean, there's a whole bunch of work on like entropy and also looking at the variability of the result and looking at ways of verifying the result. You know, you've got knowledge graphs and ontologies in ways that you can actually compute over this data or basically ontological representations of this data to understand is it logically consistent. People have spent a lot of time building these systems already way before we got into LLMs. So verifiability is a huge problem and there's a lot of smart people working in that.

47:47So I think that these people will probably come up with some solutions to this. but information is messy and information is hard to organize and information changes over time. That's fundamentally we're learning all the time. So the thing that's true today might not be true tomorrow. How do you update your knowledge so that you understand that the thing today is the thing which is true but the thing yesterday is the thing which is now not so true anymore. And a lot of people have trouble doing that. They have trouble changing their belief systems. And models are also the same. They have trouble changing their belief system as well.

48:27So, yeah, I think we can get to better systems. You can get to better systems by having less randomness, models with a definite objective function, but also by having solid verifiable outputs, leveraging more of the community in the knowledge graph community, for example, or semantic web community. They've already been building a lot of really cool stuff over the years, and I think they're becoming trendy again. So semantic web used to be Web 3, I think, at the time, and then instead crypto hijacked Web 3, and then everyone forgot about the semantic web. Right. I think the semantic web is probably going to come back again.

49:14I think it's the silent hero of AI. I think.

49:19Eamonn Maguire:So I do want to touch back on the privacy concerns with, you know, today's, you know, the AI that people are familiar with today versus what Lumo offers from an enterprise perspective. And I'm curious what you see as the biggest benefit for a company going with Lumo and why they would want a private AI in an enterprise situation. Well, I think if you reframe the question a little bit to be more like, would you be happy as a company giving all of your information to Google or Microsoft or one of these big central players? What is your, what is your USP? What is your, your IP? Is your IP important?

50:02For most companies, they're, they protected their secrets very well, right? They, people get fired for, for real. leaking information or leaking information about possible some mergers and acquisitions, for example. And a lot of people are just uploading whatever to these systems without thinking in the same way as they did before. And I think part of that is because they're kind of conversational. So it feels like you're speaking to someone, but not really somebody. They're useful for sure. So they make people speed up. So it's the whole thing about, you know, it's the efficiency drive. So, you know, it's convenient to be able to use these tools.

50:45It's inconvenient if you can't use the tool. And the convenience comes at a cost. And usually that cost is your privacy. And, you know, it's the same as using, you know, Google Search instead of using, say, Brave or something else, right? Google search has got all of your history I can use that to search it's going to have better results because it's got all of your history but then people find it hard to switch to something like Brave because Brave doesn't you might not get the best results because it's not got the same way of treating your data as a company like Google I mean in the end I think people should just think about would you be happy just passing off all of your information, all of your IP, all of your strategy information to someone that you have no idea who they are.

51:42Before it was the similar thing with cloud computing. I remember it's like there was always this joke, cloud computing someone else's computer. It's like, are you happy just uploading all of your stuff to someone else's computer? And it not being encrypted.

51:59Eamonn Maguire:The equivalent here would be someone else's company maybe. Or someone else's company. and that's what people are doing right and maybe the risk is low maybe they probably don't care about your company most of the time they probably don't care about your company but information is power right and information is important and especially these days you know let's say that it's a complicated world in terms of trade and markets and there's a lot of disruption there's a lot of distrust I mean there are no guarantees anymore over how information is treated or how it can be used and some of that stuff steps into sort of conspiracy theorist territory perhaps it sounds like that but some of it's really real right it's it it is real and people have been doing corporate espionage since time began espionage in general since time began information is incredibly valuable and if people are just uploading you know millions of companies inside europe millions of companies inside um africa or australasia everywhere else and they're uploading it to us um servers who used to know what the us could do with that information right in terms of global strategies or you know what governments are uploading and so on it's basically negates the need for actual espionage because they've just given them all the data for free.

53:30Eamonn Maguire:It's like you're no longer worried about IP theft, you know, from sharing information in China, which they've been accused of being IP thieves for years over there. Now it's like, well, everywhere with an AI model can be an IP theft. Yeah, it's a big concern. I mean, and people don't think about it that way sometimes, but I mean it's if you just step back a little bit and think it's the same for privacy from a B2C point of view you know everyone was like if you watched a James Bond film and you watched him come in and plant a bug and then he's sitting there waiting to hear what you're saying all the time or taking pictures of the documents that you have and then if you transform that into your real world where you're uploading all of your personal thoughts and feelings and everything is going into Facebook and into Instagram and into WhatsApp or whatever else I mean you've basically done the job for them they don't need to come into your house anymore because you've just given them all the information that part is kind of scary and if you I think with the way the world is going I think I'm never I'm never one that would advocate for a closed world I think open world has been incredibly valuable for travel and opening minds and everything else and it would be nice if you can trust everyone to do the right thing but I don't think you can in the end all markets are competitive everyone is looking for the age if the next big idea is sitting out there and it's you're uploading it into chat gpt and somehow someone gets access to it because of some leak and they don't have anything encrypted so it's just there how would you feel probably not great so even if they don't use your data for training I mean centralizing all user data to these small number of providers, holding it in a way which is not end-to-end encrypted, for example, where people could get access and see the information, that's a risk.

55:27So if you really care about risk, don't use that. And they say it themselves, right? I mean, Sam Altman has come out numerous times. They do not upload sensitive information to ChatGPT. I think this guy is a marketing genius or something because he basically, whatever he says to not do, everyone does it more. So I don't know what's going on with that.

55:49Eamonn Maguire:Well, I guess it kind of goes to the idea of the perfect personal assistant, right? And this is something that Google is working on now with the personal intelligence, which I mentioned at the top, is that, and connectors as well, where for AI to actually be maximally helpful to you, you needed to have as much information possible, like as much context possible about you and what you're trying to do. And, you know, with tools, give it as much access possible to manipulate the environments that you need it to work in. But that's an entire, you know, threat in and of itself, you know, from a privacy and, you know, potential security point of view.

56:31Eamonn Maguire:So how do you balance that? And maybe, you know, you can share a little bit more about where you want to take Luma from here of like the need for as much access and context and power as possible to be as maximally useful as possible, but also as private and safe as possible. Yeah, one of the big parts, I mean, for example, whenever someone does a web search, for example, when people do a search in the web at the minute, they type in the words that they intend to type, and they get the result back. When you ask any of the AI systems something, and it goes off and does a web search, is it going to search for the thing that you wanted it to search for or is it going to strip away things that you didn't want to specify for example if i talk about my health health conditions does it strip away the health conditions from the search that it's going to send probably not because the health is super relevant it's the same for you know removing pai for instance from from queries that's much harder than it seems i mean if i ask it uh who is grant harvey for example you can't remove Grant Harvey from the query because that's actually the part which is important for the query if I ask it about you know who is Eamon or tell me about me or something if I'm egotistic and want to know everything that the web knows then basically you cannot strip away this is PAI because it's there it's the person that's actually doing the search so you can strip it away so any guarantee that people say they can do you can't really do it because actually it's highly contextually dependent um when you get into things like agentic systems for example and autonomy and being able to go off and do things without you even realizing they're doing it if people should be pretty careful and my i'm very uh risk averse in general um and I would never connect one of these agents to my bank account, to my you know an Amazon account or whatever else that you have, you have no idea what's going to happen.

58:42I have multiple people that have connected agents to their calendars for example and has basically deleted everything in their calendar or the opposite which has basically filled up their entire calendar with nonsense and then they had to go and manually delete everything afterwards. so these systems they're all super new it's a very exciting time and people have a lot of fear of missing out but sometimes it's better the best thing is just not to jump in too quickly the systems are being worked out the security is not really there i mean people are still trying to figure out the security of these systems and how to put the guardrails in place and stuff as well so my advice would be not to get too swept up by all the the the hype and wait for the actual real use case where this is actually going to be really useful.

59:28Do you really need an agent to go off and book your holiday for you when it's like two clicks on booking.com or something?

59:36Eamonn Maguire:Right. All things in perspective I think. Yeah. I mean making yourself more efficient is not always the best thing. I mean most there's like some measure of like it's very philosophical now but some measure about how many billions of hours people play computer games per year. People have time. It's like time is not the problem. It's like maybe some things you don't want to do like fill out your tax forms or whatever else, but there are things that you have to do once a year and doing them well and doing them correctly is important because if you don't do it well and correctly, then you might be in trouble.

1:00:12So yeah, I think people just need to understand what they really need to optimize and what things are actually important and not to really get so swept away with the fear of missing out on on trending new things just because they look cool because everything that looks cool basically at some point in time they've they've caused some severe security issue and then uh you're going to be regretting it uh later on it's the same for no internet of things and people putting smart cameras everywhere right and then basically everyone was hacked and they've got entire websites now where people can just hack into their um people's webcams and just watch kids in their room or whatever else.

1:00:53Eamonn Maguire:I did not even know about that. That's horrible. Yeah. And that's just because people, they created these systems, they created software, they created software or hardware also with bundled software with very weak default passwords and so on. And admin is a password or something for all these internet connected devices and people can just get into them. And yeah that's that's actually shameless plug could you use cli pass for for managing your like web passwords as well you get a web password you can do that with the normal pass extension yeah okay cool yeah yeah so that's your browser extension you've got pass as well it's really good i i highly recommend pass yeah so if you if you're one of those out there who has admin as your password please please change that as soon as possible yeah I think one of the other things I think that's an interesting trend talking about pass and browser as they know like things like the Atlas browser from yeah what are your thoughts on that yeah I mean it's a natural extension of what you want to do right so you know the same for Google like Google had search and then they say well, people do lots of things that are not on search.

1:02:12How are we going to track all those things too? I know we'll create a browser or I know we'll create a phone or create an operating system for a phone because now we've got access to everything that people are doing, what apps are looking at, what they do after they go to that website, what they click on. Everything is trackable and it's trackable because they knew strategically that they needed to go into something which is at a lower level than the browser, it's like just the web search, right? And ChatGPT have the same issue, right? They want to get more information. They want to be able to lead people into purchasing, for example.

1:02:56So imagine if you basically go from a model where you've got sort of like a service-based thing, and he's like, tell me about smartwatches, for example. Google shows you a list of smartwatches, and then you go off and do the work yourself to figure out which is the best smartwatch from different information sources and so on. With these types of tools, basically, you've kind of gone from, you change this whole operation to conversational engagement. So now someone asks you a smartwatch. Oh, what do you need it for? I need it for this because I blah blah blah and then it keeps going and then it might suggest some watches that are maybe someone's paid to insert them inside there right that would be perfectly fine but in the process of doing that it's also gathering lots and lots of information about you know not not just that you're interested in buying a watch but also why you're buying a watch and that can allow you to also say well what else can you sell that person to or what can you learn about that person?

1:03:59Are they insecure? Are they whatever? So I think this is super dangerous because it's not just about Google. It's more dangerous than Google, I would say, which is interesting. You're not just saying I'm interested in this, but you're actually having a conversation about stuff. And then you can start using those conversations and following off things. you know if you've got some depression then maybe it was going to recommend some particular therapy from some guy that advertised on chatty bt and so on but you know showing people what's happening why they're seeing that information so on is a super important and being transparent about it is there

1:04:47Eamonn Maguire:a way to do this privately like is that like would you ever create a secure agentic browser or even like an agentic terminal tool like is there a way to do it safely

1:05:01That's hard to answer. I think people, we would provide Lumos an API where people can build things on top of it, build agents on top of that. You could use that then as a driver or as the agent itself. Fundamentally, one issue with agents is that they are inaccurate. right if you give it a task there's going to be a probability that that it's going to do the wrong thing the the issue is that many people aren't using one agent they're using multiple agents and chains right so when you have chains of agents you've also got chains of of inaccuracies chains of failure so by the end by the time it gets to the last part of the chain you're probably sitting at 50 50 if it's going to be the right choice or not

1:05:50Eamonn Maguire:I think that's what meters data tests, right? It's like, does it do the task as well as a human like 50 % of the time or more? Yeah. And you're doing this, if you look at the way people chain information together, there are people who are doing, I think, a good job and sort of deploying agents in the right way, but more for like operational security and this type of stuff and workflows. But there are people who are just deploying agents and not really understanding that they just treat them as some oracle system that's just going to magically come up with the right answer each time. But anyone who's ever worked with them will know that they're not accurate and they're not supposed to be.

1:06:34They're built upon technologies which are not built to be like that. But people are trying to repurpose them to be like that. And they will succeed probably in some thing because it is still early days and people will spend time making them work. but they have to make them work. They're not getting it for free, I think. And that's the people think that, you know, you're just going to get a free lunch by using these systems but you still have to do work. It's the same people who think that you can just build an entire software stack vibe coding something. It's not in any way true. I mean, sometimes you might be able to build something.

1:07:11Yeah, I mean, it's really it's exciting as an enabler but you need the direction and one of the things that I always say internally but also in other venues too is that companies for a long time especially the big companies they have people many tens of thousands of people who are very strong engineers but the value of those engineers some of them can work autonomously Some of them will generate things by themselves, which are incredibly valuable. Innovation can spike from, you know, just the ideas that come from this, from bottom up. But fundamentally, from a company point of view, it always comes more from the top down.

1:07:58What's the company direction? What's it going to do? Is that going to be successful from a revenue standpoint and so on? But those are decisions that people are making and then other people execute. if you have an agentic system and you basically replace your software stack with a bunch of agents or software developer team with a bunch of agents you still need to give it the direction and you still need to know what makes good software like being able to design systems and be able to understand that that system that this agent is creating or the set of agents is creating is actually sensible or not is incredibly important and these systems they they don't create good code.

1:08:40Many people will say they do create good code, but I would argue that they're just not good coders, so they can't evaluate properly. But sometimes they do. I mean, the vibe has shifted a bit

1:08:51Eamonn Maguire:with Opus 4.5. Have you tried it? What's your take on it? Not to tangent too hard on that, but... I think what they're doing is pretty cool. I think the models are getting better, but they still over complexify things. Yeah, I've heard that too. I mean, their job is kind of to be, I would say, more verbose. They go for verbosity instead of simplicity. They still don't typically go for reuse. They will go for regeneration rather than reuse of existing libraries within your stack. So you end up with a lot of fragmentation. So a lot of the time, yeah, you can generate code very quickly. but if you actually look at the code that is being generated you can take maybe 200 lines and compress it down to 30 sometimes for the way it's generating and that's not just because it creates lots of comments but also because it's doing a lot of things inefficiently and one of the things that I'm you know it's not my job anymore I would say but that I was most expert in was like in the data visualization side And I was very good at making D3 work incredibly well for me or things like processing and so on.

1:10:14But to know those systems, they have particular paradigms and they have ways of doing things which are much more efficient than what you would get from typical. There's a good way of doing it and a bad way of doing it. And there's more bad code on the web than there is good code on the web. So the training dataset is typically biased towards bad code and less towards the good code because you can hack something very quickly or you can design something very efficient using the same piece of the same fundamental library. And with D3 in particular, I can see that whenever you generate code with these types of things, they are very inefficient.

1:10:54They don't use any of the paradigms really correctly. and if you extrapolate that out to everything else in just this one library and how to use it effectively and you go out to every library and how to use that effectively, you quickly get to a state where you end up with a lot of inefficient code. And in the end also, you have to question the operating model of these businesses, right? Because, you know, Cursor and Claude and whatever else, they're working on tokens generated and cash reads cash writes this type of stuff this is how they're they're monetizing a lot of their stuff now as a business does it make sense to give you back the right answer the first time some people would argue that it doesn't is this then they don't make so much money you'll make you'll make any money at all probably your first response is good so if you you start then differentiating between between the people who can prompt very well and get the right answer quicker versus the people who can't prompt very well and are very, you know, it's the same as using Google.

1:11:59I was super good at finding the answers in Google before. You sit the same person down beside me and they'd ask a question to the both, to two people, the two people. And 99 % of the time, I'll find the answer much quicker than the other person because I was good at using the search. It's a bit the same of using, like prompting and understanding what you're changing. how you're changing, the scope of the changes, what's the specification, how it should behave. If you can define these things very well, you can get an answer very quickly. But most people cannot define them very well. So it ends up creating many, many loops and a large cost.

1:12:40And at some point in time, people will have to decide, is the cost actually too much? Is it going to cost more than actually having a developer at some point?

1:12:49Eamonn Maguire:Right, yeah, because at this point, I think it's a bit of a bit of a wash and certainly they're subsidizing it, you know, at the moment. But when they go public, those prices are going to go up. You better believe it. And then you have a problem of centralization, the dependencies and stuff that people are creating. It's the same for every part of the economy. If you start centralizing too much, you create risk. And people have to think, what would happen if that company disappears? Are you prepared for basically your whole software developer team leaving tomorrow? Wow. And people say, well, yeah, I can switch to another one, but they're all dependent on each other, right?

1:13:33Cursor have their own model, but they're dependent on Cloud. They're dependent on ChatGPT. They're dependent on Grok for their fast models, whatever else. So you've got all these dependencies that are being created. and if one of them goes down then basically you lose quite a big chunk of the ecosystem if cursor disappears which you know with the competition now from Cloud Code for example, apparently their usage is dropping, so you have these things which are these very early stage companies VC funded, not profitable at all people who lose interest go on to the next one, that company disappears and the other one comes up and then you lose trust, you create risk and you create sort of inertia.

1:14:19Eamonn Maguire:So yeah, you guys have been shipping a lot lately. One of the other things that I saw was Proton Sheets. I guess one of my last questions here is, what's next? Are you going to completely recreate the entire Google ecosystem and compete with them? Or what's your plans from here? I think Proton Sheets is a natural evolution from what we're kind of offering as part of, you know, providing not only businesses, but also end users the opportunity to use things like spreadsheets, which people are using for their taxes, for financial planning, for, you know, for business and uploading to Microsoft under Office 365 and so on.

1:15:06or google sheets and basically we thought that sheets is is an important thing to build because it's it's what people are using a lot maybe not everyone uses spreadsheets a lot but i use spreadsheets a lot like i have a t-shirt somewhere saying i use spreadsheets a lot

1:15:26Eamonn Maguire:i would buy that because i definitely use spreadsheets a lot and you know people especially with the type of information that goes inside also business results it is confidential information people are using other tools instead of using something that Proton had before because we didn't have Sheets so it was a gap that we needed to fill in and I think that Sheets is a really good step towards filling that it's a nice product, it looks good it works well and also our ideas is to integrate LUMO inside these types of tools as well. So provide people with the similar types of things that you can do with Gemini, for example, inside Google Sheets.

1:16:13You could also do with LUMO inside Proton Sheets.

1:16:17Eamonn Maguire:Awesome. Yeah. So maybe by end of the year, we're closer to full integration across the suite. Very cool. Love that. So I understand that you have some news that is launching around the time this comes out related to Proton's new product or service called BornPrivate. Can you tell me a bit about that? Yeah, sure. So the internet today is, I would say, unrecognizable from the one millennials grew up with. So it's no longer really just a communication tool. It's a data harvesting machine with an AI engine now attached to it as well. So BornPrivate is sort of a response to that reality. From a fundamental point of view, it's quite simple.

1:17:01Parents can reserve a private encrypted email address for their child at birth for$1. Basically, it's just to lock their foundational digital identity outside of the big tech ecosystem before the data harvesting even begins. But primarily, it's more about educating around a starting point. An email address is sort of the anchor for most people's identity online. Like you log into your Facebook account with your email address, you log into Instagram to whatever else with your email account. So it sort of makes sense that that's the foundational part. But I would say that the bigger part is around educating people from the beginning, even before birth, about what data can be harvested about your child before it is even born.

1:17:55And then what can be done afterwards whenever all that data is available in the ecosystem for deepfake generation or whatever else, right? So that's kind of the point.

1:18:05Eamonn Maguire:This kind of reminds me of Tim Berners-Lee has this concept of like the data pods. Have you heard about this? Are you involved in this at all? Not involved, no. I know about it, the solid technology. Yeah, that's right. Yeah, it reminds me of that. And I love this conceptually, but I've yet to see it like take off, which is really the idea that you control your data and it's all like on your in your pod and you choose who you get to share it with would this work similarly to that in any way or is it just like you don't share the data ever basically unless from the from the proton point of view it's always about your data is your data that's it we have no interest in and doing analytics or profiling whatever else on top of you.

1:18:49From the point of view, the email anchor is there for access to other tools, for example. You can start off with that. But, for example, if someone goes off and uses that email address then to log into Facebook, then, of course, Facebook's machine is then collecting data on top of you anyway, right? But it's sort of... If you start off with the premise of what born-private means, and you get people to understand the risks around how they're sharing information online, even just with this initial step, it could have a big impact on what parents do afterwards, right? So I was saying before, like a child's digital fingerprint sort of begins before birth.

1:19:30So your parents start searching online, for example, for some things. They might do like, you know, your gynecologist emails. You might have emails with the pediatrician after birth. You can also have ultrasound photographs. You can have lots of medical results that are going across the wires before the child is even born. And then by the age of five, there's a stat, I think it was from Dutch Telecom. Whenever Dutch Telecom did this big advertising thing around two or three years ago, which is super interesting, where they sort of took a photograph of a young girl and then aged her until nine years old and then had her go back and educate her parents, basically on the dangers of sharing her pictures online.

1:20:13So by age five, the average child is around 1 ,500 pictures of themselves online, uploaded without their consent, of course. And then by age eight, whenever I think it's like 51 % of kids get their first device, it's already too late to start from scratch, right? You've already got much of your digital identity is already known, and we can only guess what people want to do with it afterwards, like profiling and advertiser profiling, seeing what your interests are. They already know it before you even start using these tools.

1:20:44Eamonn Maguire:This is made very serious, in my opinion, based on the news that has been happening over the past couple of weeks. So we recorded the first half of this episode before all this happened, but then recording the second half after Dario Amadei at Anthropic made some big stand about you know not wanting their cloud to be used for your mass surveillance and we found out a little bit of what that looks like because they wanted to take all of the information that they have about everyone from all these different sources and basically like run a run a i don't know an llm call over it or something and actually like put it all together to make a like complete profile of you so it's a really this is a real issue yeah we already showed what you can do, we've investigated for a while, for example, how much information can you get from uploading an image, right?

1:21:37So an image contains your location metadata. So you usually have GPS. It contains information about which camera took the photograph. It has your day timestamps and so on. And then you've got the actual information about that location itself. So one picture can already tell a lot about what you're interested in, how much disposable income you have based on which camera you have, for example. And that's just one point. And if you start extrapolating that to the thousands and thousands of photographs that people have online, and then all the text and all the web searches and everything else, you can build a very good profile.

1:22:15but the problem with LLMs are fundamentally statistical models. They're not accurate to any degree, which is what his point was. And I think it's an important point to get across and to have a stand on that. I do think there's some hypocrisy in what they say because they also did release the health type stuff as well to sort of compete along with OpenAI. And we already know that that's not accurate. There was a recent study last week, which basically looked at how ChatGPT Health was over-recommending people to go to the hospital, for example, with very mild things, but under-recommending people with actual serious issues.

1:23:01Eamonn Maguire:Wow. So we know twice. I mean, it's not to say that the technology can't improve in the future with guardrails, but you have to be careful about what you release and how you test it and also be cognizant of what the limitations of the actual software is, the actual algorithm is. And it's not an all-knowing machine. It's not intelligent necessarily. It's a pretty decent approximation of intelligence, which is why it's popular. but using it for mass surveillance or for identifying possible domestic terrorists or criminality we already know that this thing is kind of it's silly as a concept it's also incredibly biased to what you've seen in the past and who's been arrested in the past and so on but and especially then if you go and you start using that for automatic selection of targets during war based on satellite imagery that would be pretty dangerous too right so um i laugh because it's like it's scary yeah it started it is dystopian i think i think it could be on a definitely on an episode of black mirror or something um for sure just that's quite a good predictor of the future possibly yeah right well this is cool i'm glad you're doing this my last question about about born private is you know we've seen a lot of parents actually pushing back against the idea of their kids having social media at all.

1:24:31Eamonn Maguire:Australia, I think, banned social media. I think it's all social media for kids under 16, if I'm remembering that correctly. So do you think that this could help parents with that? Like, maybe it gives, like, if a kid, if it was a kid's only email address or their only account, then maybe, you know, people could know, okay, this is a kid. I mean, maybe that gives away too much data, because it would then give away their age. But I'm curious how you think of this in regards to the backlash against social media for children and trying to protect kids from not just their data being stolen, but just the whole concept of social media in the first place.

1:25:08I think the social media bans are an interesting one because basically there's always this worry about overregulating things. Like when is regulation appropriate versus when is it not? And for the social media aspect, there is a clear health, public health risk for the use of social media. There's many studies also, internal Facebook studies also from before about the risks of teenagers using Instagram, for example. But also it comes down to the interest of the company versus the public interest. So when those two things become misaligned for an industry or for a particular company, for example, in this case, or a set of companies, then at some point the government is going to step in.

1:25:59So if the objective function of a social media company is over here, it's like creating profiles of kids, showing them adverts before they're ready, showing capable content because it's the most engaging, not providing protections against bullying because perhaps it also increases engagement as well. I know all these things are not in the person's interest, but they are in the interest of the platform because overall it increases their metrics. and whenever you have this sort of I would say the curves go in two different directions you need regulation I think to step in and help so I think I mean barn private is not going to solve these problems necessarily but it gets people it does get people thinking about it more it's more about educating people about what goes on A lot of people don't understand what happens on the web, even if there's like, there's plenty of information out there about, you know, extortion campaigns via WhatsApp or via Facebook Messenger or via other platforms.

1:27:11There's plenty of examples of people being bullied or harassed online. There's examples of people seeing content they shouldn't see. Or like, you know, the Andrew Tate type stuff, for example, on this toxic masculinity. So all these things are about, I mean, those things are about the content on the platform without even getting into people starting to share on the platform itself, right? And I think if people understand from a foundational basis and they start thinking from the time when the kid is born, what choices am I going to make? Or even before, as I said, what choices am I going to make about the privacy of this data?

1:27:52Am I worried about what's going to happen in the future to the data about my kids or about me as a person too, as a parent? And I think the Dutch Telecom advert from 2023 was really good because it was so potent in terms of the messaging. It had like, I think, 3.7 billion impressions or something like that. It was really huge. That's a lot. Yeah. But even so, people still do it, right? So people are still uploading photographs. They're still not super aware of the dangers, not just from harvesting information, but also, for example, say, sexual deviance online that are taking these images and then generating content, for example, which is also totally possible.

1:28:37And so a lot of people are not aware of these things. And born private, I think, is about starting the conversation about what it means to be private online, What are the dangers of being not private online, you know, from the point of view of, you know, companies and what they do, which is might be the behavioral aspect and profiling and advertising and so on. But also the deviants that might want to do something bad, like extortion, ransomware operations later on, you know, scams, and then also the, say, pedophiles as well, for example. so all these things are kind of there are many many different directions this conversation could go is what I'm saying you know there's the regulation the social media itself that's one thing right but then there's the use of the platform even outside of that sometimes is also encouraging bad behavior also people need to think you know not only the parents themselves but also people around them you know the relatives and so on people take photographs and then post it online.

1:29:43They do it to my daughter sometimes, and then I have to ask them to remove the photograph afterwards or they put smiley faces over her face, which also looks kind of creepy.

1:29:53Eamonn Maguire:Do you ever have to tell them, remember, I work at Proton, the privacy tech company? Yeah. Hello. I think it is a difficult thing to educate people on this because people really don't often think... You know, my job for a lot of my time, Proton, is thinking about how to prevent bad actors on the platform and to make the platform clean. So people don't like to think about what stuff goes on that's not in their interest sometimes. It's very easy just to go and watch a cat video and then forget about it. But having difficult conversations about difficult things is, in general, not easy for people to deal with.

1:30:37So they just ignore it. And I think what we'd like to do is to have people not ignore it and to think about this from the start, to make a commitment at the start. The$1 is more of a token thing. It's basically a donation to the Proton Foundation that's going to fight for these types of initiatives in the future. Also to promote, for example, what we're going to do in the future with overall education and so on as well.

1:31:03Eamonn Maguire:and also it kind of makes sure that there's a greater risk, greater chance that people actually remember about their account in 15 years. If you pay for something, you're more likely to value it. It's a psychological thing. Yeah. Yeah, so you've got skin in the game, I would say. You're saying, okay, I think that this is important and I'm going to reserve the address. My daughter has her email address as well in Proton. I don't think there's anything on there. Hopefully there's no spam. but overall it's a good step to have it, to reserve it, to promote Proton as an idea also for kids that are on to say actually there are other companies out there, hopefully in 15 years where maybe we're bigger than Google That would be awesome That would be pretty cool Do you think sorry i'll just ask one last question on this do you think that like with the rise of ai and more people are talking and um querying uh non let's call it non-human uh entities for their answers for things and you know maybe that's if if we go the private route maybe that's not even going over the cloud right now we just talked about how it is going over the cloud but imagine you know the models get smaller and they get better and you can talk to your computer and it's just a conversation between you and them.

1:32:28Eamonn Maguire:And then if people do, if the ideas that protons promoting do catch on, does this lead to a world where people just don't post on social media at all? Like their social media kind of goes away to a certain extent. I mean, that internet theory seems like it's buzzing along with all these bots responding to people now. It's almost like, where do you go? Yeah. So social media has different use cases, right? One is that it's a, it's the source of dopamine for people whenever they publish things so it's it's more saying i don't think this type of thing is going to go away i think it's going to maybe it'll be have a different incarnation but people people like the feedback right i mean you have you have before facebook you have myspace and bbo and all these things i'm old enough to remember bbo no i had myspace was my first social account yeah so i'm with you there and you know people have moved on from that to something else they moved on to something which is cleaner and more exclusive which is facebook facebook then became something which is too big and kind of old for old people like me and then they go on to things like tiktok and i don't even i don't even know how to use tiktok probably um my uncle keep sending me things on TikTok, but I don't open them.

1:33:47And, you know, you have all these different incarnations, and it's true that no one really knows what things would be like in the future, especially as maybe things move away from screen. I don't think there's two different thoughts here. One is like, is Boyce going to take over? Everything is people are just going to ask something and Boyce returns back. I think I talked on a podcast about Winston and Dan Brown's origin where it's just in your ear with this bone microphone thing and I thought that was a really cool paradigm but you know with Facebook's glasses their meta glasses whatever those things are I mean they're quite popular devices right they're like it's Ray-Ban's highest selling thing which is probably depressing from a point of view as a privacy company

1:34:41Eamonn Maguire:because you're saying well why yeah well there was a really bad report that just came out about that um that these news outlets in sweden did where they basically said yeah the data goes to kenya and it gets processed and people have no idea when it's recording and when it's not they don't know that they're sending all of this data and they interviewed the people who are actually looking at it and they're like yeah they are acting like they have no idea that this is on right now and all sorts of stuff and all sorts of footage shows up there. Yeah, so I mean, regardless of that, the social media thing and how people are using it, it will change.

1:35:15If it changes to something where it's more integrated into your field of vision, probably visual stimuli are still going to be important because it's a key part of the human input system, right? The colors trigger and they trigger emotions and they trigger the impulse to buy things, for example, which is why McDonald's have different colors because of this. So voice is going to have an impact,

1:35:42Eamonn Maguire:but yeah, vision will still have an impact. So I don't, maybe there'll be different forms of it. Maybe we'll see it, consume it in different ways. But I don't think fundamentally social media is going to disappear because it's kind of, it works so well. You know, if you look at the advertising revenues of Facebook and other, you can't fathom how much money they're making per minute just with these systems in place. We need what Proton is building to protect us if we're going to continue using social media. So thank you for your work. Yeah, well, you also need education on what to share, right?

1:36:17I think, and maybe we, at some point, which I think is what the regulators are trying to get into, perhaps, if you start considering things like social media, similar to things like smoking or alcohol or drug use in general perhaps it's a similar thing right it's even if it's not a direct you're not smoking and you're not consuming something you are still consuming something which is having a chemical effect in your brain right it's a the dopamine is a big addictive uh addictive chain the same as gambling right gambling you're not consuming anything but it's the act of being of gambling which you know we put in regulations and stuff and so on that that protect people against the gambling companies because the gambling interest gambling company's interest is to make money from you your job is also to make to make money but there's two part there are two very different diverging goals so you need regulation to make sure that the two the the gap is not too big the gambling company doesn't make too much money and the person doesn't lose too much money um and that you also encourage responsible behavior and now in the UK they ban they'll also ban it in football shirts and so on as well because even though it's illegal or maybe illegal is a strong term it's forbidden for players to bet

1:37:37Eamonn Maguire:right most of the shirt sponsors and club sponsors are gambling companies which is incredible I support Newcastle United for example and Sandro Tonali was banned from football for like 10 months for gambling and it goes into a football stadium where around the whole stadium there's just batting companies everywhere. Yeah, it's sort of like it's inundating it with you. It's everywhere. It's top of mind. Anyway, my point is that gambling is a good analogy for social media type cases where there is lots of tricks are being used to get people addicted to the platform, to get them addicted to posting, to get addicted to giving reactions and so on.

1:38:27And there's thousands of behavioral psychologists working on the best ways of getting every dollar value out of people. And we don't talk about that enough. And I think we also have to have protections against that. So regulation is good, I think, in this case. But outright bans, I'm not so sure. because at the end, people get around bans all the time. It's not, people find ways. If you couldn't get into a nightclub because you didn't have a proper ID, then people had fake IDs. Right. Yeah, so it's like, it's the same online.

1:39:09Eamonn Maguire:People find ways. Yeah. Yeah. Well, thank you so much for sharing this with us and for hanging out. So where can people go to learn more about Proton and everything we talked about today? They can search for BarnPrivate and Proton and their favorite search engine, hopefully a privacy-focused one. Cool. Perfect. Well, Eamon, thank you so much for joining us. If people want to learn more about LUMO and what you have cooking next, where can they go? You should keep your eyes and ears ready for new releases in the coming month or two, which will hopefully get people excited and also close the gap with the competition so they bring more people across to a more private internet and private AI assistant.

1:39:57Eamonn Maguire:I love that. I think it's very needed. And I think you all are doing a really good job of getting that message out there and also building useful tools for that use case. So thank you for what you're doing. Thank you. Thanks everyone for watching. Make sure to like and subscribe to this video and our channel. And for those of you who aren't already subscribed, please definitely check out the neuron daily.com where our newsletter goes out to 600 ,000 people every day. We try to tell you everything that's going on in AI for regular folks. So with that, thanks for watching.

1:40:47Thank you.