AI Unlocks Ancient Greek Scrolls Destroyed by Mt Vesuvius

0:00Today on the AI Breakdown, we're looking at the Vesuvius Challenge, a$1 million contest to use AI to read ancient scrolls thought lost time and volcanoes. Yesterday, a grand prize was announced. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our Discord, our newsletter, and our YouTube channel.

0:26Hello, friends. A quick note here on today's episode. When I dug into it, I realized that this was going to be longer than our normal main. And frankly, the brief news is pretty quiet today and can be lumped in with tomorrows, unless you are super, super intensely excited to hear about meta adding labels to their image generations, in which case I'm sorry that that's going to be a day late, but I think you will like this very unique and different type of story about the Vesuvius Challenge. Back to our normal format tomorrow. Welcome back to the AI Breakdown. Most days on this show, we talk about artificial intelligence in terms of exciting new products, or perhaps big policy shifts, changes to the way it will work, implications of technology and society, all of these highfalutin things which are incredibly important both on a personal and a societal level.

1:15But then every once in a while, we get a moment to just revel in awe of the human experiences that were totally impossible before that artificial intelligence can unlock. Today is one of those stories because we are talking about the Vesuvius Challenge. Now, I have talked about the Vesuvius Challenge before when some intermediate parts of the contest were won, but the TLDR on this is that for a couple hundred years, people have been interested in how they could read a set of papyrus scrolls that were found in a library in Herculaneum in a villa thought to be owned by Julius Caesar's father-in-law and which potentially contain writings that have otherwise been completely lost to history.

1:54Now, Herculaneum was destroyed in the same eruption that destroyed Pompeii, but interestingly, when it comes to these scrolls, they were on the one hand preserved and protected, but on the other hand melted into honestly what looks like tiny dried logs. In fact, when this villa was first discovered in the 1700s, some of the workers actually thought they were logs and burned them before realizing what they were. The Vesuvius Challenge was, in short, a$1 million contest to try to develop new AI techniques to actually read text from these scrolls without having to physically unroll them. Today we're going to tell the story of how this came about and where the project stands after yesterday's announcement that the grand prize of$700 ,000 has been awarded and that the team that won it did even better than the qualifications for that award.

2:41Now I highly recommend reading the Bloomberg Businessweek feature story on this by Ashley Vance and Ellen Huet. Ashley said it was one of the coolest stories that he had ever had a chance to work on and I completely agree. First of all, let's talk about what might be contained in these scrolls. As I mentioned, this villa was believed to be owned by a man that history knows as Piso, who was Julius Caesar's father-in-law. As Bloomberg points out, a wealthy, educated man like Piso would have had the classics of the day, along with more modern works of history, law, and philosophy. Quote, Piso is known to have corresponded often with the Roman statesman Cicero, and the apostle Paul had passed through the region a couple of decades before Vesuvius erupted.

3:17There could be writings tied to his visit that comment on Jesus and Christianity. Now, for those of you who don't spend a ton of time on this, modernity has lost the vast majority of writing that was created in ancient Greece. We have a tiny fraction of the dramas that were written, the philosophy. And much of what we do have was rescued from old, moldering, handwritten manuscripts in libraries around Europe in the Middle Ages at the dawn of the Renaissance. So the idea that there could be this trove of potentially tens of thousands of scrolls, at least 800 of which we know about, is absolutely tantalizing for anyone who's interested in classics or really Western civilization at all.

3:54Now, the villa at Herculaneum was rediscovered in the 18th century, and over the last couple hundred years, while the scrolls have generated a ton of interest, nobody has really had a lot of success in actually unlocking their secrets. Over the last 20 years, however, a computer science professor at the University of Kentucky named Brent Seals, has been experimenting with and inventing new techniques using advanced medical imaging technology to unlock the mysteries of these types of scrolls. His first attempt to handle a Herculaneum scroll came in 2009, but the software just wasn't even close to ready.

4:26In 2016, he and his students were able to decipher a charred ancient Hebrew text called the Ein Gedi, but the technique they used wasn't applicable to the Herculaneum scrolls because they were written with a different type of ink. Enter Nat Friedman. Nat was the CEO of GitHub from 2018 to 2021 and is one of the more prolific AI investors currently. He had overseen GitHub's first experiments with AI-assisted coding, and so was really familiar with this technology. During the pandemic, Friedman got really into Roman history and came across these scrolls. In 2022, he brought SEALs out to California for a fundraising event, but none of his friends or peers were interested.

5:03At that point, however, he suggested the idea of a contest to SEALs instead. With an initial investment from Friedman, matched by his investing partner Daniel Gross, the Vesuvius Challenge was born and announced last year. Now, after it was announced, a number of additional successful tech entrepreneurs also pledged money to the effort, ultimately leading to the ability to offer a million dollars in prizes. And in some ways, a lot of this story is a story of how public momentum can change the destiny of things. For example, once they got wind of the contest, the Italian government offered up additional scrolls, which is something they hadn't previously done.

5:36So the way that the things kicked off, these two new scrolls offered by the Italians were scanned at a particle accelerator in England, and those images became the basis for the contest. Now, to win the contest, contestants weren't required to decipher the whole scroll. That would have been far too much. Instead, they had until the end of 2023 to read four passages of at least 140 characters of continuous text. They were only eligible to win if the contest agreed to open source their methodology. There were a number of key inflection points along the way. One came from an Australian mathematician named Casey Hanmer.

6:08As Bloomberg writes, The early months were a slog of splotchy images. Then Casey Hanmer scored a point for humankind by beating computers to the first major breakthrough. Hanmer took a few stabs at writing scroll reading code, but he soon concluded he might have better luck if he just stared at the images for a really long time. Eventually, he began to notice what he and other contestants have come to call crackle, a faint pattern of cracks and lines on the page that resembles what you might see in the mud of a dried-out lakebed. To Hanmer's eyes, the crackle seemed to have the shape of Greek letters and the blobs and strokes that accompany handwritten ink.

6:37Now, when Hanmer published this crackle idea to the Vesuvius Challenge Discord, other contestants like Luke Farreter, a 22-year-old college student who was a summer intern at SpaceX last year, started trying to find crackle in the other images. Now, the important thing here is that while a tiny percentage, just 1-2%, had this sort of recognizable crackle, Contestants like Farreter were able to add those letters to the AI models that they were developing in order to train it to recognize more. Again, as Bloomberg writes, the model starts with something only a human can see, the crackle pattern, then learns to see ink we can't.

7:11A quick message before we get back to the episode today. At this point, you guys know that Notion is one of the major tools that I use day in and day out across the Breakdown Network, the AI Education Beta Project, basically anything that I'm doing in any sort of professional or entrepreneurial endeavor is going to be anchored by Notion. Now, you also know that one of the big themes that I keep talking about for 2024 when it comes to artificial intelligence is the integration of AI into our workflows. I think in many ways, it's not just about which third-party AI tool is best for any given use case, but how they actually fit into what we're doing in ways that are actually time-saving.

7:48And that's why I love that Notion now has AI so deeply integrated across its entire suite of tools, which means that it's everywhere in your entire workspace. Now, for those of you who don't know, Notion combines your notes, documents, and projects into one space that is simple and beautifully designed. It's your one place to connect teams, tools, and knowledge, so you can do your most meaningful work. Unlike other solutions, it doesn't have you bouncing between six different apps. It is seamlessly integrated, infinitely flexible, and incredibly easy to use. Now, with the new, fully integrated Notion AI, you can work faster, write better, think bigger, and take care of tons of tasks that might normally take you minutes or hours in just seconds.

8:25One of my favorite use cases is to use Notion for brainstorming. So for example, what would a great launch strategy be for some project? Use Notion AI to help you think through all of the different dimensions of how you could tell that story. Now the proof is in the pudding and Notion is used by over half of Fortune 500 companies. And most importantly probably for you guys, the teams that do use Notion, send less email, cancel more meetings, save time searching for work, and reduce spending on tools. Right now, you can try Notion for free when you go to notion.com slash AI breakdown. That's all lowercase letters, notion.com slash AI breakdown to try the powerful, easy to use Notion AI today.

9:01And of course, when you use our link, you're supporting the show. One more time, that's notion.com slash AI breakdown. Now, Bloomberg also does a good job of describing how different a pursuit this is than, for example, training an LLM. They write, unlike today's large language AI models, which gobble up data, Luke Ferreter's model was able to get by with crumbs. For each 64 pixel by 64 pixel square of the image, it was merely asking, is there ink here or not? And it helped that the output was known, Greek letters squared along the right angles of the crosshatched papyrus fibers. So in August of last year, when a new image that had lots of crackle was shared on the Vesuvius Challenges Discord, Ferreter connected remotely to his dorm computer, put the image into the machine learning system he had been developing, and when he took out his phone again an hour later, he found three Greek letters.

9:46Pretty soon, he had found 10 letters, including what the classicists involved in the competition identified as the Greek word for purple. Now, this is the point at which I first talked about the contest on this show, and the combination of the crackle discovery from Hanmer plus finding the word purple by Ferreter set off, as Bloomberg puts it, a new wave of enthusiasm among contestants, some of whom began to employ similar techniques. As this was happening, Ferreter teamed up with two other contestants, Yusuf Nader and Julian Schillinger, to better combine their efforts. Now, at the end of last year, the challenge received 18 entries for the grand prize.

10:19The winning entry from the team of Ferreter, Nader, and Schillinger was not just four sets of 140-character contiguous text, but 15 columns of text with entire paragraphs contained within them. While scholars are still translating the text right now, they believe it to be a work by Philodemus, who was an Epicurean philosopher who tended to focus on a very here and now and present sense of the world. For example, the scholars believe that this text is focused on the pleasures of music and food and their effects on the senses. Now, one interesting side note, given that Philodemus the Epicurean could be the portent of a total new unlocking of ancient history.

10:58Back in 1417, a book hunter from Italy named Paggio Bracolini had discovered a poem by Lucretius called On the Nature of Things. The discovery was the subject of a book called The Swerve, How the World Became Modern by Stephen Greenblatt. And Greenblatt basically argues that the discovery of this poem by Lucretius, who was one of the best-known Epicureans, was a huge influence on the sensibility of the Renaissance. As the description of the Swerve writes, It was a beautiful poem about the most dangerous ideas. That the universe functions without the aid of gods. That religious fear is damaging to human life.

11:31That pleasure and virtue are not opposites but intertwined. And that matter is made up of very small particles in eternal motion, randomly colliding and swerving in new directions. Its return to circulation changed the course of history. The poem's vision would shape the thoughts of Galileo and Freud, Darwin and Einstein, and in the hands of Thomas Jefferson, leave its trace on the Declaration of Independence. So for the history geek in me, the parallelism between that discovery, 606 years ago and this one, is just at least a little fascinating. Now, there are lots of challenges from here. There is a modern town built on top of the site where Herculaneum was, there are numerous properties and buildings that are atop this specific villa, and there's no guarantee that there are the tens of thousands of scrolls that many scholars think there might be.

12:13Bloomberg concludes, barring a mass relocation, Friedman is working to refine what he's got. There's plenty left to do. The first contest yielded about 5 % of one scroll. A new set of contestants, he said, might be able to reach 85%. He also wants to fund the creation of more automated systems that can speed the process of scanning and digital smoothing. He's now one of the few living souls who's roamed the villa tunnels, and he says he's also contemplating buying scanners that can be placed right at the villa and used in parallel to scan tons of scrolls per day. Said Friedman, even if there's just one dialogue of Aristotle, or a beautiful lost Homeric poem, or a dispatch from a Roman general about this Jesus Christ guy who's roaming around, all you need is one of those for the whole thing to be more than worth it.

12:49And so, upon the announcement yesterday of the challenge winners, they also released a blog post called The Master Plan, Vesuvius Challenge Stage 2. The two big problems, they say, are segmentation at scale and scanning at scale. Under segmentation, they write, the current bottleneck is tracing the papyrus surface inside the scan of the scroll. We call this segmentation. Currently, we use manual tracing aided by various algorithms. This is quite expensive, about$100 per square centimeter. We spent about$200 ,000 so far to trace enough material to read the 15 partial columns of text that were revealed in 2023.

13:20Basically, that means that with current techniques, it could cost between$1 and$5 million to unroll an entire scroll, which would mean billions of dollars to unroll all 800 scrolls. So in stage 2, they say, we are going to focus on solving auto-segmentation. We believe it's possible to bring the cost of segmenting an entire scroll to$5 ,000 or below. Their 2024 target is to read 90 % of scrolls 1 to 4 and offer a$100 ,000 grand prize for any team that achieves that milestone. The other challenge is the scanning. Right now, between transporting scrolls two at a time from Naples in custom 3D printed cases to scanning them with a particle accelerator in England, the cost for each scroll to be scanned is$40 ,000.

13:57That means it would cost$30 million to scan all 800. This is what Bloomberg was talking about when they said that the plan is to try to see if they can actually scan on site. All in all, their goal for the end of stage 2 to take place in 2024 is to have read at least one entire scroll. Once they've got that technique, stage 3 is to scan all 800 scrolls, which they hope can be done in 2-3 years. And stage 4 is to create enough momentum that there is the political will to excavate the rest of the villa. And so, that is the story of the Vesuvius Challenge from here. like I said a pretty unique moment and a totally different type of use of AI that shows to me just how fundamentally human this technology can be.

14:38Thanks for listening or watching as always until next time. Peace.