How Sougwen Chung teaches robots to pause

0:00I've been converting my brainwave signals into a genomic sequence. One way in which I'm not interested in being machine readable is for me, the work that I'm doing is about my own learning in a way. So I don't want to be machine executable. Why would I want to build a system that takes away the thing I love the most from me? And I can say to this day, I draw more than I ever have. For me, it's about extended authorship. It's much more Haraway than Hinton. In the third century, a Chinese court official named Zhang Yao invented a technique that would define calligraphy for the next 2 ,000 years.

0:42He called it duen, the pause, a deliberate moment of pressure where the brush presses down, holds, and then lifts or redirects. It was a way of encoding intention into the stroke itself, not just where the mark goes, but the weight of the decision before it moves. No original works by Zhang Yao survive. What survived was the technique itself, passed from hand to hand for 18 centuries. The pause became a kind of inherited intelligence, embodied, transmitted, alive. Su Wencheng recently discovered they are one of Zhang Yao's direct descendants. And in recursions, a 10-meter scroll performed live at Art Basel, Hong Kong, with two robotic arms guided by EEG data, the pause returns in a new form.

1:45Not in the wrist, but in the brainwave threshold that tells the robot when to act. Su Wen has been building toward this for more than a decade. Not in theory, in ink. Raised between Toronto and Hong Kong, trained at the MIT Media Lab, and an artist-in-residence at Bell Labs and Google, Chung has argued for something more demanding than AI art. That the machine can be a collaborator. That error can be generative. that technology can reveal as much about the human as it does about itself. They believed that to extend agency, we must understand the human sensorium and how technology changes what humans experience as we move through the world.

2:34In 2022, the Victorian Albert Museum acquired memory, making it the first AI model collected by a major cultural institution. In 2023, Time Neng Chung, one of the hundred most influential people in AI. Today's conversation is less about whether machines can make art than about what artists can teach us about living with intelligent systems. Su-En Chung is an artist, researcher, and founder of Solicit. Welcome, Su-En. So I've been looking forward to this since getting exposed to your art when we were in Solomeo at the Cuccinelli event. What a beautiful event. Yes. And so thank you for joining us on Possible.

3:21My pleasure. Your father was an opera singer and your mother a computer programmer. Yeah. When did you realize that it wasn't choosing one of the inheritance, maybe the kind of parenting, but blending this together? I love that question because I think it's really hard to choose between your parents. And I think early on, we all realized it's probably best not to. So in that way, we were brought up with both languages in our household. And I think when you learn something from early childhood development and from those experiences, they become really part of the fabric of how you think about things, how you cognate, how you express.

4:01so I don't think there was ever really a moment in which I chose one side or the other. I grew up playing violin and piano but also learning how to code. We had computers from a young age so it always felt really natural. At the same time I always thought that maybe through that and maybe because I'm kind of a quiet person in general I always thought that there's something that art and performance that those mediums could express that language couldn't So I just very much veered towards those directions. Out of curiosity, because this is one of the things that affected me when I was young. Did you ever read Gerdel Escher Bach?

4:41I'm going to be quite honest with you. I tried. Okay, fair enough. It's like, brah! Yeah, it's quite... I weaponized it in different ways. I tried, but I'd love for you to tell me about it. Well, so Hofstadter, genius book, Pulitzer Prize, things he's writing about human existence and consciousness and whatnot, which is part of where they kind of art and AI and all the places we're going. But it was also the parallels between, for example, you know, Gerdel, Escher, and Bach. So you play music, you're, you know, do visual art, you think in computer science. So the same kind of parallels between these systems, like how you think the relationship between music and computer programming is kind of a classic.

5:26And so that was the reason occurred to me that maybe you'd encountered it. You know, it's so funny. It's been so much ingrained in my experience. I consider myself an artist and researcher, but I just learn by doing. So I think it's been playing violin at such a young age and piano and learning through instruments, but also understanding that there's a really interesting world of digitization around sound and visual that happens also from a really young age, I think those parallels and that translation has really been a point of inspiration, of tension, and of curiosity for the work because they're readable in many different forms.

6:09No, exactly. Well, we will get to various forms of readability. But it seems that your art practice and what you're building started with the question of what happens when a robot draws alongside you. And so what did you expect? And then what did you discover? That's such a great question. It started such a long time ago. It's funny to reflect on something from 10 years ago, 11 years ago. I was thinking through this moment in 2015, particularly when I was reading an article about Lisa Dole and AlphaGo. and Lee Sedol mentioned in his defeat to AlphaGo in this sort of a landmark competition, he mentioned that even in defeat, he was incredibly inspired because he saw the beauty of the non-human move.

7:03And that was catalytic for me. That was really, really inspiring because the context was he had been defeated by a computer system, which we didn't think that was possible because he was the Mozart of the game, which we all aspire to. For a game that's been played since a time immemorial, it seems. There's a few aspects of that that I found really interesting. One, as an engineer, how do we unpack this idea? How do we quantify the beauty of the non-human move? Is that even a sensical direction to take artistic or technical development? Secondly, I thought there was something really, as I'm older and I reflect on these different waves of AI and its conceptualization and culture, you're like, he's talking about seeing beauty after experiencing defeat.

7:53This sort of fatalistic to awe trajectory that was so incisive and really exciting to me. So at that point in time, I was thinking about being a drawer, being a performer and drawing through the interface of the computer and the screen and like the mouse and all these things and feeling like the sense of a lack of embodiment to that translation of my mark into, you know, something that happens on screen. So I wanted to think about bringing it back into physical space, what that would mean. Like, I think it comes from playing violin at a young age and just understanding this expressive gesture.

8:34I wonder if it comes from my interest in calligraphy as well, this extension of the human subject through the line. So yeah, that really was the kernel from Lisa Dole. And I'm not a very good Go player, YK player at all, but I think that kernel really made me think that I wanted to explore what the beauty of the non-human move was for me. And I could achieve that through learning how to build robots and taking the one thing I knew, drawing, and seeing what would come from it. So my area of expertise. Most people's first impression when they encounter it is like, oh, what's important is the robot drawing.

9:18Yeah, yeah, yeah. Right. Entirely. Right. But what actually is important is the dynamic and what's happening with the human. Yeah. So what's been that kind of lesson of the dynamic by which the non-human move is also incorporated into the human move and the dance that is your art? Yeah, I think there's so many things I've learned from engaging in this process durationally. Ten years is not a short amount of time. We've gone through seven generations of robotic development and artistic development, I would say, where we address different themes and work with different sensors and different algorithms to bring a different type of gesture into the form.

10:00That being said, I think I've learned that there's a lot of beauty in the error state and there's a lot of truth in the uncertainty of the mark making. Again, sometimes people say that the work feels really sci-fi. But I actually always respond with and really believe that there's nothing fictional or speculative about any of the work that I do. It might feel that way, but I'm really trying to get at the materiality of these systems, at the materiality of the art form of drawing and of performance. So I really think there's something more true, not more fictitious about this idea of drawing and not really understanding what the system is outputting.

10:50There's something very, very interesting about the questions that also arise from that, which is what is machine readable, whereas what isn't? What does it mean to draw with one's own historical archives? All these interesting lessons that I've gathered. And what's been part of the journey for you? Because part of it, like one of the things that I think makes art really interesting is that personal quality. Yeah. And so what is the thing, like what have you learned on the journey? What have I learned? I've learned quite a lot. I know people like to think about optimism or pessimism. But one thing I learned, maybe not about myself, but about art in general, is that by engaging with this practice and by engaging with building these systems and understanding oneself as data and measuring all these things and trying to make something maybe beautiful or true or gestural from it.

11:47It's been a way for me to hold fear and hope in the mind at the same time, because there's a lot of speculation around, there's a lot of hype and a lot of dystopia around this idea of the human machine. It's been in Hollywood, it's been in contemporary media, it's been all over. Every major platform is talking, they have some stance of pure optimism or pure dystopia, it seems. But I think for me, what I've learned is that there's ways to chart out what that actually means for you and how you live with it in your life and how you integrate or not integrate the things that you really care about into these contemporary systems and models.

12:33I love that answer. And I think the blend of thinking about, you know, to use kind of There's a parallel to the yin and yang. Yeah. And this is what brings the human, in honor of your work, in the line between. Yes, there you go. Yeah, I like that. The contour is the line between. Exactly. So one of my super weaknesses is pronunciation. So I'm probably going to get this word wrong. But the Duen pause technique? Yeah, the Duen pause technique. That your ancestor, again, Zhong Yao. Yeah, yeah, yeah. That's great. invented in 230 CE, survived only because practitioners, right? Because not necessarily the work.

13:17So your recursion system carries your gestural archive forward through a neural network. So what does it mean that, like, what another line between those, another union is like, there's one of the oldest techniques with now one of the newest. How would you kind of draw those lines. There's so much mystery for me in how we connect to our past, how we, in the words of Audrey Tang, might be good ancestors, which I still find very incendiary. So what I'm interested in in the recursions system and what I recently showed and performed at Art Basel Hong Kong and what I've been thinking through is how do we make decisions within a system?

14:05What direction or proposition that is non-cognitive, can I give the system that will allow it to draw or stop, right? Which is very similar to the pause technique performed by and originated by my ancestor. And those developments have not been maybe developed in parallel because we occupy very different, we live in very different times to say it mildly. But when I realized that this was my ancestry and my genealogy. And I realized that I've been really developing something that's quite related without that knowledge of his invention. I felt like there was a kind of recursive temporal inheritance happening because my interest in calligraphy doesn't come from an interest in or even the knowledge that this was my ancestor or my interest in drawing was very almost innate from when I was very young.

15:04But when I learned about this overlap, I'm excited and kind of nervous to talk about it because it felt so apt, it felt so layered and interesting. I've been working with my own archival data and bringing up my parentage when I think about my practice. And it was exciting to relate something as ancient and as substantive as calligraphy to the marks that I'm making with my performative gestures. That's awesome. And one of the things that you said recently is to become a machine collaborator, you've had to become machine readable. So I want to come back to this because you've gestured a couple times, the readability, it feels much more of a gesture at modern life, right, through art.

15:55So say a little bit about how you've had to make yourself machine-readable. What's been the kind of learnings from that? And then kind of how does that go to art and then to kind of part of kind the gesture of the human experience. Absolutely. So when I do a little bit of mentorship for different artists, I always talk about there's so many tools, there's so many opportunities, there's so many possibilities with how we work or don't work with technology today. And I found what's the meaning-making engine for my own work? It's drawing. It's most simple. So when I, in 2015, decided to train a return neural network on two decades of my drawing data, I was like, this is the thing I care about.

16:45This is the thing I know. This is the thing I understand in my bones. What does it mean to digitize that work from the level of volume? What is the labor involved? What are the challenges? For instance, in that very specific example, at that point, You couldn't get, maybe that's why I'm obsessed with time, but you couldn't get the pauses of the gesture. You would only be able to read the drawing as a sort of finished image that you need to reverse engineer. So a lot of the decision-making pathways of the data are not available because it's just not. So I developed from then a tool that allowed me to trace the pauses, to trace the time, to put the time back in the data of drawing.

17:31And in that way, it's been really rewarding for me to think through the nuts and bolts of what makes art or what makes art not machine readable and what parts are readable and legible to the system and what parts aren't. And what parts do we want to be legible to the system? And what parts don't we want to be legible? So that's been really educational. It's been really grounded because I think a lot of the times, I think fear comes from this sense of a lack of specificity because you're just afraid of all of it, especially when it comes to machines replacing artists and that automation cycle. So in that way, it's given me a real sense of grounding and also this idea that I can train my own muscle memory by working, by putting what is machine readable in my drawing back towards the space of the canvas as a collaborator.

18:29So you make yourself very machine readable. Is there anything you want to not make machine readable? and kind of where is that kind of, I don't know if it's negative space, but alternative or human space, anything in that thread? Yeah. So I think oftentimes, I think we'll get to some issues of like human presence versus not later on too. But one way in which I'm not interested in being machine readable is for me, the work that I'm doing is about my own learning in a way. So I don't want to be machine executable, maybe is the right word. So I think there's a lot of artists and projects that extend personal data into a dry machine.

19:18We've seen that quite a lot. There's a lot of art historical precedents for it. And sometimes people think that's what I'm interested in doing. But for me, it's really about creating the conditions of relation and the feedback loop between human and machine. So I'm never creating or designing or authoring myself out of the equation. And that's been my North Star because why would I want to build a system that takes away the thing I love the most from me? And I can say to this day, I draw more than I ever have. And I also have worked with these systems for maybe a little bit longer than a lot of people.

20:03So I think there's a world where both can exist. Well, 2015 is definitely well before most people had realized an AI revolution was coming. Yeah, people I think thought I was, they were a little bit concerned when I was making this project because it wasn't in the dialogue at the time. It was a very different, it was like real-time graphics, maybe starting to be a little bit of gaming interest. But that feedback loop has been at the center of what I'm curious about for a long time. Well, the other aspect of this, you know, kind of putting the timing back in is that LLMs, neural networks, are actually pretty bad at concepts of time.

20:43Yeah, that's so true. Yes. And so say a little bit about the temporal experience, because that's actually one of the places where, again, where you kind of say, well, what is quintessentially human in the place between and the expression and the dance. But say a little bit about that temporal element and what you've learned on that. When I brought this idea of time and that pause in was when I started working with EEG. And that was during the pandemic because I did not want to draw anymore. It was quite, you know, it was a dark time for everyone, I think. I think a lot of creative people and a lot of artists and just people in general felt quite unmoored, to say it mildly.

21:23So I really went to meditation as not a creative outlet, but just an existential outlet maybe. And I wanted to train a way to enhance and deepen my sense of the alpha state threshold, which is the state that you're in when you are in deep meditation. So I wanted to create a configuration that allowed me to do that. So I actually developed the system, the spectral system, paints only when I'm at that alpha state threshold, and it doesn't when I'm not. Which maybe for the tradition of meditation, maybe that's somewhat blasphemous. But I found it was a nice way of enacting a non-conscious type of control that wasn't gestural.

22:12And control is not really the right word here, but maybe understanding, proprioception. and I use that to make a pause meaningful and sort of sensible and I found, I don't know how it is for you, but I think creativity happens in the silence or imagination and real reflection happens in the quiet and the pauses and I'm not surprised sometimes that some of these LLMs don't really understand that even in that feedback loop between human and system There needs to be, I was joking with a friend of mine today about, you know, wouldn't it be nice to just sit in silence? There's so much that can happen in those pregnant pauses that I really wanted to bring that back into the data set and back into the space of drawing.

23:03So what have you had to quantify about yourself? And then what have you, if anything, have you deliberately kind of held back to protect yourself in your work? I try to restrict myself to movement, whether that's my own electrical signal movement or my gestural movement. I do so because it's this constraint that creates a lot of adaptations that I find really interesting. I like that constraint because I build my world through movement and line. And I also, I'm interested in writing and speech and image, but that's a very different thing. I love the focus of seeing what I'm made of in movement.

23:47And is there anything that you've begun thinking about since you're both a program thinker and also an experienced thinker where a language of that movement? I'm starting to think about what that might look like, actually. But it almost feels like notation plus. You know, I show the work and the data set as its own art piece. I've been converting my brainwave signals into FASTA format, like a genomic sequence. So I'm really starting to play with what that encoding looks like. I think it's super interesting. But I just know that it's maybe I have too much respect for language and maybe poetry, but I like keeping almost text.

24:34I love that inscription version of thinking about text versus a more verbal version of it. Yeah, no, I look forward to your extending that line and drawing a line. So you have this drawing operations unit generation series, you know, robotic systems trained on your gestural archive, which we've been talking about, built to draw with you, then simply execute a command. I don't know if you refer to it as Doug deliberately. Is it a personification? Is it like what's with how the name and then how the name becomes also part of that kind of non-human gesture along with the human gesture? You know, that has really changed a lot through time.

25:19And I miss those playful days sometimes because I think in the beginning I've been thinking about anthropomorphization. I was thinking about that quite a lot in the early days because I didn't know so much about how anything really worked. I have more of a computer-sized background, not a robotics background at the time. So a lot of these kinematic levers I didn't really understand yet, I just projected a lot onto the system. And drawing operations unit generation one, Doug one sounds a little bit like Su Gwyn. And it felt very cheeky. It kind of felt playful. It felt weird. It brought a lot of strange dimension to the first experiment that I think was such a gift looking back because I think the work can be really serious, but there is a playfulness to it that sometimes we don't get to have in our everyday.

26:12You know, we talk about these big topics about humanity and machines and the future of work. And there was a playfulness in the very beginning that I really still enjoyed. Maybe a playful ignorance because I did anthropomorphize quite a lot and everyone joined in. We had gallery curators and museum staff being like, how's Doug? And I still love that, actually. But it's given another entry point to robotic development, to this dialogue that I still really cherish. Is there anything in this play, because you're probably familiar, there's multiple theories of it's not homo sapiens, it's like homo ludens, like we're playing.

26:56Is there anything in that play that has illustrated for you both some aspects of human humanity and some access of, as it were, machine readability and interaction? Sure, sure. I think this particular maybe projection has taught me a lot more about art than developing robots, actually, because I've talked about how these systems are us in another form. And I don't – how do I phrase this? I think so much of that early conception of Doug as a sort of discrete unit was very much my own projection. And that's what people bring to the work as well. And I feel like that's a deeply human trait. I think it's one that comes from empathy.

27:48It is one that comes from imagination. And so much and so much of the work, it's like this machinic other that's of my design and authorship and creation that I think can be a really beautiful thing. I think that's what art is. You know, that's what artists bring to their mediums. And the same can be said for robotics as well. And so one of the beautiful, I think, expressions and another, maybe I'm overusing the gesture metaphor, but I love it. I'm here for it. Yes. But one of the things is the earliest Doug introduced some deviations. and your response was to say, no, this is poeticizing error.

28:35Say a little bit more about that. And that phrase of poeticizing error is something that I think I may plagiarize. Yeah, please do. Please do. You heard it here first. No, I really do think it was such a gift, that first generation, because I don't think I really knew what I was doing when I first started performing this interaction, this drawing with Doug in 2015 in a dark basement with a quite janky robot, actually. And when I think about poeticizing the error, it was a gift that came from the fact that I performed it live as one of its first iterations. And it was a gift that came from the audience.

29:19Because in tech development, you don't poeticize error. It's always a bug. It's not a feature. But in that performative moment, I knew I wanted to explore something very real and gestural and responsive. But when things didn't work according to the simulation that I had sort of on the side, people really responded to that moment, that error, with a lot of empathy, with a lot of delight. and that was a really strange spectacle at the moment in the moment as a performer because we were all kind of hallucinating through this sort of machinic other in a way but as a visual artist and as a developer I realized that if there wasn't error in that interaction it would be really uninteresting because I would have nothing to adapt to and And it would just be the first generation was a mirroring of my own gestures.

30:21So without that error distortion, it was sort of like an artistic style of the robot at the time. It made it quite dynamic. And that really gave me a lot of permission to develop more ambitiously and widely. Because I'm like, if something that's barely working can spark all these creative ideas for myself and for an audience, then there's really something here that's really potent. And how, if you've given any thought to this, would you help other people generalize using art as a lead, whether it's other human interaction with chatbots, maybe with hallucinations, labs is what they're trying to do in alignment training.

31:02How would you say, hey, echoing forward this art, this gesture of poeticizing error, here's a good way for people to think about how to approach their own interactions with AI? I think a lot of that comes from a little bit building from the ground up. There's a lot of optimization that can happen when you use more of these maybe proprietary and off-the-shelf tools. But I think when you really think from the ground up, you can think about more sustainable materials for different types of sensory development. You can think about low cost, low compute models. And then you can think really have a stronger dialogue with the artistic intent, which is always going to, you know, I've been saying lately that artists are great at breaking things.