Why Deep Networks and Brains Learn Similar Features with Sophia Sanborn - #644

0:07All right, everyone. Welcome to another episode of the TwiML AI podcast. I am, of course, your host, Sam Charrington. And today I'm joined by Sophia Sanborn. Sophia is a postdoctoral scholar in the Department of Electrical and Computer Engineering at the University of California, Santa Barbara. Before we get going, be sure to take a moment to hit that subscribe button wherever you're listening to today's show. Sophia, welcome to the podcast. Thanks so much for having me on. I'm looking forward to digging into our conversation. This will be an interesting one, a bit interdisciplinary, starting from your background, which is interdisciplinary.

0:46But we'll be touching on some of the work you're doing in neuroscience and building neuroscience models using artificial neural networks. But why don't you get us started by sharing a little bit about your background and how you came to work in the field? Yeah. Well, so to make sense of my background, maybe let's start with my motivation, the big picture questions that really drive my research. So I've always been fascinated by the concept of a representation, the phenomenon of representation. Living systems can be differentiated from non-living systems in that living systems represent the external world.

1:27So information is coming in through sensors, it's being transformed, it's being encoded in chemical or electrical information and stored. Somehow all of the relevant world, information of the world out there is encoded in that information. and essentially our brain is transforming photons that hit the retina or you know oscillations of air pressure that hit the cochlea into this electrical activity which somehow has this rich perceptual experience associated with it this rich cognitive experience so that's always really fascinated me and that's been the big picture question that's driven my curiosity since the beginning of my undergraduate research.

2:10So yeah, since the beginning of my undergraduate studies, that's really the big picture question that has been driving me. As a undergraduate who is interested in big questions like this, I was naturally drawn to the philosophy department. Oftentimes, people are a little bit surprised when they know of my research now in geometric deep learning, computational neuroscience. They're a little surprised to learn that my background is philosophy, sometimes people don't realize how deep the connection is between analytical philosophy and the origins of computer science. So a lot of what I learned in my undergraduate education was this long history of people who were thinking about this problem of representation and intelligence and trying to formulate that in terms of formal models, the formal mathematical models that could characterize that behavior.

2:57So as an undergrad, I was really studying this question more from a symbolic perspective. You know, there's the early 20th century thinkers like Bob Shannon, Alan Turing, Kurt Gödel, Bertrand Russell, Gottlieb Frege, people who were trying to come up with these formal systems. And of course, this gave rise to the Turing machine, you know, Shannon's notion of information theory. In my undergrad studies, I was tackling it at this level of symbolic. And this naturally led me to start a PhD in computational cognitive science. But along the way, it started to kind of drive me crazy that I, you know, this symbolic level of abstraction for thinking about intelligence doesn't really give you a handle on physically how this information is represented.

3:40So the way I posed the question at the beginning was really more of a biological question. And this sort of symbolic approach can't give you an answer that goes all the way down to the physics. I started my PhD at Berkeley thinking more from this background of symbolic reasoning. And then I found myself drawn to the Redwood Center for Theoretical Neuroscience, which is a group of engineers, physicists, biologists, and mathematicians who are studying principles of neural computation from the perspective of how information is really encoded in neurons. and that really drew me in and I ended up switching my PhD to study computational neuroscience along the way this was sort of the early days of neural networks and deep learning really catching on so this was 2015 so Alex and I came out in around 2012 and you know what used to be kind of alternative perspective of connectionism started to be the main deal in machine learning and AI.

4:40So naturally, I was simultaneously studying deep learning and really trying to dig into the mathematics of it and understand how we can go beyond linear algebra, which gives us one set of tools for representing information to richer mathematical systems that can represent more structured information like the geometric structure that we have in the world. This is maybe a bit of a digression, but you mentioned analytical philosophy in there. What is that? So in Western philosophy, there's sort of two main traditions. There's analytical philosophy and continental philosophy. And the analytical philosophers were a group of philosophers who were trying to understand meaning from a mathematical and formal sense.

5:23So they were a bit obsessed with this question of if you could write formal statements that are completely true, like with logic. This was the development of a lot of the more advanced formal logic came out of analytical philosophy. And as I mentioned, those developments were happening sort of in parallel with the early computer science, the origins of computer science with, you know, Alan Turing and Claude Shannon. So analytical philosophy is really about trying to come up with a mathematical rigorous way of communicating, of translating ideas, meaning into symbols. The continental philosophers were sort of the older tradition of philosophy that wasn't as concerned with precision in that sense.

6:07And so the way in which ideas in continental philosophy are communicated is a bit more poetic. The analytical philosophers were sort of railing against that and saying, well, I can't even really tell what you're saying. It's too vague. It's too much analogy here. We need to get very precise. So can you maybe give us some concrete examples of papers or research projects and how all these ideas that have formed your background kind of come to the fore in concrete research? Yeah, absolutely. So maybe I'll start with something that I found quite inspiring for the question that originally drove me back when I joined the Redwood Center.

6:44So when I joined the Redwood Center, I learned about the research of my PhD advisor, Bruno Olshausen. Some of the work that he did was on this problem, you know, trying to understand principles of neural representation. There's this old idea in neuroscience, which is that a fundamental principle that organisms need to abide by is the notion of efficiency. So, you know, biological systems are fundamentally resource constrained. Biological systems can't just transmit information arbitrarily. There is a drive towards just sending only what is essential, because every spike that you send in a neuron comes with a metabolic cost.

7:27So there is this idea that brains might be optimized not just to encode information in any arbitrary way. There's so many different ways that you can encode the same amount of information. Think about how we encode it in a computer in terms of 16-bit numbers or whatever versus the way that we're encoding them in physical, biological substrate. There's many different ways to do it. So maybe one way to constrain that is that biology is optimized to be as efficient as possible. And if you look at the way that brains transmit information, the activity of brains is highly sparse, meaning that neurons are mostly off all of the time.

8:04And they send, they spike very infrequently. Of course, that would what you would try to do if you were trying to be optimal in that sense. And so what's really fascinating is that there's this very old finding from the 50s, the neuroscientists Hubel and Weasel, who won a Nobel Prize for their discoveries back in these days. And they found that there are neurons in the primary visual cortex, which look like they're basically feature detectors. This finding is really like the foundation of all of neural networks now, this basic idea that individual neurons in the brain are representing semantically meaningful features.

8:41And those features in the primary visual cortex are looking for oriented edges. So they found, it was actually an accident how they discovered this. They basically were doing these experiments with kittens, kind of brutal, cutting open the heads of kittens, stuffing electrodes in there, and showing these kittens different visual stimuli. And allegedly, they were looking for all kinds of things that would make the neurons fire. They were having a hard time doing it. But at some point, they accidentally moved the slide sheet, just the edge of the piece of paper, past the visual field. And then all of a sudden, they found this neuron that was just lighting up like crazy.

9:19And what they discovered is that there are neurons that were selective for edges that were oriented in a particular direction. And in fact, those neurons are selective not just for a particular orientation, but also for a particular width of edge. And so this was kind of a groundbreaking finding because up until that point, you know, we were really having a hard time getting a handle on what, you know, how neurons were encoding information. And this was a kind of nice story is that individual neurons are looking for different features. So you can imagine building up now a hierarchy of visual representation.

9:52this is exactly what deep convolutional neural networks do, starting from edges and going to more complicated features like curves and then circles and all the way at the top of the hierarchy of something like a cat neuron or a face neuron. And as a very crude approximation, that's sort of what visual neuroscientists over the next 50 years discovered is happening in the brain, roughly, of course, with many caveats. But anyway, so those neurons, these oriented edge detectors, That's sort of a canonical finding from neuroscience. Now, one explanation for what they're doing is that those neurons are looking for edges.

10:27And, you know, you can kind of come up with a reason why that makes sense. Sure, the world is composed of things that have edges. That's not really a principled explanation. It doesn't really help us come up with predictions for what other kinds of features we might expect neurons to be looking for. Bruno Olshausen, my PhD advisor, he did this work in the 90s, early 2000s, showing that if you use that idea of efficiency, of efficient coding, this fundamental resource limitation as the guiding principle to constrain representation learning, then those features naturally emerge. these oriented edge detectors naturally emerge.

11:06So basically what he did was he took a very simple neural network, artificial neural network, just a single one layer, you know, linear transformation followed by a non-linearity. And he trained the network to encode sort of natural images, things, you know, things you would see nature of trees and whatever, just a pretty diverse set of stimuli. If you just try to encode that information into a compressed representation where your neurons are spiking infrequently. So it's sparse. And then you're just trying to reconstruct the input. So basically preserving the information, but encoding it in a sparse way.

11:40So doing a transformation that preserves only what is essential. Then you learn exactly those features. We have ways of sort of visualizing what individual neurons in the brain are selective for. We call it the receptive field. So if you visualize that, and then you look at what falls out of this algorithm, they're pretty much identical. It's really striking. That is striking. It seems like there could be many different ways that... It sounds like a dimensionality reduction, and it seems like there could be many different representations, and that the representation that earlier research has found is what's happening in the brain is what comes out of this algorithm is pretty interesting.

12:21Precisely. No, for me, I found that absolutely fascinating, because why would it be exactly the same. With the relatively loose constraint of essentially solving, just as you put it, this dimensionality reduction problem, we discover the same solution that the brain discovered through evolution. That means that there's something that's really fundamental about the constraint we've put on the system. Potentially the constraint we've put on that system is the same one that's guided the evolution of the brain. But just as you said, you would think that there's more than one solution to that problem.

12:51So that's quite mysterious. And that's a big theme that's my work is this notion of universality, because this has been happening sort of over and over in both understanding neural representations and then also understanding how deep networks work, is that consistently the same kinds of features show up in both of these systems. You know, there's ways of doing like mechanistic interpretability research on deep neural networks. You can take individual neurons in the deep network and we basically do neuroscience on it. But now we have the gradients, so we can do backprop all the way to the input and say, what's the input that's going to maximally stimulate this neuron?

13:27That's exactly like the artificial version of the neuroscience experiment with the kittens. We say, what's the stimulus that I can create that's going to maximize the activity of this neuron? And when you do that for neurons in the deep network, these features show up that are strikingly consistent across different deep networks that are actually trained for different tasks. So if you do a lot of different convolutional neural networks, let's say that are trained, you know, one's trained for classification, you know, one's maybe a generative model, they're trained on different data sets, they have different sizes, the same features keep showing up.

13:59And then they keep, you know, there's this continued consistency with the kinds of things we see in biology. So that means that it's far from random, and that there are these principles that are guiding this whole process. And what's more is that these features are highly mathematically structured. So in particular, the edge detectors that I told you about. These are actually the way that people started modeling them mathematically in the 80s was using Fourier analysis. What they realized is that they aren't just edges. So you think about how you sort of model an edge mathematically. It's sort of like black on one side and white on the other, right?

14:37You're looking for something that's positive, you know, high stimulation and then low stimulation. There's this change in contrast, but it's not as simple as that with these edge detectors. What researchers found out is that the behavior of these neurons can be very accurately modeled by a wavelet or a Gabor, you know, a Gabor function. What that is, is it's basically, you take a two-dimensional Fourier plane wave. So that kind of looks like a sine wave in 2D, kind of like the curtains that are behind you. So you take a 2D Fourier basis, and then you localize it by putting a Gaussian window around it.

15:13And there's basically this, you know, this oscillatory dynamics. So it's kind of goes up and then it goes down. And so if you were to shift something left to right over a localized plane wave like that, the activity of the neuron, its response is sinusoidal. So it kind of goes up and then it goes down and goes back. That's exactly how these neurons behave. So mathematically, they seem to be using something like this Fourier analysis. Similarly, the features that are showing up in these deep networks, we get these beautiful Fourier-based features, Gabor features, and other things that have very striking mathematical structure and symmetries.

15:50And now things like Fourier were not developed at all in the context of biology, if memory serves. They were developed in the context of like radar, well, wavelets in particular, I think was radar systems, maybe like World War II timeframe or something like that, if I'm remembering this correctly. I could be making this all up? I'm not sure about the history of wavelets in particular, but Fourier analysis was developed by Joseph Fourier, mostly in the context of thinking about physics and heat, so thermodynamics. And there's actually a very deep reason why all of this, the same math can be used to model physics and to model visual information.

16:27And we can get into that in a little bit. I know that wavelets have a long history, but they've been used in computer vision for quite some time. They were not developed in the context of neuroscience, but then neuroscientists realized that exactly that math described what neurons in the brain were doing. I could be hallucinating like a LLM. I remember something. I studied a bit of DSP and wavelets in grad school, and I thought I remembered something about radar systems as maybe it was one of the uses as opposed to the impetus for it. But in any case, I'm sure you're right about that. These were models that weren't created in the context of biology.

17:08They just happened to work really well once applied to biology. Yeah, exactly. And probably why that is, is because a lot of the problems, you know, engineers were developing wavelets for doing signal processing. And a lot of the problems that they were trying to solve and that wavelets seemed to be a solution for. Well, those are actually similar problems to the problem that biology is trying to solve. Biology is also doing signal processing. It's taking in information, trying to figure out what an optimal transform is to put it into a good representational space. And so you can think of it sort of as convergent evolution.

17:42Engineers designed wavelengths in biology, also designed wavelengths in a parallel way. Yeah. That was kind of some of your advisor's work or colleagues' work in context to some of your research as well. You were getting to your research in the field. Yeah, exactly. So that was for me, a very inspiring example of an approach towards deriving what we observe as emergent properties in these systems, biology, artificial networks, from greater principles, which actually give us a handle on what the system is trying to do. Now, when I learned about these results, the thing that was striking to me is that Bruno and a lot of the other people who are doing this work were thinking about it in terms of compression, efficient coding, dimensionality reduction.

18:35But actually, there's this deeper mathematics that's underlying those same properties. So the Fourier transform, a lot of people learn about the Fourier transform in the context of engineering as a tool for signal processing. We learn it's a way to transform something like audio into its component frequencies, and we can find what the coefficient is on each of the sine waves. So we think about it more from an engineering perspective as a tool. Fourier analysis actually has its really deep roots in the mathematics of group theory. Group theory is an area of abstract algebra, which is all about modeling transformation structure.

19:14So a lot of the kinds of transformations that we observe in the world, such as two-dimensional rotation, three-dimensional rotation, translation, scaling, stretching, squishing, all of these things, mathematically, these are describable as what are called groups. groups are essentially just sets of objects and some composition and then there's these constraints on the way that those objects can be composed that's a very abstract definition but yeah so a group is essentially a set of transformations so a good example is like the set the group which is called so3 the special orthogonal group of dimension three which can be represented with 3D rotation matrices.

19:57So a group is just the set of all possible rotation matrices, and those matrices can be composed with matrix multiplication. And then for something to be a group, it has to adhere to certain axioms. So the operation has to be associative. There has to be an identity element, which is of course the identity matrix, which basically is the do-nothing operation. If I apply that, nothing changes. There needs to be an inverse for every element of the group. So every rotation matrix has to have one that undoes it, and the group needs to be closed. So that means that if I were to compose any two elements, I would get another element from this set.

20:33So with rotations, you can combine one rotation and another, and you're always going to get a valid rotation. So groups are essentially the mathematics to describe much of the kinds of transformations that we observe in nature. Groups were used heavily in early 20th century physics. They provided a way of unifying the models of the fundamental forces and particles, and especially the mathematics of Li groups, which are infinite groups, continuous groups, such as rotation. So interestingly, the Fourier transform is related to group theory because essentially what we're doing when we take a Fourier transform is we are, I'm going to use some technical language here, but I can unpack it, we're projecting a signal onto what are called the irreducible representations of the group of translation.

21:22So now I use this term irreducible representations. A group is inherently naturally an abstract thing, but it can be made concrete by mapping it into linear algebra. And that's what a representation is. So a representation is essentially a mapping from an abstract group to matrices. Like I just said, we can make the notion of 3D rotation concrete by representing it with sets 3D matrices, 3D rotation matrices. So the irreducible representations are just sort of a sort of special irreducible set of those representations. There's many different ways to represent a group. They're sort of a very fundamental signature of an object.

22:02And the Fourier transform is using the representations of translation. So there's a well-known property of the Fourier transform, which is that it is was called equivariant to shift. It's the Fourier shift theorem is sort of a very fundamental idea. If you were to take your signal and then take its Fourier transform, sorry, if you were to shift your signal and then take its Fourier transform, that's equivalent to taking the Fourier transform and then shifting it in a certain sense. This is this notion of equivariance. Have you come across this idea of equivariance? It's been coming up recently, especially in geometric deep learning and extensions of convolutional neural networks.

22:40Yeah. It's based on exactly that mathematics. So what was interesting to me is this suggests potentially an alternative explanation for what these neurons in the visual cortex are trying to do. You could say, okay, they're compressing it, and that's why we end up seeing these wavelets. The wavelets provide a good basis for compression. And or alternatively, you could say, actually, maybe these neurons are trying to model the transformation structure. So because this particular feature is equivariant to translation, it's basically converting transformation out in the world, which is sort of if you just get it in terms of raw pixel intensity, it's in a form that's difficult to model because it's kind of dispersed all over the retina.

23:23Now, like your whole image is getting translated left to right. But once you take like essentially a Fourier transformer or a localized version of that, now you've converted the translation into this continuous change on the activity of a single neuron in a way that makes it really easy to model and generalize that information. So I got really interested in understanding basically an alternative explanation for those features in terms of groups. and that led me to this recent paper of mine which was published at ICLR this year called Biospectral Neural Networks in which we basically you can think of it as doing what Bruno was doing in the sparse coding work but for the idea of trying to capture transformation structure so there he was saying okay if we use efficiency as the fundamental principle and we try to learn features from the data, then these things emerge.

24:20Well, we said, let's use invariance as the fundamental principle, because this is a very core problem that the brain needs to solve is to form invariant representations. So we need to be able to model that transformation structure that's out there in the world, and then also remove it so that we can recognize objects when they're in all of these different poses and things like that. So in this paper, we construct an artificial neural network with a particular structure. And there's some sort of, we'll put a tag on that in terms of what that is precisely. But we use this downstream objective of invariance to try to learn interesting representations.

24:55And what's really fascinating is that what falls out of that problem. So we're basically, by the way, the invariance objective is, you can really think of a contrast of learning, which is very popular, a sort of learning scheme for deep networks. Contrast of learning is essentially doing an invariance objective. So it's saying, take all of these things which we're going to say are the same we don't really know how they're the same but we're going to consider them to be the same we want to map those to a some to the same point in output space and we want that point to be different from this other class of things which are the same yeah so that's the contrast of framing so we're basically using that kind of setup but we're doing it in this really simple very controlled network and we want to see under these constraints do we end up learning these fundamental objects from group theory that we predict that you should learn because they essentially allow the system to make explicit the transformation structure and then remove it in a way that's precise so that you can be invariant.

25:55And what we find is that indeed you do. What exactly are you applying that invariance constraint to the output of some network? Exactly the output of the network. So just as you would do for contrastive learning. So let's say I have some big data set of things. I pass it through the network. I get some transformation. I just want the output of my network to be invariant to the, you know, when you do it in contrast to learning, it's basically to the class label. So you want everything that's within the same class to be equivalent. Here, what we're doing is we're taking images that have been transformed according to some kind of group transformation, like rotation or translation.

26:32And we say, we want those to be going to the same point. Got it. Okay. For translation, 2D translation of images, which are just sort of shifting like this, we end up learning exactly the 2D Fourier basis precisely, perfectly, like canonically, every single 2D Fourier basis function. It's perfect. It's so perfect that we can actually recover the structure of the group from it. Because there's this correspondence between representations, irreducible representations of a group and group elements themselves, we can use the learned weights to actually go back and read out what's called the group's Cayley table, which is the fundamental structure of the group.