The science of sound and music

0:16Hello and welcome to the Naked Scientist podcast, the show that brings you the biggest breakthroughs and talks of the major movers and shakers in the worlds of science, technology and medicine. I'm Rachel Ralph and today we examine the role of science in sound and music. Are beats and Brahms all just physics? And when do babies start to boogie?

0:44Music is one of the most universal parts of life. It makes us happy and it makes us weep. It gets us on the dance floor, connects us to other humans across the globe and can even influence our development when we are still in the womb. We are going to explore the origins of music and how it has evolved into something we know and love today. But to begin, we are going to find out how sounds are made and the role that science plays in its production. I went to Trinity College, Cambridge to meet a man with a variety of instruments to find out.

1:20Hi, I'm Hugh Hunt. I'm a professor of engineering at the University of Cambridge. So the first question, what is sound? Well, sound is quite interesting because air is not a solid, it's a fluid. And that means that waves can move through the air just like waves can move on the surface of a pond. Well, we can see those waves, but we can't see the waves in air. You can imagine that the waves that propagate through the air, they're called compressive waves because air gets compressed and then it decompresses and it's kind of like imagine you've got a line of people and you thump the person in front of you and then they thump the next person and then they thump the next person that's kind of like a wave that propagates through the line of people but then the speed that that happens if if you thump people really quickly and the wave of thumps going through the line of people happens more quickly then that's a higher frequency that's a higher pitch.

2:22So how does frequency affect the sounds we hear? So frequency is the number of vibrations in a second so middle c on a piano is 256 roughly cycles per second. Now I can play that I've got my laptop here and I can play that middle c which is this one here now I can then play an octave above that so in other words I can play at double the frequency you might say it's the same note but it's double the frequency I can double the frequency again and I'll still get that note but then I can pick up frequencies in between so for instance this is not double but it's three times and you think oh three times the frequency and then i can do five times the frequency so as you change the frequencies you get different notes what is frequency it's just how fast the sound waves in the air are changing i know as we get older the range of frequencies that we can hear decreases so what's the science behind that well so my hearing is definitely getting worse and i can't really hear very much above about two kilohertz but you almost certainly can and we can do a little test so let me try this is um three octaves above middle c okay now you can hear that right now i'm going to go four octaves above middle c i can't hear that okay now let's go one more i don't know whether your microphone will pick this up let's go one more can you hear that there's no way i can hear but that is a frequency of 8 000 hertz eight kilohertz you probably can't hear much higher than that a typical high quality audio recording will stop at about 20 kilohertz so we reckon that humans aren't interested in sounds above about maybe 10 15 kilohertz so talking about audio recordings let's move on to instruments and how different types of instruments produce sound so how would a string instrument produce sound compared to a woodwind instrument for example you can either produce sound directly so for instance if i move the air directly so i can whistle and that is air coming out of my lips and it's it's vibrating and it produces sound directly or you can take a stringed instrument now you've got a guitar there I fancy that you happen to have brought a guitar so you can play on the string any string okay and a different different note okay so that's two octaves above but now the reason that we can hear it the string is very thin and it doesn't move the air very well but the guitar body, if I knock on the guitar body, it's very reverberant.

5:57So when you play a note on the guitar, the string pushes on the bridge, which is attached to the body of the guitar. And it's not the string that's vibrating, it's the body of the guitar which is vibrating. And then it's the body of the guitar that moves the air. The same is true with a violin or a cello or piano. But for wind instruments, say for instance the clarinet or the oboe, it's got a reed and the reed vibrates backwards and forwards and then if you just had the reed by itself, it would be a bit squeaky. But then if you then take a tube which picks out the frequencies nicely, you can get really lovely pure tones.

6:47So clarinets and oboes do that. Now if you take an instrument like a trumpet or a French horn I happen to have a horn here. Now if I take the mouthpiece off the French horn this mouthpiece is only a few centimetres long and if I start to blow into it not a very nice sound. but if I put that onto the horn and the horn has now got a long length of tubing and the tubing has its own harmonics so I can get the different harmonics and depending on the shape of the instrument you can get different types of harmonics So a trombone will sound a bit different from a trumpet, which will sound different from a tuba.

7:46And they're all just very subtly different. Speaking about the clarinet, so my best friend at school, she played the clarinet and she would play it really well. But then when one of us would try to play it, we would just squeak so, so much. Well, I think learning to play an instrument is a real skill because a lot of instruments just don't work unless you know how to do it properly. The clarinet's a pretty difficult instrument to play, so you can't just pick up a clarinet and hope that you can play it. For most people, trying to get your lips and your teeth and everything in the right place, I don't blame you that you found it difficult.

8:27And now we know how sound is produced by different instruments, how does that sound travel from the instrument or a singer, for example, to the listener's ear? you've probably been out in a field in wide open space and you shout out and you feel as if your voice isn't very loud and maybe somebody on the other side of the field can't hear you at all and that's because you're not getting any reflections off any walls or anything the opposite is possibly true you might have been in a small bathroom with your friends or something and and it's incredibly noisy because the sound reflects off all these hard surfaces.

9:10So what sound sounds like depends a lot on the nature of the room that you're in. So a room that's got lots of carpets and soft furnishings will sound very different from, say, a municipal swimming pool which has got hard surfaces. There is a thing called the reverberation time as well. So if you go into a big cathedral or something, and you might hear that when somebody stops playing the organ or when whatever sound that is stops, the sound echoes. There's a reverberation that might go on for quite a few seconds. Well, really big buildings have a long reverberation time. But your average living room with carpets and curtains and things has virtually no reverberation at all, which is just as well.

10:00otherwise you wouldn't be able to carry on a normal conversation in that kind of room. Maybe Hugh and I could start a band. Or perhaps not. So now we know how sounds are made, we are going to explore how our ancestors used them to communicate and turn experimentation into music and instruments. Who better to help guide us than Ian Cross, an emeritus professor of music and science at the University of Cambridge. It's terribly easy just to think of music as it is in the present day, when it's something that is actually pretty much a commodity, produced by a select few and consumed by the many. But actually, in most world cultures, music is something that is done, and it's done in concert with other people.

10:43The most prevalent form of music worldwide is probably singing together. And if we think of music then in terms of what's been called participatory music making, then it's probably as old as human origins at all. Probably the first thing that humans did was communicate with each other. So could we say that sound came before language or is this something that came hand in hand? Fish are sensitive to sound. So yeah, sound came before language. But language, it's not just sound, is it? Even as I speak at the moment, I'm gesturing with my hands. Why am I gesturing? You can't record that. But I'm gesturing because it's what we do.

11:25Speech communication is multimodal. is not just sonic, it's also gestural. And how has the use of sound and music and language as a form of communication evolved? Our hearing sensitivities in the modern form were probably set, ooh, about 450 ,000 years ago or perhaps longer. Neanderthals certainly had a hearing apparatus very similar to her own, which is maximally sensitive at frequency regions between about 2 ,500 to 3 ,500 hertz, which is roughly the loudest area in the spectrum of speech. So there's probably been a co-adaptation of capacity to produce sounds and to perceive sounds that is focalised around speech, around social signals.

12:12Which makes sense because humans are a preternaturally social species. Take one human and they're pretty useless. Take a bunch of humans and they can generally work out something that probably is the marker of human beings, that we are a social species. So it makes sense that one of the key early elements in our repertoire of behaviours are behaviours that are oriented towards enabling us to interact socially, whether we call that speech or music. So in terms of music, when can we trace the origin of instruments back to? The earliest instruments we knew of are Born and, interestingly, mammoth tusk ivory pipes dating back about 40, maybe 42 ,000 years ago, found in, I think it's Vogelherd in Germany.

12:56Mammoth tusk ivory is interesting because to make a mammoth tusk ivory pipe, you have to kill a mammoth. And then you have to split the mammoth tusk in half, core it, rebind it, pierce it with holes and fix some sort of mouthpiece on. So there's quite a lot of actual complex construction goes on, really time-consuming. Now, 42 ,000 years ago is about as soon as humans get to Germany. So more or less the first trace they leave is music, which means it was probably considered to be quite important and worth investing a lot of energy, a lot of time in. Now, those are very sophisticated instruments, analogous to historic musical instruments, never mind prehistoric.

13:39so it's almost inevitable that there were some forms of musical instrument before that. I had a project about 25 years ago looking at whether or not you could use things we could think of as flint tools to make sounds that we could interpret as music and to explore whether or not having used these things to make sounds you could tell that they've been used to make sounds. The answer to both questions is yes and yes. You can use flint tools to make quite interesting sounds And having used them to make sounds, you have diagnostic patterns of use wear on the surface that are spatially differentiated according to the vibrational characteristics of the object.

14:21Effectively, we were using Aurignacian type flint blades, the type of flint blades that would have been found in northern Europe 40 ,000 years ago, which are long, thin and not wide. They behave like chime bars, which means that they flex around certain points of null movement, nodes of vibration. Now, if you hold them by one of the nodes of vibration and you tap them at that point, they won't make a sound. If you tap them at an antinode, one of the points at which the stone will move most, then you do get a nice sound. You also get a mark. so if you look at the surface of a flint of that type a blade and there are marks at antenatal points then it's probably been used to make sounds fascinating so you said these flint blades originated from around northern europe but how has music evolved differently in different cultures around the world as different as languages as different as cultures um the way i think of music now is as a medium for affiliative social interaction.

15:23That's pretty broad. So any way that you can deploy sounds, gestures, and almost always dance. Most world cultures really don't distinguish in music and dance. If you can figure out a way of deploying movement, sound, to make people feel good about being together, it'll work. Whatever it sounds and looks like. What impact did recording technology or the development of recording technology have on the way music is created and shared? It's had a huge impact but it's not just the recording technology, it's who then owns what is produced. When does music become a commodity? In Western Europe that's probably around about the 17th, 18th centuries and the form in which music becomes a commodity is as printed music as sheet music.

16:11And the people who own that are not the people who wrote it, not the people who made it. They're the people who've got the engravings, the publishers. So as soon as you've published this composer, it's yours. So effectively, you have a situation where music in one form or another, in the 18th century, in the form of engravings, sheet music, was becoming a commodity. And control over its ownership and access was being assimilated into a capitalist system. The same thing happens through the 19th century and 20th century. There are ways in which music becomes commodified as sound, in the case of gramophone records, cassette tapes, CDs.

16:52And now music isn't even a commodity, it's just a means of monetising engagement that enables large platforms to scrape data from the users. Ian Cross, Emeritus Professor of Music and Science at the University of Cambridge. We'll be hearing a little more from Ian a bit later on. The Naked Scientist podcast is produced in association with Spitfire, cost-effective voice, internet and IP engineering services for UK businesses. Find out how Spitfire can empower your company at spitfire.co.uk. music in the program is sponsored by epidemic sound perfect music for audio and video productions this is the naked scientist podcast with me rachel ralph today the science of music in a moment as ian was just alluding to we'll be unpacking how both ai and the latest technology are ripping up the song sheet but first we're going to explore when humans first perceive music research suggests that this happens at the end of the second trimester of pregnancy, when the foetus develops the ability to hear.

18:02Interestingly, studies have also shown that music can play a part in brain development before birth. For example, a study from the University of Helsinki in 2013 showed that newborn babies could remember a version of Twinkle Twinkle Little Star when played to them in the womb, and responded differently when alternative versions of the song were played. Other studies have shown that babies are born with a natural ability to detect musical beats, where we often see videos online of toddlers waving their arms and bouncing to music. However, Qin Yuan, a researcher at the University of Heidelberg, was interested in finding out when babies develop this ability to respond to music with movement and how this develops in the first year of life.

18:46She's been speaking with Chris Smith. So when we set out to do this study, we were quite inspired by lots of videos and our own experience of babies dancing. But the question that nobody has answered thus far is when does this start? So how does it emerge? And when we then look at the literature, we can see two types of studies. One type of study is looking at how babies perceive music. So what parts of the music can the baby already process in their brain? And then the other studies, which are very few, look at how babies move to music. So they were playing Mozart to the babies. They were playing Backstreet Boys to the babies and trying to see, OK, can they dance already?

19:29And they didn't find anything. So we wanted to take these two types of studies together and then look into, OK, how does the baby perceive the music in the brain and then translate that into movement and potentially dance? They say you should never try to work with children or animals. You've made life very difficult for yourself. Presumably that's exactly what you've had to do then, to actually present sound musical stimuli to babies and measure those things. Yes. The babies were upset because we put them in a chair and we put a cap on them to measure their brain activity. Once we played the music, that's when they calmed down and were actually happy, at least for five minutes, to listen to the music.

20:13And how do you know when you play music that that's music to the baby and it's not just the fact that there's a stimulus sound that is making them respond? So this is what we exactly manipulated. So we first played them normal music, meaning children's songs, Spanish and Hungarian play songs, so that the infants would be equally unfamiliar with those songs. And then what we did with those songs is that we shuffled them. So they were a bit jittered, meaning that the tempo or the timing of the song was a bit all over the place. And then what we did as well is that we shuffled the melody. So it had the same tones, but just in a different order, meaning that we actually played random sounds of them trying to see if those two conditions then differed in terms of how the babies responded in their brain and in their movements.

21:01What about pitch though? Because one of the other things that babies allegedly do is respond differently to higher and lower frequency sounds. Perhaps that's something to do with maternal voices being higher pitched or whatever but a lot of the nursery rhymes and things that you play to babies they often tend to be dominated by high frequencies so did you explore that? Exactly so we explored the pitch aspect as well and as you just said mother Reese is characterized by high pitch so we're kind of thinking okay if it's infant directed music or children's music it will probably be the high pitch as well but what's really interesting is that what we know from adult research is that it's actually the low pitch that is driving dance or movement in general.

21:46There was this really interesting study by a Canadian group where they had a DJ in the lab and he manipulated the low frequency of the music and found out that then the people would dance more if he played more of that lower pitch music. So we're trying to kind of contrast these two in our study as well by playing the same songs, but either an octave higher so in a higher pitch or an octave lower meaning in a lower pitch. And what did this all show? So if you could sort of take us through age by age and stimulus by stimulus what you saw and how the babies responded. So the infant age groups were three months, six months and 12 months because we really wanted to see that potential trajectory over the first year of life.

22:32So what we showed them first was the music and the shuffled music condition, right? And what we found there is that the infant's brain responses were very consistent to the music. So they responded to each tone in the music in an enhanced manner, while the shuffled music condition didn't really elicit any robust response. That's interesting, isn't it? So it's almost like a baby's brain is pre-made to decode music. We would need more studies to show this, but I think the sensitivity to music in comparison to like random sounds is really, really interesting. Yeah. How did that change over the year?

23:11What we can see is that the brain response in itself seems to mature, meaning that it becomes stronger and it becomes faster. And in terms of the frequency effects, what we find is that there is a high pitch sensitivity in the baby's brain, but only at six months of lives and then it kind of dilutes. And what happens with the movements? What we find is that the babies, they show some sorts of movement that is informed by the music. So there's some rough coupling. Yeah, we call it that way because what we did in terms of analysis is use the music to predict the movement and there we see some correlation.

23:53But then in terms of robust responses in the movements, that's something we only see in the 12-month-old infants. So in a nutshell, babies are born able to listen to music and respond to it, but they don't get to dance until they're a little bit older. That's true. Why do you think that you see that pattern? Some of the accounts say that we're able to move in time to music because it helps us process the music. It is rewarding. Like being able to extract these patterns in music, in pieces, and songs. That's just something that's really enjoyable to us. And then on the other hand, there are accounts saying that also once we're able to dance together or once somebody else sees these types of movements in us, that's some sort of signaling.

24:36It's a signal that either I'm in the same group with you, like in birds, you can say I'm trying to court you. That's when you see birds dance as well, right? I wondered also if this is actually partly how we learn to speak, because we match a movement with a sound stimulus. And obviously that's important for creating expressive speech as well as comprehending speech. And I wondered if we've used that evolutionary trait and turned it into a mechanism for learning languages. Yeah, I think that is absolutely an option. What the theory says, or what the theory doesn't say, is whether language and music came first or whether it was language that then influenced music and dance.

25:20So we don't really know the directionality there, but there's a lot of parallels between language development or language evolution and music evolution in terms of speech and song and then gestures and dance, as you just said. Qin Nguyen at the University of Heidelberg in conversation with Chris Smith. And that study was recently published in eLife. If you want to listen to our podcast series with eLife, visit thenakedscientist.com slash podcast slash elife dash podcast. And finally, we look at the present and the future. Artificial intelligence has been integrated into many aspects of life, and the music industry is no exception.

26:01From generating melodies and lyrics, to assisting with music production algorithms, AI is transforming how music is created, distributed and experienced. However, while some of these technologies are tools to expand creative possibilities, others raise important questions about authenticity, copyrights and the future careers of real-life musicians. Here's Yacopo De Baradinis, an expert on music and AI at the University of Liverpool. AI has been integrated in many ways in the music industry. If we think, for example, of a system that can automatically analyze music, listening to hundreds of thousands of minutes of music and extracting information like the emotion, the structure, the tempo, the rhythmic structure of the song, the playability, the danceability, and so many other features, this is technically called music intelligence.

26:56So it's basically extracting a lot of information from music in an automatic way using machine learning models and then using essentially this information to power a system like recommendation and automatic music retrieval. And this typically supports other applications in the industry. For example, you can separate music into different sources. You can separate the guitar from the mix. You can annotate a song with chords and other annotations. You can even extract segments from the songs. You can mix different loops together. You can find the missing loops. There are so many applications. So touching on the music recommendations and discovery, let's talk about how AI is impacting music streaming and the listeners' experience, for example.

27:48So I know Spotify has its own AI DJ, DJX, which is quite funny to listen to. But yeah, how has AI been impacting this experience for the listener? So this is typically an example of music personalization and is also very much linked to what I mentioned before. So as part of the music intelligence platform of a music streaming company, you have essentially a huge database of songs and audio that, first of all, you want to annotate, you want to describe. So that's why you want to extract this information like the emotion, the playability, the dance ability, the rhythmic complexity and so many other features so that you have essentially features and information that can be used for personalization.

28:30so on the other hand while you're listening to your favorite song on spotify or any other streaming platform the system is basically watching you so depending on for example how often you skip a song what is your favorite album what are the songs that you play the most so they're basically trying to build a profile of your own music likings so that they can recommend songs that match your profile essentially and that's how all those features manually computationally extracted before are going to be used so basically matching a song which hasn't been found by a user automatically without even knowing the metadata so essentially the title and artist and so on and this is extremely powerful because this allows user to discover new genre new artist even emerging artists.

29:23So this is actually a problem in music streaming and music recommendation. So some artists which are very popular may take the precedence over other minor artists. But through this good use of AI, you can basically discover missing information and missing music to recommend to users regardless of their provenance. So do you think those smaller artists, is there evidence to show that this is working for them and they are getting more listeners with this AI software recommending their music? Yes, I believe this is a good use of AI. And this is something which is basically allowing system to annotate and listen to many songs and extracting information that can be used for personalization.

30:08So it doesn't matter if it's from Coldplay or from a local artist in Liverpool, the system will still be recommending that song to other users. So in In theory, the technology is strong enough to power this kind of applications. So what about music creation itself, talking about artists? We hear that there's a lot of AI-generated music now. So what do we mean when we say that and how common is it becoming? So if we start on the good side, for example, we can think of a new system that can personalize not just the listening experience, but also the creation of new songs. Like today, for example, everyone can be a photographer, a professional photographer, and can take amazing pictures because there is an AI which is enhancing essentially the configuration of your camera, your setups, and you can take beautiful pictures even without knowing anything about photography.

30:58So this day you can do the same with music and you can create maybe a symphony or even a complex piece of music without knowing anything about music composition. So in a way, AI is democratizing access to music creation to basically any user without the necessity of having a very strong skill set in this particular case. But at the same time, we should ask the question, is this what we want? Is music a service? Is music a product? And how do we move forward from this? Yeah, I can imagine there must be a lot of ethical concerns associated with using AI and music generation. For example, who should you credit for the music and copyright challenges as well?

31:43So what are some of these other ethical concerns? So this is a very, very challenging domain at the moment, because the success of this model pretty much depends on the quality of the data that is used to train them. So these models will be basically nothing without all the thousands of hours of good music that they've listened to. There are copyright issues, there are ethical issues, especially related to how can we also recognize the contribution made by human artists. Is there a way that can allow us to give credit to the artist and also redistribute the revenues made from generated AI music?

32:25That is definitely not something that a single expert can solve. It requires a multidisciplinary approach. Jacopo de Boradines at the University of Liverpool. So that's it for today. We'll have the latest science news stories from the week on Friday, including a sharp rise in children's mental health referrals. We'll also have our usual updates on LinkedIn, X and Instagram. And if you would like to support what we make, you can do so at thenakedscientist.com slash donate. It wouldn't be a show about science and music without a bit more music. So here is Ian Cross, who we heard from earlier, to play us out.

33:04I'm Rachel Ralph, and from everyone here on the team, thanks for listening. And until next time, goodbye.