When you stream a song on Spotify, it's delivered in an audio format — imagine these formats to be containers as literal as a phonograph record — cheekily named "Ogg Vorbis." YouTube, one of the most popular music streaming "services" in the world by volume, prefers something called AAC, or "Advanced Audio Coding." Radio stations, whenever possible, tend to prefer lossless WAV files. If you've played music from your hard drive as far back as the early aughts or as recently as this morning, you may have clicked play on an MP3. You'll be able to tomorrow, next week or a decade from now, provided your music player of choice still has the electricity to do so.
We've been the source of some confusion on that subject recently. An article published here 11 days ago quoted Bernhard Grill — one of the developers of the MP3 format and also of the AAC format for the German research institution Fraunhofer — saying AAC is technologically superior to the MP3 and also that AAC is one of the most widely used formats for listening today, through its ubiquity within streaming, if nothing else (and there's more). That article's headline oversold Dr. Grill as having characterized the MP3 being dead. It would have been more accurate — and more interesting really — if the writer of that article (the same as this one) wrote that the MP3 may be dying. (That article didsay explicitly that people would go on using MP3s, but hey.)
To parse this tipping point (and to explore whether we're even in the midst of one) we called — after he reached out to us in response to that earlier article — the author of MP3: The Meaning of a Format and a professor of communication studies at McGill University.
As with VHS tapes and cassettes, no format is "dead," as long as the equipment to play it is still alive. The resurgence of vinyl records is illustrative of that point. But the tide seems to have turned — and that turn, like most of media history within capitalism, is complicated by economic interests, advances in technology, an ever-increasingly ephemeral connection to media and, either most or least importantly, the subjectivity of our personal experiences. All of which we explore with Sterne.
Fraunhofer's patent on the MP3 may literally be dead, and with it the institution's monetization of it — but there are, as Fraunhofer itself points out, still trillions of MP3s out there. Or, as Sterne explains: "MP3s are still the most widely used audio format in the world."
Andrew Flanagan:What does Fraunhofer's patent lapse allow developers to do now that they couldn't do before?
Jonathan Sterne: "That's a great question. So... people have a really hard time understanding intellectual property. If you think of it like a trade monopoly, it's really easy to understand. Who has the right to trade in a certain code or perform or record or claim authorship over a song? Things like that, those are trade monopolies. They're not property in the sense of 'If you steal my car I can't use it.' Right? So the patent is a monopoly — which, actually, Fraunhofer didn't fully exercise. First of all, users like me, listening to music at home, never had to pay for the rights to use the MP3 decoding protocols. In fact, if you build a little free software thing that made no money, you also wouldn't have to pay. So there are already unlicensed MP3 encoders and controllers out there."
Like.
"Yes. So they're not paying Fraunhofer already. But, if Apple puts out an MP3 decoder, they pay Fraunhofer. And that's what's going to stop. Fraunhofer has always maintained that AAC is better than MP3. It's more efficient and, at low bitrates in test situations, people can sometimes hear the difference in terms of fewer coding artifacts and things like that. When you put on earbuds and get on the metro, you won't hear the difference at all."
If you're using pretty nice headphones you might?
"Not on the Metro. One of the people involved in the MPEG group said it's like when you buy a new home. First, you noticed a nice view out the living room window, and it's only later that you notice a little crack in the window. It's like that. You have to learn to hear the artifacts.
"And that's just the way it works. That's true for any new audio or video format, people start noticing things that they didn't used to. Whether those things bother listeners is entirely subjective and it's entirely situational. If you think of the way people listen to music most of the time, to say AAC is better than MP3 — yes, by technical standards, absolutely. If you're sitting in a quiet room paying rapt attention to the music, sure. The minute you put it through earbuds, or go for a walk, or play it in your car, or listen distractedly, those differences matter less and less."
At the root of a lot of the feedback I got from the article is the availability of the underlying technology of the MP3. I'm wondering: If the MP3 is not dead but finally free, what is it free to do?
"Honestly, I don't think much is going to change. I wouldn't say 'oh the MP3 is finally free and all these things are going to change.' All it's going to happen is Apple is going to stop paying Fraunhofer, or whoever. (I don't actually know that Apple's paying Fraunhofer because I'm not privy to their licensing agreements, but whoever's paying them is going to stop paying.)
"That's why I pushed back on the 'MP3 is dead.' It's not like the switch from vinyl to compact disc, where U.S. record distributors stopped accepting returns on unbought records from record stores, at which point stores had to switch over to CDs or take on additional financial risk. There's nothing that Fraunhofer can do to prevent people from using MP3s.
"So, since it's the most common form of audio format in the world — right now — a more likely scenario, if you look at the history of audio formats, is that over time its use will continue and then eventually it will decline, like all audio formats before it. It will eventually be replaced by something else, but that won't be because Fraunhofer stopped supporting it. It'll be because something else came along, and people will have a reason to switch either in terms of consumer motivation or because of some digital rights management scheme, or some kind of technological system that is a closed system."
So things aren't really going to change, regardless of the patent, or regardless of the availability of superior technology, whether they be OGG or AAC.
"So, the one story you get with audio is that there's this inevitable march of progress towards higher definition, and higher definition is better and more realistic — which isn't always true — and that MP3 is some great aberration where we took a step back. That has nothing to do with how the history of audio technologies have actually unfolded. Yes, the CD has much more theoretical dynamic range than an Edison cylinder. Of course. But what you see are, every few decades, format wars, where one format ekes out another, usually because of social conditions, like which players in the industry are more powerful. Or accidental conditions — the reason MP3 took off in the first place was because their shareware program was hacked and people started using MP3 encoding for file sharing — and then companies caught up to it, and Fraunhofer benefitted. They actually didn't very successfully market it.
"When things started going well for them, they were able to seize the opportunity. And, you know, they use a lot of money to fund their research. It's not like there's some big corrupt media corporation. It's just that, the minute they invented AAC they said 'this is better.' So they've been saying that for probably, I don't know, 20 years now, give or take. And now they have no financial interest in MP3s carrying on.
"So it's a great headline to say MP3 is dead, but they're still all over the Internet, people's hard drives are still full of them. This is really saying like, '45rpm vinyl is dead but 33rpm vinyl continues.' It's a sort of narcissism of small differences."
His quote — Bernard Grill's, who helped develop the MP3 and AAC — was that AAC is in such wider use on platforms like YouTube and streaming services. What I took was that it's dead as far as the usefulness from now into the future.
"I would totally disagree with that, I think it's totally useful for people who want to listen to their music that they've already acquired or paid for.
"You know, I was actually getting e-mails from people saying 'does this mean I have to like, re-encode all of my music or acquire it in some different format?' And that's exactly what you don't want. This sort of media industry model where everybody buys the same movies TV over and over again."
Let's make that very clear: MP3 and AAC are free for any user (not company) to take advantage of.
"Yes, absolutely. I mean the reason why the free software people didn't like MP3 was because it was tied to royalties and Fraunhofer, whereas OGG was completely open source, anybody could use it anyone could contribute to it."
The MP3 and other formats are based on some psychoacoustic assumptions that I was told last year were at least mildly off the mark.
"Any psychoacoustic assumption is a generalization, because we're all walking around with two sets of ears and one brain, and psychoacoustics are all based on aggregate data populations. And in some cases, weird populations. So like 20Hz to 20kHz — the frequency range for 'normal' hearing, is based on research done on New York City schoolchildren. My colleague has written about that.
"The original psychoacoustic assumptions that Karlheinz Brandenburg tried to code into the OCF, which was the format before MP3, he took out of Eberhard Zwicker's book Psychoacoustics — he literally took those numbers and put them into the software. And they sounded terrible. The reason they sounded terrible is because psychoacousticians do tests using tones or noise bursts, and MP3 was designed to encode speech and music. And so they started doing this basic research to say: 'If we tweak it this way, do the cymbals sound worse or better? If we tweak it this way, does the articulation of the voice get muddier or clearer?'
"A lot of the things that people complain about with MP3s are not the result of the removal of data but rather the imprint of the format on that sound. So things like changes to the stereo image, that's not a result of removing data, that's the result of the way the format codes.
"So yeah, it's always a compromise, and especially as you get to lower bit rates, you're asking the algorithm to basically choose. 'Where is the stuff people are least likely to miss?' And it can guess wrong, because it's an algorithm, not a person. And it has limited information. What's amazing is how well the MP3 worked, and that speaks I think to the standardization of recording, and audio, and radio sound across the world. People expect well-recorded or well-broadcast sound to sound a certain way, regardless of where it comes from and what genre it is."
Now I'm wondering about the end game of fidelity.
"If you look at history, which is really the only empirical evidence we have, sound fidelity has been used to market new audio formats basically since the invention of the cylinder phonograph. So in 1878 you can find articles saying that Edison's tinfoil phonograph reproduces sound with perfect fidelity. So there's always the story where we're at 'the end stage of high-definition,' and that greater definition will bring people greater musical pleasure.
"What actually happens is the aesthetics switch — so people want more bass, they want less bass, they want more high-end, they want less high-end. People talk about how 'organic' [vinyl] records sound, but of course every record, in the U.S. anyways, goes through the RIAA curve. So it is in fact a highly processed and arbitrary and standardized sound, in exactly the same way that WAV or AIFF or MP3 is. So it's been a successful technique for marketing audio.
"The second thing is, I want audio engineers and record industry professionals to care about audio quality. Does that mean that higher audio quality will save the music industry (which is still highly profitable, it's just not as profitable as it was)? Does it mean that it will get more young people interested in paying for music? Does it mean that the people who pay for music will pay more for music? I think that's a social question. The person who buys the nicer speakers for their home stereo is going to be interested in this. The person that's like, playing music out of their telephone speaker in the back of the bus for their friend, this isn't going to make any difference to."
It might be apocryphal, but it reminds me of that Steve Jobs quote. They don't know what high-fidelity is until they hear it.
"And when they do, they may or may not care. This has been true since the term high-fidelity was invented. And it's independent of love for music. You can find people who are incredibly devoted to classical music and listen to it out of a small six-inch mono speaker on a desktop radio. That's fine for them, and they will tell you they love the music. They might even go to the symphony and give money. High fidelity isn't part of their experience. High definition has sold more TVs lately, but is it going to sell more music, is it going to save the music industry? I don't know. Probably not."
There was a lot of, I think, misinterpretation of the article, from open information advocates and open source software advocates. I was curious if the availability of the source code for most of these formats has been pretty widespread for almost two decades. What are they missing out on?
"I mean, I think it's just a matter of ethical commitments, that software that's widely used ought to be ought to be... well first of all, it ought to be free. It ought to be freely available to modify yourself. I mean, I'm sympathetic, but I'm not a member of that community. I use plenty of proprietary software in my daily life. So I can't claim to be an authentic open source person and I don't code."
Like.
"Yeah no, I'm certainly not Richard Stallman. The kernel of it is this, and I think that this is true for all technology, which is that a more democratic technology is a technology that you as a user, even if you don't understand it, you have the potential to understand it, repair it and modify it. Where the inner workings of the technology are not controlled by a third party over whom you have no power and on whom you are dependent.
"I wrote a paper with Dylan Mulvin on the history of the switch from monochrome to color TV in the 1950s. In the 1950s the US government told the TV industry — remember that the broadcast bandwidth is a public resource and it's managed by the government, they control the national television system. The government said sure, by all means move to color, but you can't change the standard in such a way that you will force people to buy new televisions. Completely different business model from Apple releasing a new iPhone every year to get you to buy a new one.
"In Europe they have things like extended product responsibility, where companies are legally liable for the disposal of the products that they produce. In the software world like it seems like this is all just about code, but of course code runs on hardware you need something to run it. And so every time you obsolete a standard, that's a chance for a company to either force or try to persuade someone to buy something new, which means something else is likely to go into a landfill."
Which brings us back to the MP3.
"Yes, it does."
Copyright 2021 NPR. To see more, visit https://www.npr.org. 9(MDEwMTk5OTQ0MDEzNDkxMDYyMDQ2MjdiMw004))
