Do Higher MP3 Bit Rates Pay Off?
Posted 04/19/07 at 07:07:49 PM | by Logan Decker
In a tidal shift akin to the Sexual Revolution, the heyday of Napster (that’s 1.0) markedly changed Americans’ perceptions of intellectual property rights and fair use. Some folks considered the music-swapping orgy a shameful, lawless epidemic of hedonism that would inexorably lead to social and economic chaos, while others considered it a thrilling, lawless epidemic of hedonism that liberated all of us from the musty stench of an increasingly bland and commercial culture. But everyone understood one thing: MP3s were here to stay.
Back then, most MP3s were encoded at constant bit rate (CBR) 128Kb/s, striking a balance between acceptable audio quality and file sizes that were small enough for easy trading over a dialup connection. But what was born out of necessity endures today, as most of the music available on rogue peer-to-peer networks is still compressed at this rate. It’s been called “near CD” quality, but we know better—it isn’t even in the same Zip code as CD audio.
But at what point do higher bit rates stop paying off and simply take up too much space? 160Kb/s? 192Kb/s? And can a hardcore audiophile really tell the difference between a 320Kb/s track and an uncompressed one? What about a normal music listener?
These are the questions we wanted answered when we set out bear traps around the office and came back later that afternoon to retrieve the snared employees. From the bunch, we selected four representing a range of musical tastes and quality demands. We handed them headphones, pressed the play button, and got some surprising results.
How We Tested
Each lucky participant was asked to bring in a CD with a track that he or she has listened to for years and knows so intimately that a single missing hi-hat tap would stand out like a sudden blast from a tuba. We ripped each track using iTunes at three quality levels: 160Kb/s, 320Kb/s, and uncompressed WAV. The compressed files were ripped using variable bit rate (VBR) encoding, meaning that a 160Kb/s VBR track allows the bit rate to rise and fall depending on the complexity of the music while maintaining the selected bit rate as the minimum bit rate for the track.
In a quiet room with mood lighting and kitschy Scandinavian furnishings, the participants put on a pair of Sennheiser HD 580 headphones that were attached to our test PC’s Creative X-Fi soundcard. The participants listened to not only the three versions of their own track, but also the three tracks from each of the other participants, for a total of 12 tracks in all. Each participant was allowed to listen to each track as long as he or she could stand it, and was allowed to repeat portions of the track and do A/B testing with the other tracks.
How Good is Your Encoder?
The thing that most often fools newbs is not knowing what kind of sounds are evidence of bad encoding or heavy-handed compression, but there are a couple tracks you can throw at your software to see how well it’s doing. Of course, you should begin with uncompressed files, so you can’t download these test tracks from an online service. If you don’t want to pony up for the CDs, you might want to check your local library.
The first is “Neon Reprise” from Metropol by Lunatic Calm. The song contains a whooshing upward sweep that audio compressors have a very difficult time with. Turn off Joint Stereo and keep upping the bit rate if you hear any warbling (like a cassette tape that’s been mangled).
The second comes from the legendary 1965 RCA Victor Opera Series recording of Verdi’s La Traviata. In Act II, Track 10, position 3:16, Anna Moffo belts out a bone-liquefying high note. Focus your attention not on her voice, but on the instrumentation behind it. If there’s any fuzz or rumbling in the background, again, dial up the bit rate.
Comments
Print
Email
The listener matters
Submitted by nantucketbob on Wed, 2008-10-22 06:06
This is a good article. The comment by Chris is revealing in that he is a trained listener, and he has access to professional equipment. I would like to see a test done with quality consumer grade equipment (PC, MP3 players, stereo systems) with professional trained listeners, such as "Chris." I would also like to see a test on professional grade equipment by both trained and untrained listeners, even though the conclusion should be obvious. Trained listeners would include audio engineers as well as musicians (probably conductors). Enjoyable article!
Submitted by anon on Sun, 2008-09-07 03:45
Greg Calbi and you can believe whatever you want. I am not asking you to believe me. I am asking disinterested readers to go to those three links, read, use their brains, and decide for themselves. The quality of the wire does matter. That is not to say that expensive wire will always sound better. That is to say that expensive wire will never sound better. It is to say that many audiophiles are suffering from confirmation bias and post-purchase rationalization.
Fools and their money are soon parted.
Submitted by orlanthrex on Wed, 2008-09-03 06:28
I have not met an audiophile who hasn't been an @sshole.
they like all other tech cultists, (Apple devotees, and Volkswagen cultisti just to name a few) are insufferable to be around.
if you actually can hear the difference between them....good for you....you must be a miserable human to be constantly disappointed in life.
Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn
In his house at R'lyeh dead Cthulhu waits dreaming
Submitted by ctm on Fri, 2008-10-10 08:38
True, my friend. I was thinking of going on a journey to find the perfect stereo that would show me exactly how bad my music sounds. Then I was thinking of simply turning the stereo I currently own WAY UP and bobbing my head to some groovy tunes while I carry on in ignorant bliss. shoot, the bass sounds awesome!
Submitted by fanfarefan on Wed, 2008-06-04 05:44
Actually the scores are not necessarily humiliating, and they are very good when you look at favorite songs only. I would say the test indicates that people can indeed hear the difference, although not every time.
Expected avg random guess score: 4
The actual scores: 3, 6, 5, 5
Avg score: (3+6+5+5) / 4 = 4.75
Favorite songs score: 3+3+0+1 = 74.75, the average score for the four testers, represents 19% more right answers than predicted by chance.
Not bad, and it gets much better when we consider everyone's favorite song only. The single-song scores for the favorites are (3, 3, 0, 1). So the total score for the four favorites is 7, almost double the number of right answers the testers would have been likely to get by guessing.
To figure the expected average random guess, consider that for each song, the testers are being asked to put the three versions in order by quality. For three things, there are six possible orders {ABC, BCA, CAB, ACB, CBA, BAC}. If "ABC" represents all three right, then the six orders give scores of {3, 0, 0, 1, 1, 1} respectively. Random guesses would randomly get one of those scores, for each song. So for each song, the expected avg random guess score is (3+0+0+1+1+1) / 6 = 1. For four songs, the random score would be four times that: 4 x 1 = 4.
Submitted by anon on Wed, 2008-05-28 05:53
There is a lot of nonsense regarding what can and what cannot be heard; the only proper way to do this is with double-blind listening tests. First, use what may be the best MP3 encoder—LAME 3.97. Second, use a program to perform double-blind listening tests--http://wiki.hydrogenaudio.org/index.php?title=ABX.
Submitted by EarthBoundMisfit on Sat, 2007-11-17 23:02
Registered Linux User #404122
Microsoft has encountered a critical system error and must now shut down. Better get Bill Gate$ on the phone for this one.......
[img]http//i35.photobucket.com/albums/d183/NunofyaBidness/404122.png[/img]A very nice demonstration of what I already knew.
I have a friend who claims he 'can tell' when he hears an mp3...at ANY bitrate.
As a former disc jockey for 2 1/2 years...what many folks do not know...is that music you hear overnights or a lot of programs...is now in mp3 format.
Nicely done.
Glenn Condrey
Gulfport, Mississippi
Submitted by HWood on Tue, 2007-06-12 07:50
This can be a bit misleading and confusing. Anyone trying to make a comparison using headphones should think about a few things. First headphones are not a good speaker to perform a comparison. Lets go back in time a little when Compact Disc came about most people said CD was better than LP's. Most people have never owned or listened to a good Turnable Phono Cartridge setup ever. So how would they even know. Now with that said, most people now are listening to Earbuds, car stereo's, computer speakers, and House audio systems. Which are all very forgiving of the source. Most of those people have never owned or heard a good stereo system (Not Home Theater System). I am not surprised by the results. A very simple but conclusive test could have been done using a good consumer grade CD player, Stereo Receiver and a nice pair of floor standing speakers or a Subwoofer and 2 smaller speakers. I believe the results would be different.
Submitted by soggybomb on Fri, 2007-06-01 17:24
I have had great success with Creative's Mediasource Audio Converter. Because Creative values audio quality (and because the software is packaged with my X-Fi soundcard) it sounds great after encoding.
Submitted by transmothra on Sat, 2007-05-26 23:27
All you really need to hear it instantly is a song with cymbals and a piano. Boom! If it's wobbly-sounding, or kind of garbled like it's almost underwater, crank up the bitrate. Nothing to it.
If you don't notice a difference, then it doesn't matter - just don't share your crappy mp3s with me.
~jer
http://jeremyjarratt.com/
Submitted by SysAnalyst on Mon, 2007-05-07 12:06
For the longest time MPC recommended using Exact Audio Copy and LAME for encoding to MP3. Recently, Will Smith, I believe, mentioned using straight iTunes for ripping. This was a surprise to me as I had been following the EAC/LAME method for my whole CD collection. Why this change in method?
Submitted by lacalaca on Mon, 2007-04-30 07:32
Your test can be misleading. Using a cheap CD-drive to grab music data and make them play via "the finest consumer-level soundcard" as reference is not the real quality-alternative to MP3 listening.
CD sound is compressed anyway (in term of dynamics) and represents a 25-year-old digital technology (which at the moment of its birth had its compromisses: low sample rate, low resolution). A cheap PC-CD-drive will read only a portion of that low-rate music data, and the software tries to interpolate the missing data (which are missing because of the low sample rate AND the poor disc-reading). It sounds like a lossy compression, isn't?
I suggest you a test:
Config A: Your PC with that "finest consumer-level soundcard"
Config B: A really good consumer-level (not an "audiophile") CD-Player (try a Marantz SA 7001 with its built-in headphone output, or use a decent external headphone-amp with reasonable cables).First, drop in two identical original CDs, and compare the uncompressed PC-sound to the Marantz. You don't need to lean forward to hear the difference... And if you switch to MP3 on the PC, that difference will be more clear (watch the trebles, the stereo room-rendering, and the body of the instruments). Use acoustic music and human voice to reveal the unnatural sound of MP3.
And then drop an SACD into the Marantz (the Miles Davis "Kind of Blue" does the job), and compare to the CD-ripped, MP3-compressed PC-sound of the same music... Now, THAT is the real alternative.
Submitted by rogergraham on Wed, 2007-05-30 07:12
"Your test can be misleading. Using a cheap CD-drive to grab music data and make them play via "the finest consumer-level soundcard" as reference is not the real quality-alternative to MP3 listening.
CD sound is compressed anyway (in term of dynamics) and represents a 25-year-old digital technology (which at the moment of its birth had its compromisses: low sample rate, low resolution). A cheap PC-CD-drive will read only a portion of that low-rate music data..."
*cough*bullshit*cough*
Any cheap PC CD drive will retrieve every last bit of every CD, barring bad scratching etc.
How many programs fail to isntall due the CD drive read errors? How much data do people loe due to their drive not reading properly? You guessed it: absolutely fark all.
Submitted by cris on Tue, 2008-09-02 23:38
Haha. Oh yeah. You're an expert... NOT.
It's not bullshit. You're an ignoramus.
"Any cheap PC CD drive will retrieve every last bit of every CD, barring bad scratching etc."
You see the devil is in the details. Data discs are VERY different to audio discs. When you read a data disc in a pc cd rom, there is a thing called a checksum, and if it doesn't match, the sector is read and re-read until they do (or it fails). Thus a data disc will read perfectly, or not at all.
However, ripping an audio CD is a totally different task. It suffers from something called "jitter", and it DEFINATELY affects the quality of the sound. A trained ear and top gear is needed to hear it (the gear used in this test is ANYTHING but top shelf). Look jitter up. Have a look at the Plextor mastering drives, and their different error rates.
And you should stop pretending to knoweverything and realise you have a lot to learn. Believe me. I've been an Audio Engineer for over a decade now. The equipment used here will not reveal the difference. It is simply not good enough. They are dynamic headphones FFS, so you are looking at a good 20dB roll of from about 15kHz at least. So you are losing all the fine detailed high frequency information where you REALLY CAN TELL the difference between compressed tracks and uncompressed tracks.
All this test proves is the equipment they used is unable to resolve the subtle differences between the tracks.
Submitted by anubisjr on Mon, 2007-04-23 03:29
...if you want a fair test, use either the LAME or Fraunhauer codecs. iTunes uses an inferior mp3 encoder so their AAC will sound better by comparison. Don't believe me? Try it for yourself using both iTunes and Music Match Jukebox. The results are startling, even at 320kbps.
Submitted by sackynut on Sun, 2007-04-22 21:30
Personally I rip all my music in apple lossless. It lets me have a lossless copy of my music in the case I lose the physical CD, plus i have plenty of HD space.
For me, I need SUPER high quality headphones, and then songs that I can tell. Some songs (even with good headphones) I cant tell the difference. But others I can. I guess it depends on the song, quality of recording and quality of headphones (or speakers).
This is an awesome article and I was actually looking for my old MaxPC issue with an article similar to this, I was bummed I didnt find it. but now i found it yay!
Submitted by Enlightenment on Fri, 2007-04-20 20:01
I started converting my 500+ CD collection back in 1998 with L.A.M.E., way before most of the iPod punks ever heard of MP3 or any compressed audio. I had a necessity to do it because it was a major hassel to haul hundreds of CD's to DJ parties.
Back in those days when you couldn't buy 100's of GB hard drives for next to nothing so I had to experiment with what was reasonable. Even 9 years ago, I knew that 128Kbps CBR (constant bit rate) MP3 didn't sound good enough. I really wanted to use 320Kbps, but hard drives weren't large enough to hold my entire collection at that rate, so I used the next best thing of VBR (variable bit rate), over time I re-ripped my collection multiple times, and years ago finally said enough is enough, and just went to 320Kbps CBR MP3.
I have tried some comparison tests of my own over the years, and come to the conclusion that if I picked a lower bit rate, it might sound ok for a lot of my songs, but eventually I would find a song that didn't sound very good. The solution was to just encode everything at the highest bit rate of 320Kbps and just not worry about the storage that it used. Now that I can buy 1TB drives, even 320Kbps isn't a big deal, heck I could rip everything as WAV files and still not fill a 1TB drive.
Audio tests are hard to do, but I can tell you for a fact when I hear a really badly encoded song off the internet, oh my it hurts to hear it. Good examples of songs to find for the tests are ones that have: cymbols, snare drums, anything that rings, anything that has very high frequencies, anything that has stereo movement of sound, wide dynamics of volume in a short amount of time. When I listen to the music and not trying to do a test, sometimes I hear weird encoding problems and they are usually associated with those types of sound.
Submitted by maxhodges on Fri, 2007-04-20 18:31
thanks. could have been done better, but wasn't done terribly either.
in the end some people just CAN'T tell a difference between compression for some types of music, while others CAN tell a difference for some types of music, but not all types of music.
pretty much confirms things I've already known.
for most people, moderately compressed MP3s are 'good enough', and that's why they exist: because there isn't a lot of market pressure driving higher bitrates. The unfortunately thing is that very avid listeners who buy things like $1,600 amplifiers and $350 speaker wire (like me), can't enjoy the conveniences of digital downloads because their bitrates are determined by the tastes of the lowest-common-denominator mass-market consumer.
garbage in, garbage out
another thing is that a lot of the music they were listening to was probably already damaged pretty badly by the time it got on CD. sound engineers these days really mutilate the music by compressing dynamic range and trying to get the signal as hot as possible--so that it stands out the radio--but clip transients in the process. etc.my own test CD is Miles Davis "Kind of Blue"--but that wouldn't work for other people unless they've listened to it enough to know the duration of every breath. reminds me: I went into a Home Entertainment once (think they got bought up by Tweeter) to audition some speakers, and when I put my disc on I instantly noticed that the speakers were wired backwards. I prodded the salesperson a bit, but he didn't seem very willing to straighten things out, so I went and dropped my $3000 budget somewhere else.
Submitted by anon on Wed, 2008-05-28 06:22
Two myths are that expensive amplifiers and wires sound better than inexpensive amplifiers and wires: http://bruce.coppola.name/audio/Amp_Sound.pdf
http://bruce.coppola.name/audio/Greenhill.pdf
http://www.theaudiocritic.com/
Submitted by cris on Tue, 2008-09-02 23:42
You should tell that to Greg Calbi. I'd believe him over you anyday. The quality of the wire does matter. That's not to say that EXPENSIVE wire will always sound better. Its to say QUALITY wire will sound better.
Submitted by livewyre on Fri, 2007-04-20 17:04
"Many people agree that a Vorbis file remains indistinguishable from the original source material at q5 (~160kbps), whereas for MP3 a bit-rate of 192kbps is required to reach the same standard."
Quote from my website discussing optimum bit-rates for compressed audio (Vorbis being the variable-rate). For more please see:
http://www.cassette2cd.co.uk/mp3.htm
Good study, though after invesigating this myself, I am not at all surpised to 'hear' your results. I disagree in principle with the view that monitors (even huge theatre speakers) would reveal the imperfections, but that in itself would make another interesting experiment. Just because a piece of equipment delivers massive volume doesn't naturally make it more accurate when it comes to faithful reproduction does it?
Submitted by anon on Wed, 2008-05-28 06:04
Perhaps about 175 kbps: http://wiki.hydrogenaudio.org/index.php?title=Lame_Compiles#Recommended_encoder_compiles_and_source_code.
Submitted by azogue on Fri, 2007-04-20 10:47
As a professional theatrical sound designer and engineer, I am regularly asked to accept mp3s from directors and dancers.
Maybe not on headphones, but on LOUD concert and theatrical sound systems the compression can certainly be identified, even with variable bit rates.
Additonally, editing is not accurate with mp3s.
Casual listening, in the car or on headphones...fine.
But for any other use give me 24-bit 96 khz!
Submitted by Cybermynd on Fri, 2007-04-20 08:11
Although many would consider headphones to be the ultimate transducer for this type of testing there are people who consider themselves audiophiles that will argue there are more revealing systems that should be used for testing.
Not to say that most people will ever acheive or exceed the excellence of your test setup. I guess my point is that perhaps using a high end, speaker based setup might convince more people.
Then again, anyone who buys $8000 speaker cables can safely be considered a questionable listener...
Thanks for an illuminating article!
RESOURCE CENTER
THIS MONTH's ISSUE 
FEATURE Windows Tips: Find out what works and what doesn't as we test the most commonly prescribed Windows tipsHOW TO Customize and streamline your Windows desktop Core i7 Check out Intel's next-gen chip, up close and personal The Reactor We preview the first production-ready oil-immersed PC
Technology News
Computer Cooling Fans
Computer Cases
PC Game Controllers
PC Games
Computer Hardware
Headphones
MP3 Players
Stream Video
Computer Mouse
Monitors
Motherboards
NAS Storage
Networking
Laptop Computers
DVD Burner
Digital Cameras
Portable Storage
Computer Accessories
Smartphone
Antivirus Software
Sound Cards
Speakers
Computer Systems
Thumb Drives
Video Cameras
Video Card Reviews
Water Cooling
Gadgets
- Keyboards
© 2008 Future US, Inc. All Rights Reserved.