Comments from test takers

Last update: March, 15, 2015

Thank you very much for posting this test in the internet, it was revealing. Now I know I can’t tell the difference between CD vs. AAC 256k and I don’t need to large amounts of space in lossless music.

Spain

I would like to think I heard a difference but not really sure…

I was listening to Lars Danielsson and got the answer key from the first song.

ok here is my third submittal… I guess I failed miserably !

Ron,

New York, USA

Good site to test if you can tell the difference.

Hu,
New York, USA

I could not confidently discern any differences listening to track 4 (Adele) with ATH-M50x headphones connected to a MacBook Air. It was tiresome listening to the same section 16 times,  so I stopped halfway through. Then I was disappointed to not see the answers immediately, though I assume my guesses were no better than random. So, if the WAV files truly contained AAC and WAV audio in each clip, then I can say that at least listening to this Adele song through my headphones and computer, hi-res audio isn’t necessary.

Steve,
Houston, USA

At first I thought I could maybe tell a difference but towards the end I don’t think I could and guessed most of the last few.

Elliot,
Dublin, Ireland

Well done ABX test.

Evan,
Centralia, Washington

Great test!

Frank,
Frankfurt, Germany

This is a great test, and I like that you’re collecting data as well.
I appreciate your efforts!

Someone from Toronto, Canada

Had a difficult time with clip #1 but think this one (#2) was much easier to tell the difference.

Ricardo,
New York, USA

I still think it’s all the same thing and it’s a massive troll.
😀

Victor,
Canoas, Brazil

Great way of doing the test.  Harder than I thought.  Guess I’ll find out soon if I was able to tell the difference.

Ricardo,

New York, USA

Love this.   Really excited to get the result!

Stephen,

L.A., USA

This was more difficult than anticipated. I possibly chose quite a bad song i.t.o. production quality (Set Fire To The Rain),  but apart from that I got the impression that the lossless version was not as good as my own. Could it be that the lossless version’s digital volume gain could have been adjusted (downwards) to get the volume levels matched? If so, it would be detrimental to the audio quality.

JvR,
Parys, South Africa

No, no adjustments of volume were made to the lossless version of any of the clips.

Very interesting, if somewhat time-consuming to run through all the tests.

Eric,
South Africa

This test is way too flippin’ long for your average joe, half would do just fine.

Andreas,
Denmark

It’s not easy to hear the differences between the best lossy formats and lossless audio. For some songs it’s much easier to hear than for others.
Nowadays, storage space shouldn’t be much of an issue for most people so my advice to anyone would still be to use lossless audio. Even if the difference is minimal, you would know you can’t get better unless a better, remastered version of the song is made.

Nico,
The Netherlands

I couldn’t identify the difference.

Lucas,
Sao Pablo, Brasil

It’s really nice and useful of you to give such an opportunity to test one’s hearing and I’m very thankful to you.

Vanessa,
Bologna, Italia

Very interesting test, I like competitions :).

Magnus,

Stockholm, Sweden

Thanks a million for posting this!

Adam,
Ankeny, USA

You are welcome, Adam!

It’s pretty hard to differentiate the first few times but it gets easier the more you listen. Or not haha I have no idea what the results are going to be yet, but I think I did pretty well.

Bram,
The Netherlands

I could not pinpoint any objective difference, just impressions.

PhilDahAgony,
Italy

It really is very hard to tell. Thanks for using the same sample.

Chris,
Phoenix, USA

I have no idea if I did well. The test itself was a little long but I understand that you are trying to get statistically significant results.

(Anonymous), USA

Really hard to hear differences!

Bojan,
Vienna, Austria

Most of the song sounded the same, but a few bits with large dynamic range had a difference.

James,
Waterloo, Canada

Great test! Keep it up!

Edward,
Arkansas, USA

This is a very difficult task for me, despite being a self proclaimed audiophile. Makes me feel a lot less confident about my ears.

Fergus,
Bristol, UK

In the age of instant gratification, it would be nice to have a smoother process with an easily available result.
I think this is an interesting topic that many don’t take into consideration, and it would be better to appease the gnat like attention spans (such as myself) to better inform the masses.

Sam,
Nelson, BC

Pretty good little test. Sometimes, it took some thinking and replaying, but i’m proud in the clarity that I can hear from my 5.1 system which I use for making dubstep/glitch hop.

Chris,
Raleigh, USA

In most comparisons I was reasonably convinced there were audible differences that affected precision of 3D imaging (“more like being there”) and sonic nuances (“more musicality”).

Mark,
NYC, USA

I enjoyed the test and the fact that it had varying types of music, but all very nicely mastered.

Allan,
Little Rock, AR, USA

This is a great idea. I’ve been following Ken Rockwell for a while and he’s said 192kbps VBR AAC is as good as original CD’s to his professional ear. I’ve tried a few comparisons myself, but never a blind comparison such as this. I could never be certain of any difference I was hearing at 192kbps VBR, and now with this blind test you put together I know I’m not hearing any differences at 256kbps AAC.

Allan,
Portland, OR, USA

Thanks for helping to answer this question. An extremely useful app (no need to download ABX comparators or plugins and create different quality samples), to satisfy curiosity or arguments!

Akshay,
Palo Alto, CA

Interesting and frustrating at the same time. Differences [were] not noticeable enough to really matter. Look forward to the results.

Alexion,
Margate, UK

I was confident at first, but not so sure now !

Gil,
Seoul, South Korea

It’s incredibly more difficult than I thought. I had tried replicated blind tests by myself using a downloaded compressed song and my cd version and the difference seemed completely clear at the time but this time it seemed very difficult to tell the difference, which leads me to believe that the bad quality I have heard before in compressed music was possibly due to the bad quality of the particular compressing case and not the potential quality of that compressing method in general. 🙂

Vitor,
Lisboa, Portugal

Great online test! Was looking for such one.

Andriy,
Lviv, Ukraine

I noticed this test seemed fairly easy initially with fresh ears.  I did not have a hard time distinguishing the first 5-6 tests, but i noticed that listening to the same track over and over became more difficult over time.  By the end of the 8th question, I was having a very hard time telling the difference.  (I’m also not terribly patient).

Chris,
Valdez, Alaska, USA

I love bass so I chose the audios in which it appears more (or at least I think I chose them), but I don’t really know if this is the best since I don’t have any musical training.

Fabiana,
Sao Paulo, Brazil

I feel a greater variety would have been nice, it got a bit tiring. It was, however, understandable. Thank you for the effort.

Donovan,
Dubai, United Arab Emirates

Thanks, was fun trying.

Mattias,
Stockholm, Sweden

I wish I could test this on my speakers on my stereo as the computer is not the best way to determine this.

?,
New York, USA

There are several solutions for sending the raw digital audio data from your computer to a Hi-Fi or high-end system.

I bit to long for impatient guys 🙂

Kári,
Reykjavík, Iceland

Good test. Need to try again at home with my good equipment.

Doug,
USA

The differences are very subtle. I was fairly certain I could hear depth in the brass section where some horns were very slightly off key, compared to a flatter sound that seemed to lose that detail, and the cymbals seemed fuller also compared to the flatter version. My guesses attributed the depth feeling to the CD. Even if that logic was correct, I’m expecting my results to not be better than 55-60%. The test seemed long. By halfway through I was tempted to check the ‘can’t hear a difference’ box option just to get the results for the first eight. But I thought I could hear a difference, so for science, I kept going. Would’ve been happy to do eight of each clip instead of 16 of one. I’m looking forward to the results. My motivation is somewhat pragmatic. I’m a perfectionist. If I get a totally random result, I I’ll have a perfectly good reason not to keep all my music in FLAC.

Joseph,
Lancaster, PA, USA

I liked the test. I think it was well done. I understand the theory behind using the same clip over and over, but I think a variety of clips might have helped fully explore the differences present (or not present).

Dan,
Ames, USA

I couldn’t really tell a significant perceptible difference.  I may have noticed a difference in the crispness and brightness of the string sections and the ‘shimmer’ of the cymbals, but I don’t know if that’s just my imagination; I’m looking forward to seeing the results.

Rebecca,
Columbus, OH

Fantastic selection of audio clips, some of the artists now will belong to my collection. The higher quality sound just screams with it’s clarity.

Vlad,
Sacramento, USA

I frankly don’t believe I have scored better than random… Although I had the impression to discern some subtle differences, I suspect it was mostly not real. We will see the result…

Luca,
Bassano del Grappa, Italy

I think CD quality has fuller and spacier sound and treble is cleaner. CD quality has more “life”.

Jyrki,
Helsinki, Finland

I have been on both sides of the argument of lossless vs lossy formatting and have wanted to put the issue to rest for some time.  Thanks to this easy test, I can honestly say that I don’t believe there is enough of a difference to matter that much.  Will keep examining the differences as I convert to lossy formats (ogg, ogv, etc).  Looking forward to the end results of this test!

Nile,
Tooele (/tuːˈɛlə/), Utah, USA

Well I did this on a whim, and my “sound system” is a 40 inch flatscreen, so take that for what it’s worth. Pretty much fun, though I don’t hold much hope in being able to distinguish one from the other. LOL

Steve,
11 miles west of the Brooklyn Bridge.

I thought the test was really well done. It was easy to follow the instructions and the procedure worked flawlessly. I have been a pianist/guitarist/musician my whole life and was a professional piano tuner for 15 years. I will be very interested to see my results because I believe I could hear a difference in a fairly predictable way. Of course reality is always preferable to me, so if it was just some sort of projection on my part it will be fascinating and extremely instructive. Thanks so much for taking the time and effort to design and execute such an informative experiment. On a side note, what made the test particularly engaging was clip# 2 which I thoroughly enjoyed (enough to be still entertained by it after more than 32 times). I would very much like to know the name of the piece and the artist. Was it by any chance E.S.T (the Esbjörn Svensson Trio)? Again, thanks so much.

Rai Singh Khalsa,
Knoxville,TN – USA

I was trying to decide what format to rip my cd’s to an external hard drive. I have been researching for months what would be the best format to use and there are so many opinions out there. After taking your test and not really able to hear much difference, I realize I am wasting too much time trying to do what’s “right” when I could just be listening to the 100’s of good CDs I have. Thanks!

Heather
Columbus, GA – USA

I’m amazed people still say that compression is bad, when I know at high rates it’s really hard to tell, as it was in this test.

Tim,
London

Good test, I just can’t tell the difference even with good headphones.

Jordan,
Athens, USA

Impressive! It was difficult to tell the difference, and I have no idea how I did.

Noa,
San Francisco, CA

Thank you for providing the test. Hope you are wonderful 🙂 Greetings from Denmark.

Michael
Rødekro, Denmark

I have tinnitus, but I know I can normally hear when a recording has bad audio. I think your clips could make an interesting psychology test, since clips that start with a low tune, and end with a high tune, will be more likely to make the person feel that the second set is better, because it makes them feel better. High to low will do the opposite. A person really needs to hear a complete song, and it needs to be one that they are familiar with.

Jess
Phoenix, USA

I actually did this test with high-end personal audio equipment before and I found this setup (just MacBook Pro and cheap $5 earbuds) to be more revealing of the differences. Interesting. I tended to pick the 256 AAC version as being “better-sounding” to me because the treble response of these earbuds must be really weird or something; the CD-quality tracks often sounded a bit harsher to me.

Michael
Seatle, USA

Thanks for your effort! Always wanted to do something like this. I’m more the Rock n Roll type, but I think I can sense if something’s off. Very tough to say with this encoding format. Absolutely sufficient for my uses. Says the guy who is in the process of re-buying his favorite music in 96/24…one day, there will be a portable player (looking forward to PONO) and a quiet 3 minutes on the bus to work so I can FINALLY hear my favorite song and go: “wow, it sounds so much better now!”. But seriously, I’m a firm believer that it will make a difference, even to people like me. A year ago, my brother thought 160 kbps mp3s are okay sounding. I converted him. Maybe it will happen to me, too. Thanks again for setting this up!

Jakob
Zürich, Switzerland

Gruezi, Jakob! Getting feedback like yours is very rewarding. By the way, maybe PONO will indeed be a great portable player, but my guess is that it will still be very hard (if not impossible for most of us) to tell the difference between HD audio, lossless and AAC 256 kbps. We’ll see!

Useful and convenient 🙂

Miro
Piešťany, Slovakia

I sense that the PC sounds a little better than the phone. Might be imagination. Anyway I wanted to redo the test using the Asus. Still it was very hard for me to tell which one was CD, both parts sounded very good. Not sure if the order is randomized for each new test or if it’s the same when I do it this second time. Anyway I tried to not care about the previous result, I just gave it my best guess.

Sven-Olav
Stockholm, Sweden

Thank you for repeating, Sven-Olav. This will add statistical significance to your trials.  Unfortunately,  I haven’t been able to randomize the trials for the same clip. Different clips have different randomized sequences, though.

It was very hard for me to decide, aac at high speed [bit rate] seems very good. My hearing is good enough to let me enjoy listening to music with my bluetooth headset through the conventional wire rather than bluetooth – aptx. Still, if I got better than a random result here this would be surprising and inspiring. In any case I think I’ll try again soon with one more sample. Many thanks for making this available.

Sven-Olav
Stockholm, Sweden

Thank you for this test and showing me that I don’t have to worry about converting my music into AAC 256kb/s!

Janko, Germany

Very good test

Phil
Salisbury, Australia

You should be able to switch back and forth on the fly between the two different formats for a truer blind test.

Chad
Kalamazoo, MI

That would be ideal, Chad, I totally agree. But I am not a developer, and this is as far as I can get. You won’t find any online test (so far) that meets your requirement.

Got bored pretty fast. Please reduce either the number of samples to distinguish or the length of tie individual parts. You could make less tests but with two different songs as well.

G.
Hamburg, Germany

A minimum number of trials is needed for statistical significance. Sorry you got bored 😦

I’ll probably use the test in my classes on critical listening, if you don’t mind 🙂

Perfecto
Barcelona, Spain

Hard to hear a difference! A and B sections sounded essentially similar.

Marius
Oslo, Norway

So hard! Listening for transparency, depth… Many multiple listens. iPhone 4S with ear buds on a quiet night.

Aaron
St. Louis, Missouri

Very nice test. Would like a bigger bar for adjusting position, as it was very small.

Chris
Oslo, Norway

Yes, I would like a longer bar myself. Unfortunately, I don’t know how to tweak the browser’s built-in player to accomplish it.

liked it

Pete
Melbourne, Australia

That economy of words gives extra meaning to your comment! Thanks for not leaving the field blank!

I did not do the test.  I wanted to get to this screen to say I do not use Apple Quick Time software which is required for this listening test.  I would guess there may be some bias toward iTunes?

Gary, USA

Gary, in order to take my test, your browser must be able to play uncompressed .wav files. Otherwise the test would not make any sense! I have chosen .wav because they stream seamlessly in all my browsers (Chrome, Safari, Firefox and Opera). I don’t know if they require having QuickTime installed in your system or you can play them directly from your browser using another plug-in (you should, I guess). I tried FLAC and ALAC, because they take up less space in the server, but they were much harder to deal with (FLAC would not stream in any of the browsers and ALAC necessarily required QuickTime and played  only in Safari and Opera). I use iTunes to manage my media and I compressed all clips in the test using iTunes encoder. I give this information so that test takers know exactly what they are comparing, because different encoders indeed lead to (slightly) different results in the outcome.

It’s really hard to tell the difference between the two clips. In many cases I was confused and not sure of my guess. Most probably I did a lot of mistakes. Having this said, I’m not really sure that the quality iTunes provides is really comparable to what you get on a regular, good quality CD. For instance, I noticed some differences on Bruce Springsteen’s “Wrecking Ball” album. Anyway, very good test!

Tierry
Milan, Italy

Honestamente no distingo la diferencia. Mis respuestas se basan en un dato no musical: el tiempo de descarga de cada ejemplo. (Esto es impropio de mí pero alerta de que la prueba no es del todo ‘blind’.)

Pedro
Barcelona

Pedro, el tiempo de descarga puede tener fluctuaciones aleatorias. Las muestras tienen todas exactamente el mismo tamaño: las comprimidas se reconvierten al formato original sin comprimir para “camuflarlas” correctamente.

Very interesting test. I suspect that unless you listen to the audio through a high quality system you’re not going to be able to hear a difference.

Keith
Japan

Awesome test!

Hans
The Netherlands

1. Thanks for doing this, super interesting! (and humbling, I fear!)

2. This is my second attempt. This time I used my home studio monitors instead of my headphones.

Stephan
Montréal, Canada

Good blind test. A variety of genres would be preferred, but that’s OK.

Maigonis
Liepāja, Latvia

Very interesting… curious to the see the results, would think I did ok, but will probably be shocked.

Joe
in the jungle in Thailand

I have some of the best headphones in the world.  Also, two or three tracks wouldn’t play.  However, I do not believe I could reliably distinguish much if any of the differences.

Brandon
Bethlehem, PA

Very cool test, well executed. I can say it was difficult!  I am almost always confident to recognize wavs from mp3 on the productions I do on my system, main things I notice are a very subtle plastic thing around songs, little loss of transients, details and space. I had never done this kind of tests before, it’s difficult to do all songs that you don’t know at once because I kind of get confused on what to hear for, but i think that’s the trick of uncovering the encoder sound.  Thank you!

Ignacio
Argentina

Pretty tough, I grew less confident as it went on. In any event, both are quite “listenable” formats. At first I felt I could differentiate the two more, and that one was ever-so-slightly more present but in a way that actually could be subtle distortion. By the last example, I wasn’t so sure.

Liam
Petaluma, USA

Entré en conflicto con la respuesta porque a mí “me gusta más” B (AB clip #5). Creo que es la mejor calidad porque me parece más limpio el sonido, más liviano, menos saturado. Si es verdad que la calidad Itunes pierde detalles quizá sea eso lo que a mí más me gusta aunque se escuchen menos detalles… Para no confundir, yo me decido por B como lo mejor, ya sea CD o AAC. De todas maneras si no me hubieran instado a notar una diferencia, desde luego que no la hubiera notado.

Nieves
Heidelberg, Germany

I am not a valid participant because I already heard about the tentative results. Congratulations for the design and implementation of the test.

Xavier S.
Barcelona, Spain

Can’t we have a third option response for where we can state that the two samples are indistinguishable? For 11 tracks I found the first and second samples indistinguishable. I just marked them all as “A” then did not mark the “!,” which I did on the ones I could hear the differences in. Others might have forced themselves to “hear” a difference, or used pure guesswork.[…]

Ezra
Manila, Philippines

(Now you  can, in tests #1 and  #2)

This one was much harder to differentiate than sample 6. Honestly all the other samples I do not think I could tell them apart. They are too narrow and lack the sharp percussive elements that are required in the sampled portion. I’m also not as confident on my ability to discriminate on this sample vs the previous one. There are very short sections which I know are different, in each version, but I’ve been at this for almost 2 hours. So I am just going to stop here.

T.Chon (highest score so far)
Limerick, PA, USA

En realidad, no pude notar la diferencia!

Diego
Folsom CA, USA

If I am right on most of these I have to say that there is a substantial noticeable difference on most tracks. There were however towards the end a couple almost indistinguishable tunes with only minute differences. So source material may respond differently to the compression process. Also the main difference is in bass extension/ fullness of sound and clarity detail in brass and vocals.

Josef
Bradford, England

Great test!

Bob
Los Angeles, USA

Both samples seemed high quality.  On some I felt like I could hear a different level of compression similar to how a master/limiter plugin would be in an audio plugin.  It provided a different sound where a lot of the A versions seemed a bit fuller.

Nathan
California, USA

Nice test. Needs a lot of patience, but effective. Thanks!

Jordan
Beijing, China

‹ AAC vs mp3 – A comparison through null testing

Tips to build and manage your digital music collection (I) ›

Posted in Uncategorized