AAC vs mp3 – A comparison through null testing

As you may already know, AAC and mp3 are compression algorithms that manage to significantly reduce the size of music files by discarding part of the signal based on psychoacoustics, that is, based on what our ears are really able to perceive. The basic principle is that soft, quiet sounds are easily masked by louder ones. Just think of the difference in loudness the ring of a phone can have when heard at home at night, or during the day, in a noisy environment. The ticking of a clock would be an extreme example of the same phenomenon. These algorithms scan the wave and decide which features can be “safely” removed without (significantly) degrading the audio quality of the original. In my opinion, they are very clever, and do an outstanding job. Of course, the quality of the output will depend on the criteria through which these decisions are made, so different algorithms will lead to different results. In what follows, I will use null testing to analyze the discarded data when using one format or the other.

It is said that AAC (Advanced Audio Coding, 1997) offers better results than mp3 (1995) at equal bit rates. My impression is that this is true, at least at 128 kbps. Let’s see what the null tests say. All the encoding is done with iTunes built-in encoder.

Note: Please let me know in the comments if you experience problems playing the sound samples.

1. AAC 256k VBR vs mp3 256k VBR

Original CD clip:

13 All Names - clip.m4a


 

AAC 256k VBR version:

13 All Names (clip) AAC 256k VBR.m4a


 

mp3 256k VBR version:

13 All Names (clip) 256k VBR.mp3


 

Now let us listen to the discarded audio signal.

AAC 256k VBR nulled track:

13 All names AAC 256_mezcla.wav


 

mp3 256k VBR nulled track:

13 All names mp3 256_mezcla.wav

For this clip. the spectrograms show that AAC discards more data in the lower part of the spectrum and less in the high frequencies. The spectrogram for the mp3 nulled track is much more uniform along the whole range of frequencies.

Listening to these nulled tracks,  you may be surprised at how much audible signal is removed from the original. It really sounds like a lot to be sacrificed. Notice, however, how quiet these nulled tracks are, compared to the original. All this audio signal is chosen to be discarded because it is almost perfectly masked by the louder part. If you find this hard to believe,  you should try my blind test! You’ll see how difficult it is to tell de difference between the original and the compressed tracks at this bit rate. (Whether you find it hard to believe or not, please, try my blind test anyway!)

2. ACC 128k CBR vs mp3 128k CBR

128k bit rate still offers good quality audio, but differencies can be easily detected by a trained ear. Let’s see what the null tests say in this case. We will use the same original CD audio clip, which I am not including again at this point.

AAC 128k CBR version:

13 All Names (clip) 128.m4a


 

mp3 128k CBR:

13 All Names (clip) 128.mp3 (I don’t know why my spectrum generator shows a time scale of 1 min in this case. I couldn’t fix it, but the bug is not relevant).

Notice that, in the AAC 128k version, the algorithm starts cutting at high frequencies above 16 kHz, but there is a lot a data kept up to 18 kHz. In the mp3 version, the cut at 16 kHz is much clearer, but there are some bits of data that go up to 20 kHz. This clearly shows that the algorithms are indeed making different decisions on what to discard.

The discarded audio signal sounds as follows:

AAC 128k CBR nulled track:

13 All Names AAC 128_mezcla.wav


 

mp3 128k CBR nulled track:

13 All Names mp3 128_mezcla.wav

I guess you can clearly hear that the discarded audio is much louder now. I also perceive that the cut in low frequencies is much noticeable in the mp3 version. In my opinion, AAC clearly beats mp3 at this bit rate.

Finally, tastes and subjective perceptions aside, my hope is that you realize how useful null testing is for proving (or disproving) claims in the audio world.

Advertisements
Tagged with: , , , , , , , ,
Posted in Uncategorized
8 comments on “AAC vs mp3 – A comparison through null testing
  1. h0rst says:

    Hi,
    i agree with the results of your test, did similar myself. But there is a few things to add:
    While the aac encoder in iTunes isn’t actually the best (there has been a comparison on a german website a few years ago where the Nero codec got much better results) the mp3 encoder in iTunes is in fact totally outdated. Apple is using code from the Ninetees here!
    You should also mention that aac encoding is generally using a vbr routine which is always more efficient than a fixed one like you used on mp3.
    The LAME vbr quality is excellent but the encoder is not supported by iTunes (at least the Windows version, when i switched from Apple back to MS there was no way to integrate it).
    To me the VBR files, while smaller, sound more transparent than 320k CBR mp3 but in my blind tests the ears got tired without before getting a clear winner and i tend to hear artifacts in what later comes out as the uncompressed original WAV.
    However, with EAC there is a free and much better CD-Ripper than iTunes.
    with best regards, h0rst… computer audio enthusiast for more than 15 years now.

    • cdvsmp3 says:

      Thank you very much for your insight and information. I have always trusted iTunes as a good choice for mac users, especially for AAC, but there should be better mp3 encoders out there, as you point out in your comment. Regards. (Sorry for not answering earlier, I was on vacation)

  2. I think that 320 kbps (constant mp3 -ENCODED WITH LAME) quality, is much more faithful to the original cd than any AAC.

    • Brian Beltré says:

      Never, AAC has so many profiles, Wikipedia said you can step down two times and the song can be comparable with the MP3. Any LAME encoder can beat a good AAC encoder, such as Nero, FAAC, or QuickTime at the same bitrates, either VBR or CBR.

  3. Shay says:

    These null tests don’t take into account things like phase differences, e.g. a 100% copy of the original signal time-shifted will result in a noticeable difference signal even though it’s the same.

    • cdvsmp3 says:

      The whole point of this site is to give some insight into what ‘noticeable’ actually means in this context. When tested properly through blind testing, those who claim they can hear these ‘noticeable differences’ just fail to prove it.

  4. I adore this website – its so usefull and helpfull.|

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: