GSMArena feature lab: Enter new goodies

Audio quality test

Until recently, sound quality used to be a subjective matter in our reviews. Realizing that mobile phones are more than ever used as portable music players, subjective just won't do any more. That is why we at GSMArena will be including a new audio tests in our reviewing routine - the audio quality test.

To give you a more objective view of how the music players perform in all those handsets we review, we decided to record audio samples of each device, analyze the results and publish those for your geeky pleasure.

It must be pointed out that we are not testing the retail headsets that come with the various handsets - our audio quality test concerns only the quality of the music player output. A mediocre set of headphones will obviously make things worse than they are in our tests.

In order to record the samples needed for analysis we use the M-Audio Fast Track Pro external audio interface, which has decent enough capabilities to allow testing even the most high-quality portable music players. You can find its quick Line Input specs here.

M-Audio Fast Track Pro external audio interface

Having the right hardware, however, is not enough to do the job. We chose the RightMark Audio Analyzer (RMAA) software to do all the analyzing. First off, we use a calibration sound (generated by RMAA) in order to set the input levels right each time. Then we play a special test tone (again, generated by RMAA) which allows the software to evaluate the signal captured real time with the M-Audio Fast Track Pro.

All this gives us the opportunity to create a controlled environment, which should produce accurate and comparable results throughout all the units tested. With all the handsets, we test how they perform both at the maximum volume level, as well as several notches down. This is mainly due to the reason that some music players can produce a lot of distortion at the highest volume levels, while they would play perfectly fine at a moderate volume level. It's also worth saying that during the tests we disable all available equalizer settings and sound enhancements, that the handset music players may be equipped with.

The RMAA software produces results on several important aspects of sound quality. Usually, the high-quality sound reproduction should demonstrate minimal or unnoticeable levels of noise and distortion, and an accurate frequency response.

The frequency response curves seen on our RMAA generated graphs are often used to indicate the accuracy of amplifiers and speakers in reproducing audio. Good frequency response does not guarantee a specific fidelity, but only indicates that a piece of equipment meets the basic frequency response requirements. As an example, a high fidelity amplifier may be said to have a frequency response of 20 Hz - 20,000 Hz ±1 dB. This means that the system amplifies all bands from 20 Hz to 20,000 Hz with a maximum positive or negative deviation from the straight line of only 1 dB. Larger deviations below 40 Hz or above 15 kHz are OK, since the human ear is unable to perceive those sounds. Here lower absolute numbers mean the handset treats all sounds equally; it won't impose its will on the music but will allow you to hear the music as it was recorded, which is what a good piece of audio equipment should do.

The noise levels are the next important aspect of sound reproduction. As seen in our tests, noise is usually measured in relation to the signal levels, otherwise known as the signal-to-noise ratio. In less technical terms the signal-to-noise ratio compares the level of a desired signal (such as music) to the level of background noise ('background' meaning unwanted interferences created by the equipment). So the higher the signal-to-noise ratio, the less obtrusive the background noise, hence the reproducing equipment is of a higher quality.

The dynamic range is another important factor. The human sense of hearing has a very high dynamic range. A person is capable of hearing anything from a quiet murmur in a soundproofed room to the sound of the loudest rock concert. A difference like this can be up to 100dB. A person however cannot perform these feats of perception at both extremes of the scale simultaneously - you cannot hear a whisper in a noisy street for example. Nevertheless, a good quality audio reproduction system should be able to reproduce accurately both the quiet sounds and the loud at the same time. Audio engineers often use the term dynamic range to describe the ratio of the loudest possible undistorted sound to the quietest or to the noise level, say of a microphone or loudspeaker. This dynamic range of an audio device is also sometimes referred to as the dynamic window. To mathematically determine a dynamic range you must take the difference between the ceiling and the noise floor of an audio device. For example, if the ceiling of a device is 10 dB and the floor is 3 dB then the dynamic range is 7 dB, since 10 - 3 = 7. So the larger the dynamic range (or otherwise the dynamic window), the better the device performs at reproducing quiet and loud sounds simultaneously.

The total harmonic distortion (THD) and the intermodulation distortion (IMD) might be the next items you will notice on our measurement results. By the name you know they are a measure of the unwanted altering of the original signals. Distortion is the name given to anything that alters a pure input signal in any way other than changing its magnitude. So, as you might have guessed it, the lower the number, the better the sound reproduction.

The stereo crosstalk is another thing of importance that we measure in our tests. Crosstalk measurement is made to determine the amount of signal leaking across from one channel to another or in purely non-technical terms it measures how good the stereo is. So, the higher the absolute value, the better the stereo.

We are unable to post results retroactively for the mobile handsets that have already been reviewed at GSMarena.com but we will do our best to collect sound analysis data for all the handsets we can get our hands on from now on.

In the meantime, you can have a look at the data collected from handsets that were available at the time we made the tests. It is presented in a tabular form but you will find that there are links to frequency response and noise graphs. You might like to browse and compare them too.

Update 20 Sep 2010: Recently we decided to add some additional testing in our usual audio quality testing routine. The numbers we've published so far reflect the phone performance when plugged into an active amplifier (i.e. your car stereo or your home audio system). Since recently, we started testing each phone two times. The second try is with headphones attached. As it turns out in many cases the added resistance of the headphones makes the audio output quality worse.

We use a pair of AKG headphones with impedance of 32 ohms for each of those special second runs.

If you pay closer attention, you might notice a suspicious resemblance between the results produced by the Samsung handsets and the iPod unit we were testing. We were caught off guard too and we even thought we had some kind of a problem with our test setup. But after we checked on the matter, it turned out that Samsung are in fact supplying Apple with audio processors.