GSMArena feature labs: The tests

Update Mar 4, 2020: This review test has been discontinued. You can find the full description of the test bellow for reference purposes but it's no longer carried out on new devices.

Audio Quality

The series of audio quality tests we perform measure how well a device is able to reproduce sound, based on a scientific approach rather than our own subjective opinion. Keep in mind that these tests do not take into account any retail headsets that come with the device. We use the same pair of AKG headphones when we test the output with headphones attached to the phone to even up the field.

First, we put special test tracks on the device and play them through the audio output jack to our M-Audio Fast Track Pro external audio interface. This rig has has decent enough capabilities to test even the highest quality portable music players. You can find its quick Line Input specs here, if you are the audiophile type.

Next we run the resulting recordings through the RightMark Audio Analyzer (RMAA) software, which does all the analyzing for us..

At the beginning of every test, RMAA generates a calibration sound in order to set the correct input levels. Then we play a special test tone (again, generated by RMAA) which allows the software to evaluate the signal captured real time with the M-Audio Fast Track Pro.

We test all handsets we review on how they perform at the maximum volume level. We disable all available equalizer settings and sound enhancements that the handset music players may be equipped with. All this gives us the opportunity to create a controlled environment, which should produce accurate and comparable results throughout the tested phones and tablets.

The RMAA software produces results on several important measurable areas of sound quality. These are frequency response, noise (signal-to-noise ratio), dynamic range, harmonic and intermodulation distortion and stereo crosstalk. Typically, high-quality sound should have high signal to noise, low distortion and cross-channel interference, an accurate frequency response, as well as high dynamic range.

Frequency response is used to indicate the accuracy of amplifiers and speakers in reproducing audio, and is a measurement of how much sound deviates (in decibels dB) at a given frequency (in Hertz Hz). As an example, a high fidelity amplifier may have a frequency response of 20 Hz - 20,000 Hz ±1 dB. This means that the system amplifies all bands from 20 Hz to 20,000 Hz with a maximum positive or negative deviation of 1 dB.

It is this deviation that we look at when evaluating frequency response. In our tables, you will see two numbers listed under frequency response--the highest positive and negative deviations of the sound produced across all audible frequencies. The closer each of these numbers is to 0, the better the frequency response of the device. Lower absolute numbers also mean the handset treats all sounds equally; it won't impose its will on the music but will allow you to hear it as it was recorded, which is what a good piece of audio equipment should do.

We also include a graph which shows exactly what sort of sound deviations occur at the various frequencies.

As shown in the graph, the frequency response tests are performed twice. The first time when plugged into an active amplifier--the same as what you'd expect when plugging it into your car stereo or home audio system--while the second run is done with headphones attached, as the added resistance oftentimes makes the audio quality output slightly worse. As mentioned above, we use the same pair of AKG headphones with an impedance of 32 ohms for each of these second runs.

Good frequency response does not necessarily guarantee a high level of audio fidelity, but rather how well a piece of equipment meets the basic frequency response requirements. It's important to note that larger deviations below 40 Hz or above 15 kHz should not be counted, being beyond the audible rage of the human ear.

Noise levels of a given device are usually measured in relation to the signal levels, otherwise known as the signal-to-noise ratio. In less technical terms, the signal-to-noise ratio compares the level of a desired signal (such as music) to the level of background noise ('background' meaning unwanted interference created by the equipment). So, the higher the signal-to-noise ratio (the further this value is away from 0), the less obtrusive the background noise is, meaning the audio equipment is of a higher quality.

Dynamic range is the measure of how well a given device is able to play both loud and quiet sounds simultaneously. The human sense of hearing has a very high dynamic range; a person is capable of hearing anything from a quiet murmur in a soundproofed room to the sound of the loudest rock concert. A difference like this can be up to 100dB. However, a person cannot perform these feats of perception at both extremes of the scale simultaneously - you cannot hear a whisper in a noisy street, for example.

Nevertheless, a good quality audio reproduction system should be able to reproduce accurately both the quiet and loud sounds at the same time. This dynamic range of an audio device is also sometimes referred to as the dynamic window. To mathematically determine a dynamic range you must take the difference between the ceiling (the loudest sound) and the noise floor (the quietest sound) of an audio device. For example, if the ceiling of a device is 10 dB and the floor is 3 dB, then the dynamic range is 7 dB, since 10 - 3 = 7. So, the higher the value in the dynamic range field, the better the device is at reproducing quiet and loud sounds simultaneously.

Total harmonic distortion (THD) and the intermodulation distortion (IMD) measure the unwanted altering of the original audio signal. When an audio signal passes through a non-ideal, non-linear device (such as your smartphone), additional content is added to the original frequencies in the form of interference, which results in distortion. Distortion is the name given to anything that alters a pure input signal in any way. So, as you might have guessed, the lower the number in either of these fields, the better the audio output of your device.

Stereo crosstalk is a measurement of the interference between the two audio channels of a given device. All modern digital audio players feature two distinct analogue channels for audio (left and right) to match the stereo recordings, but most often a certain degree of inter-channel interference can occur. The stereo crosstalk measurement determines the amount of signal leaking from one channel to the other. In purely non-technical terms, this rating measures how good the stereo effect is. The larger this value is (the further away from zero), the better the quality of the sound coming from each channel.