Since we all know you are in desparate need of more information on how and
why DBT's are used by Sean Olive and Floyd Toole I include the following for
your edification.

Audio - Science in the Service of Art
by
Floyd E. Toole, Ph.D.
Vice President Engineering, Harman International
Senior Vice President Acoustical Engineering, JBL and Infinity
INTRODUCTION
Audio products must sound good. That is a given. However, the determination
of what constitutes "good
sound" is a matter that has been controversial. Some assert that it is a
matter of personal taste, that our opinions
of sound quality are as variable as our tastes in "wine, persons or song".
This would place audio manufacturers
in the category of artists, trying to appeal to a varying public "taste".
Others, like the author, take a more
pragmatic view, namely that artistry is the domain of the instrument makers
and musicians and that it is the role
of audio devices to capture, store and reproduce their art with as much
accuracy as technology allows. The audio
industry then becomes the messenger of the art. Interestingly, though, this
process has created new "artists", the
recording engineers, who are free to editorialize on the impressions of
direction, space, timbre and dynamics of
the original performance, as perceived by listeners through their audio
systems. Other creative opportunities
exist at the point of reproduction, as audiophiles tailor the fundamental
form of the sound field in listening rooms
by selecting loudspeakers of differing timbral signatures and directivities,
and by adjusting the acoustics of the
listening space with furnishings or special acoustical devices.
To design audio products, engineers need technical measurements.
Historically, measurements have been
viewed with varying degrees of trust. However, in recent years, the value of
measurements has increased
dramatically, as we have found better ways to collect data, and as we have
learned how to interpret the data in
ways that relate more directly to what we hear. Measurements inevitably
involve objectives, telling us when we
are successful. Some of these design objectives are very clear, and others
still need better definition. All of them
need to be moderated by what is audible. Imperfections in performance need
not be unmeasurably small, but
they should be inaudible. Achieving this requires knowledge of
psychoacoustics, the relationship between what
we measure and what we hear. This is a work in progress, but considerable
gains have been made.
This paper reviews scientific work performed by the author and his
colleagues, that aimed to determine
the extent to which listeners agreed on their preferences in sound quality
and, beyond that, to identify
relationships between listener preferences and measurable performance
parameters of loudspeakers. Given that
loudspeakers, listeners and rooms form a complex acoustical system, some of
the effort was necessarily devoted
to identifying the aspects of performance that maximize the performance of
the entire system in real world
circumstances.
ART, SCIENCE AND AUDIO
Science has allowed us to do some remarkable things. It has enabled us to
travel safely back and forth to
the moon, to drive around Mars, to harness atomic energy for peace and to
unleash it for war. Its principles
allow engineers to build tall buildings, immense bridges, and vehicles that
roll on the earth and those that fly
through air and space. These and most other aspects of scientific
accomplishment are associated with "things",
or theoretical concepts far removed from our everyday lives. However, that
same foundation of scientific
knowledge, principles, and methodology has given us the ability to enjoy the
excitement, emotions and sheer
beauty of music, whenever or wherever the mood strikes us.

Music, itself, is art, pure and simple. The composers, performers and the
creators of the musical
instruments are artists and craftsmen. Through their skills, we are the
grateful recipients of sounds that can
create and change moods, that can excite and animate us to dance and sing,
and that form an important
component of our memories. Music is part of all of us and of our lives.
However, in spite of its many capabilities, science cannot describe music.
Beyond the crude notes on a
sheet of music, science has no dimensions to measure the evocative elements
of a good tune. It cannot
technically describe why Pavarotti's tenor voice is so revered, or why the
sound of a Stradivarius violin is held as
an example of how it should be done. Nor can science distinguish, by
measurement, the mellifluous qualities of
trumpet intonations by Wynton Marsalis, and those of a music student who
simply hits the notes. Those are
distinctions that must be made subjectively, by listening. Some scientific
effort has gone into musical
instruments and, as a result, we are getting better at imitating the
desirable aspects of superb instruments in less
expensive ones. We are also getting better at electronically synthesizing
the sounds of acoustical instruments.
However, the determination of what is aesthetically pleasing, remains firmly
based in subjectivity and the arts.
Our audio industry is based on a seemingly simple sequence of events. We
capture a musical
performance with microphones, whose outputs are blended into an electronic
message stored on tape or disc,
which is subsequently reproduced through two or more loudspeakers. This
simple description disguises a
process that is enormously complicated. We know from experience that, in
some ways, the process is
remarkably good. For decades we have enjoyed reproduced music of all kinds
with fidelity sufficient to, at
times, bring tears to the eyes, and send chills down the spine. Still,
critics of audio systems can sometimes point
to timbral characteristics that are not natural, that change the sound of
voices and instruments. They point to
noises and distortions that were not in the original sounds, rendering even
the most eloquent tunes in a brittle and
harsh fashion. They note that closing the eyes does not result in a
perception that the listener is involved in the
performance, enveloped in the acoustical ambiance of a concert hall or jazz
club. They point out that stereo, as
we have known it, is an antisocial system - only a single listener can hear
the reproduction as it was created.
For all of these criticisms there are solutions, some here and now, and some
under development. All of
the solutions are based on science.
How can science, a cold and calculating endeavor if ever there were one,
help with delivering the
emotions of great music? Because, in the space between the performers and
the audience, music exists as sound
waves. Sound waves are physical entities, subject to physical laws, amenable
to technical measurement and
description and, in most important ways, predictable. The physical science
of acoustics allows us to understand
the behavior of sound waves as they travel from the musician to the
listener, whether the performance is "live" or
recorded.
To capture those sound waves, with all of the musical nuances intact, we
need transducers to convert the
variations in sound pressure (the sound waves) that impinge on them into
exact electrical analogs. Microphone
design uses the science of electroacoustics a blend of mechanical and
electrical engineering with a dose of
physics. Once the signal is in the electrical domain, the science and
engineering expertise in electronics is
brought to bear in preserving the integrity of the musical signal.
Contamination by noise and distortions of any
kind must be avoided, as it is prepared for storage and when it is recalled
from storage during playback.
Emerging from the tape or disc recorder, radio or TV, the musical signal is
too feeble to be of any
practical use, so we amplify it (more electronics), giving it the power to
drive loudspeakers or headphones (more
electroacoustics) that convert the electrical signal back into sound. Again,
this all must be done without adding
to or subtracting from the signal, or else it will not sound the way it
should.
Finally, the sound waves radiating from the loudspeakers propagate through
the listening room to our
ears. If there is a seriously weak link in this entire process, this is
probably it. Rooms in our homes are different
from any that would have been used for live performances, or for monitoring
the making of a recording. Our
eyes and our ears both recognize that. The acoustical properties of rooms,
large and small, are in the scientific
domain of architectural acoustics, and from that we could predict that there
would be problems of the kind that
we experience. The complex interactions between room boundaries and speaker
directivity at middle and high
frequencies, and speaker and listener position at low frequencies are
powerful influences in what we hear. With

careful acoustical design and electronic signal manipulations, we are
finding ways to make speakers more
"friendly" to rooms, both in recording studios and in homes.
THE STEREO PRESENT AND THE MULTICHANNEL FUTURE
In stereo there are only two loudspeakers to reconstruct an illusion of the
complex three-dimensional
sound field that existed in the original hall, club or studio. The choice of
two channels was based on the
limitations, then, of what could be stored in the groove of an LP. Even at
that time, it was known that two
channels were insufficient to recreate truly convincing illusions of a
three-dimensional acoustical performance.
The old argument that we only have two ears is valid only in the context of
binaural recordings, where it is
necessary to send specific sounds to each ear independently.
The excitement and realism of the reproduction is enhanced if we have more
channels and more
loudspeakers. This is a relatively new development in music recording and,
understandably, it will take some
time for artists and recording engineers to learn how to use the new format.
Examples from the abortive
quadraphonics era, and from some current efforts, show that not everyone has
learned how to merge
multichannel technology with good taste. However, properly used, the
combination of digital discrete
multichannel technology, and the appropriate loudspeakers can provide the
basis for some amazing sound
experiences - both realistic and contrived.
And, what might the appropriate loudspeakers be? Two-channel stereo, as we
have known it, is not a
"system" of recording and reproduction. The only "rule" is that there are
two channels. At the recording end of
the chain, there are many quite different theories and practices of miking
and mixing the live performance -
ranging from the purist simplicity of two coincident microphones to
multi-microphone, multi-track, pan-potted
and electronically-reverberated mono. They all can be great fun, some of
them even very good, but they are all
different.
At the reproduction end of things, loudspeakers have taken many forms:
forward facing, bipolar
(bidirectional in phase), dipolar (bidirectional out-of-phase),
omnidirectional, and a variety of multi-directional
variants. These, and various sum/difference and delay devices, have been
employed in attempts to coax, from a
spatially-deprived medium, a rewarding sense of space and envelopment. In
the two-channel world, therefore,
the artists could not anticipate how their performances would sound in
homes. It was left to the end user to
create something pleasant. Stereo, therefore, is not an encode/decode
system, but a basis for individual
experimentation.
Nowadays we have elegant active-matrix technologies, such as Lexicon's Logic
7, and Citation's 6-Axis
that allow us to play conventional stereo recordings through multichannel
systems, to introduce us to being truly
enveloped in sound. Whether the experience is realistic, or even tasteful,
will depend on how the recording was
made - remember, there are no standards in stereo.
All of this will get much better when recordings are created to be
reproduced through multiple channels.
Then, if we do not like what we hear, we can legitimately complain to the
artist. Among the discoveries in this
new era, may be that some of the loudspeaker designs that were flattering to
stereo recordings, will be less
appropriate for multichannel sound. For example, many listeners have come to
favor loudspeakers having wide
dispersion, or even multi-directional radiating characteristics, for stereo
recordings. The principle at work here is
that the reflected sounds in the listening room embellish the sounds from
the two loudspeakers, pleasantly
enhancing the impressions of acoustical warmth, depth and spaciousness. This
is as it should be.
Multichannel recordings are created with the knowledge that there are
loudspeakers positioned around the
room in locations optimized to create different directional and spatial
illusions. This is a powerful advantage, in
that it permits the artist to create an enormous range of effects: first, in
movies:
.. close-up speech, intimate whispers (listener in a strong direct sound
field),
.. being surrounded by reverberant space or cheering fans at a game or
concert (multidirectional discrete
sounds),

.. fly-overs, drive-bys, ricochets, etc. (multidirectional discrete sounds),
and in musical
performances:
.. being in a superb concert hall or club (direct sounds from an orchestra in
front, with reflected and
uncorrelated sounds from beside and behind), and
.. being in the middle of a band (multidirectional direct sounds).
These will be convincing only if the loudspeakers can deliver the
appropriate sounds to the listeners' ears.
If this "spatial dynamic range", as I call it, is to be achieved in normal -
i.e. not acoustically treated - listening
rooms, we will very likely need loudspeakers of differing directivities in
different locations and, if we are truly
fussy, we will need more than five channels.
We have begun a voyage of discovery and, depending on how we approach it, it
can be long and tortuous,
or short and sweet. If it is approached with an appropriate blend of art and
science, it could be the latter. It
would be wonderful if the music industry could agree on a standard
methodology for multichannel sound.
However, based on experience, that is extremely unlikely.
At this point it is necessary to introduce the science of psychoacoustics,
the study of relationships
between physical sounds and the perceptions that result from them.
Psychoacoustics allows us to understand and
interpret measurements in ways that relate to what we hear. Such knowledge
is absolutely necessary if we are to
make significant progress in designing better products, especially when
price is a concern.
THE SCIENCE OF AUDIO
Individual points of view are a part of human nature. They enrich our lives
in many ways. It would be a
boring world if we all were attracted to the same music, food, wine and
people. However, it is a serious
disadvantage if one is trying to design products that will be attractive to
the majority of the consumer population.
A point of view, commonly expressed, is that sound is "subjective", that we
all "hear differently", and
therefore not all of us prefer the same loudspeakers. It is also alleged
that different nationalities, and regions
have different preferences in sound. I have always regarded these assertions
with suspicion because, if they were
true, it would mean that there would be different pianos for each of these
regions, different trumpets, bassoons
and kettle drums. Vocalists would change how they sang when they were in
Germany, Britain, and the U.S. I
wonder what Pavarotti's Japanese timbre sounds like? I don't believe it . .
.. and it doesn't happen that way.
The entire world enjoys the same musical instruments and voices in live
performance, and the recording
industry sends the same recordings throughout the world. True, from time to
time, there have been regional
influences that have made differences. I can recall the "east coast / west
coast" sounds associated with some
powerful brands in those locations in the USA. In Britain, the British
Broadcasting Corporation (BBC) provided
loudspeaker designs that became the paradigm for a few years. Everywhere,
there are magazines and reviewers
that are influential. All of these factors change with time. In truth, they
are really minor variations on a common
theme. Underlying it all is a powerful attraction to reproduce sounds as
accurately as possible.
Now, what about those individual preferences in sound quality? This
interesting issue was settled by
conducting many, many listening tests, using many, many listeners and many,
many loudspeakers. To reduce the
influences of price, size and style - factors that we absolutely know can
reveal individuality - all of the tests
were conducted blind. Other physical and psychological factors known to be
sources of bias were also well
controlled [refs. 1-3]. The results were surprisingly clear. When the data
were compiled, it turned out that most
people, most of the time, liked and disliked the same loudspeakers. There
were also differences in the way
listeners performed; most were remarkably consistent in repeated evaluations
of the same product, while some
others changed their opinions of the same product at different listening
sessions.
10
ONE STANDARD DEVIATION FOR
9
HIGH FIDELITY AND EXPERIENCED LISTENERS WITH
F
8
STUDIO MONITOR NORMAL HEARING (3% - 5%)
SPEAKERS
I
7

FIGURE 1: Judged on a FIDELITY scale of 0 to 10, terrible to perfect, this
figure shows results of many
listeners evaluating many loudspeakers from different categories. To the
same scale are shown, symbolically,
the variability of repeated judgements by experienced listeners with hearing
levels close to audiometric zero, and
those who exhibited broadband hearing loss. Also shown is the variability of
data combined from several
listeners with normal hearing.
Figure 1 shows, as might be expected, a rather large range in performance of
the loudspeakers
themselves. It also shows that, in repeated judgements, listeners with
normal hearing exhibited standard
deviations small enough for high statistical significance to be associated
with small (about 0.5 scale unit) rating
differences between products. This important observation is
VARIABILITY
JUDGMENT
0 10 20 30
BROADBAND HEARING LOSS (dB)
paralleled by another, perhaps even more important one: that
groups of such listeners closely agreed with each other. The
finding that hearing loss is a factor is not surprising - listeners
who cannot hear all of the sound must make less reliable
judges. What is surprising is that the deterioration in
performance is so rapid. It is well defined within hearing
losses that would not be regarded as alarming by conventional
audiometric criteria, see Figure 2. This difference, one
assumes, is related to the difficulty of the task: judging sound
accuracy, as opposed to understanding the spoken word. The
hearing loss, in this case, was defined by the average threshold
elevation at frequencies below 1kHz. Those exhibiting this
form of hearing loss, also tended to have loss at high
frequencies. High-frequency loss, by itself, was not a clearly
correlated factor. Reference 4 covers this in detail.
FIGURE 2
CHOOSE YOUR LISTENERS WITH CARE
FIDELITY
10
LISTENERS WITH LOW
9
JUDGMENT VARIABILITY /
NORMAL HEARING
8
7
6
LISTENERS WITH HIGH
5
JUDGMENT VARIABILITY /
4
HEARING LOSS
3
2
1
0
FIGURE 3: A
comparison of
loudspeaker
FIDELITY
evaluations
performed by two
groups, one with
low variability in
their judgments
and the other with
high judgment

Listeners with hearing loss not only exhibit high judgment variability, they
can also exhibit strong
individualistic biases in their judgments. This comes as no surprise, since
such individuals are really in search of
a "prosthetic" loudspeaker that somehow compensates for their disability.
Since the disabilities vary
enormously, so do the biases.
The evidence of Figure 3 is that the group of normal-hearing listeners
substantially agree in their ratings.
Interestingly, the second group shares the opinion of the truly good
speakers, A and B. However, speakers C and
D exhibit characteristics that are viewed as problems by the normal group,
but about which the second group has
substantially no opinion. Based on individual opinions, either C or D could
be the best or worst speaker in the
world. It appears that their disabilities prevented some of the listeners
from hearing certain of the deficiencies.
Sadly, some listeners who fall into the problem category are talented and
knowledgeable musicians or audio
professionals whose vocations may have contributed to their condition.
However articulately their opinions are
enunciated, their views are of value only to them, personally.
The conclusion is clear. If there is any desire to extrapolate the results
of a listening evaluation to the
population at large, it is essential to use representative listeners. In
this context, it appears to be adequate to
employ listeners with broadband hearing levels within about 20 dB of
audiometric zero. According to some
large surveys, this is representative of about 75%, or more, of the
population - an acceptable target audience for
most commercial purposes. This is not an "elitist" criterion.
PRACTICE MAKES PERFECT - USE TRAINED LISTENERS
Listeners in the early tests gained much of their experience "on the job",
while performing the tests.
Some listeners were musicians, and others had professional audio experience,
but most were simply audio
enthusiasts. Probably the single most apparent deficiency of novice
listeners was the lack of a vocabulary to
describe what they heard. Without such descriptions, most listeners found it
difficult to be analytical in forming
their judgments, and to remember how various test products sounded. It was
also clear that, without the
prompting of a well-designed questionnaire, not all listeners paid attention
to all perceptual dimensions, resulting
in judgments that were highly selective.
As the relationships between technically-measurable parameters and their
audible importance became
clearer, it was possible to design training sessions that focused on
improving the ability of listeners to hear and to
identify specific classes of problems in loudspeakers. With the aid of
computers, this training has been refined to
a self-administered procedure, which keeps track of the student's progress
[5]. From this we have also been able
to identify program material that is most revealing of the defects that are
at issue, thus improving the efficiency
and effectiveness of the tests.
BLIND vs. SIGHTED TESTS - SEEING IS BELIEVING
Knowledge of the products that are being evaluated is generally understood
to be a powerful source of
psychological bias. In scientific tests of many kinds, and even in wine
tasting, considerable effort is expended to
ensure the anonymity of the devices or substances being subjectively
evaluated. In audio, though, things are
more relaxed, and otherwise serious people persist in the belief that they
can ignore such factors as price, size,
brand, etc. In some of the "great debate" issues, like amplifiers, wires,
and the like, there are assertions that
disguising the product identity prevents listeners from hearing differences
that are in the range of extremely
small to inaudible. That debate shows no signs of slowing down. In the
category of loudspeakers and rooms,
however, there is no doubt that differences exist and are clearly audible.
To satisfy ourselves that the additional

rigor was necessary, we tested the ability of some of our trusted listeners
to maintain objectivity in the face of
visible information about the products.
The results are very clear, and strongly
supportive of the scientific view. Figure 4 shows that,
in subjective ratings of four loudspeakers, the
FIDELITY
differences in ratings caused by knowledge of the
products is as large or larger than those attributable to
the differences in sound alone. The two left-hand
striped bars are scores for loudspeakers that were large,
expensive and impressive looking, the third bar is the
score for a well designed, small, inexpensive, plastic
three-piece system. The right-hand bar represents a
moderately expensive product from a competitor that
had been highly rated by respected reviewers.
When listeners entered the room for the sighted
tests, their positive verbal reactions to the big beautiful
speakers and the jeers for the tiny sub/sat system
BLIND SIGHTED
FIGURE 4: A comparison of blind vs. sighted foreshadowed dramatic ratings
shifts - in opposite
evaluations of the same products by the same directions. The handsome
competitor's system got a
group of listeners. higher rating; so much for employee loyalty.
Other variables were also tested, and the results
indicated that, in the sighted tests, listeners substantially ignored large
differences in sound quality attributable to
position in the listening room and to program material. In other words,
knowledge of the product identity was at
least as important a factor in the tests as the principal acoustical
factors. Incidentally, many of these listeners
were very experienced and, some of them thought, able to ignore the
visually-stimulated biases [6].
At this point, it is correct to say that, with adequate experimental
controls, we are no longer conducting
"listening tests", we are performing "subjective measurements".
"ZOOMING IN" ON THE PROBLEMS
Inevitably, as we make progress in improving products, the listening tests
no longer include examples of
bad performance. On the 0 to 10, junk to jewels, "fidelity" scale,
therefore, one ends up listening exclusively to
devices that score in the top ten percent, or so. When reminders of bad
sound are removed from the tests, an
interesting thing happens: listeners spontaneously expand the scaling of
their responses to fill more of the range.
In the absence of reminders about how bad things can really be, we become
more critical of the relatively good
sounds we are evaluating.
10
9
8
7
6
5
4
3
2
1
F
I
D
E
L
I
T
10
9
8
7
6
5
4
3
2
1
10
9
8
7
6
5
4
3
2
1

FIGURE 5: What happens when listeners evaluate products that are closely
ranked at the top of a range of
products. Without any "anchor products" to remind them of how things sound
at lower ratings, the response
scale expands to fill a "comfortable" range.
A consequence of the phenomenon illustrated in Figure 5, is that, when they
are auditioned in isolation,
listeners tend to exaggerate the importance of small differences. If one is
focusing attention on the small
differences, that is a good thing. If, however, one is attempting to arrive
at an evaluation of a phenomenon in a
manner that relates to its importance in a total system, it is a distorted
judgment. Real-world examples of this
occur when, for example, one does comparisons of devices that are
fundamentally similar. Many electronic
devices fall into this category, as do loudspeakers that differ only in
minor details. The listener impressions may
be that there is an undisputed winner, a 9 versus a 6, but the reality is
that, if there were any other variables in the
test, the ratings may differ only by fractions of a point, and may even be
in a different order.
Because of this, another response-scaling method is used when we are
interested only in the relative
performance of devices. It is a preference scale.
1
9876
5
432
1
0
RATING
PREFERENCE
REALLY LIKE
SLIGHT PREFERENCE
LIKE
NEITHER LIKE NOR DISLIKE
MODERATE PREFERENCE
DISLIKE
REALLY DISLIKE
STRONG PREFERENCE
FIGURE 6: The preference rating scale that is used when comparing sounds on
a relative, rather than an
absolute, basis. On the right are the suggestions for rating differences
according to the strength of the
preference.
The function of this kind of response scaling is to establish how listeners
respond to differences between
sounds. Since the scale is not "anchored" by reference to sounds known to be
very good and very bad, there is
nothing to indicate where they stand in any absolute sense.
TECHNICAL MEASUREMENTS
SUBJECTIVE MEASUREMENTS
With statistically reliable, repeatable, numerical subjective ratings of
loudspeakers in hand, it is an
unavoidable temptation to look for orderly relationships with technical data
on the same products. Figure 7
shows that the axial response of a loudspeaker needs to be very smooth, flat
and wide-band in order to achieve
high subjective ratings. One can rightly conclude that this is associated
with a perception of timbral neutrality - a
lack of coloration. But, there is more to the story. Since loudspeakers
function in rooms, it is necessary that the
sounds radiated in other directions also be well behaved, so that the ears
receive similarly neutral sounds after
single and multiple reflections from floor, walls, etc. The extension to the
requirement, therefore, is that the off

axis frequency
responses,
including sound
power, also be
well-behaved
[7,8]. The
design target,
therefore, is a
smooth, flat,
axial response,
with a constant directivity as a function of frequency. Just as axial
behavior, by itself, is an insufficient criterion,
so is sound power.
FIGURE 7: A simplistic view of the relationship between the
spatially-averaged axial frequency response of
loudspeakers and their subjective ratings. This is a necessary, but not
sufficient, criterion of excellence.
Since absolute perfection in transducers and enclosures is still a remote
possibility, it is important to be
able to identify, in measurements, the presence and the audibility of
defects such as resonances. In this way
designers can choose to build the best product possible or, by making
appropriate compromises, the best product
at a given price level. The first step in this process is to develop a
measurement method that allows the eyes to
identify the presence and magnitude of defects that are audible. The second
step is to identify the measured level
at which the defect is audible - the detection threshold below which the
defect ceases to be a problem.
F
I
D
E
L
I
T
Y
015L
15UP
15 DN
FIGURE 8: A simple form of spatial averaging, in which
measurements on several axes can be viewed individually, or in
combination.
A frequency response curve contains evidence of both
resonances and acoustical interference. It is important to be able
to identify which detailed features in the curve are attributable
to each phenomenon. Why? Because, in a room, resonances
will be easy to hear, and interference will be perceptually
attenuated.
Spatial averaging is a simple way to identify resonances
15R since, in the data for each of the microphone positions, the
resonances will be relatively unchanged, while evidence of
acoustical interference will change as a function of microphone
position. When the collection of frequency response curves is
averaged, the visual evidence of resonances remains, while that
for the interference is diminished. It is a very simple, but very effective
analytical method.
0
10

FIGURE 9: A frequency response measurement before (top curve) and after
(bottom curve) spatial averaging.
Data processing of multiple measurements, as in Figure 9, allows us to see,
in high resolution, the
amplitude and "Q" of resonances in complex mechanical and acoustical
systems. These data were gathered in an
anechoic chamber, where it is possible to measure with high resolution over
the entire frequency range.
A practical problem is that many measurements these days are made with FFT
or TDS systems that timewindow
the data so that anechoic measurements can be made in normal rooms. A result
of the time windowing
is that the frequency response data have limited frequency resolution, most
noticeable at the low frequency end
of the spectrum. As a result, it is not
True Level
CANNOT MEASURE WHAT WE HEAR
possible to see high-Q phenomena at low
and middle frequencies.
10
FIGURE 10: Identical high-Q resonances
B
(Q=50) at equal intervals between 20 Hz
0
and 20 kHz, adjusted to the threshold of
audibility, as measured by an FFT
measurement system having a time
20 50 1 00 500 1K 5K 10K 20K window of 17 ms, corresponding to a
FREQUENCY (Hz) frequency resolution of 60 Hz.
Resolution limitations of the kind shown in Figure 10, and worse, are common
among loudspeaker
designers and reviewers, because of the scarcity and expense of anechoic
chambers, and the practical difficulties
in measuring outdoors, in nature's own anechoic space. It means, simply,
that many commonly-used and
published measurements simply cannot reveal visual evidence of certain kinds
of audible problems falling within
a critical portion of the frequency range - that of the human voice.
Another common measurement is one in which the audible frequency range is
divided into equal fixedpercentage
bandwidths, such as 1/3 octaves, or in which a high-resolution measurement
is heavily smoothed, or
spectrally averaged, on a continuous basis. These spectral-averaging devices
have extremely limited utility in the
design and evaluation process. Spatial averaging adds information, whereas
spectral averaging removes spectral
details, making curves look smoother and prettier. The sound, though,
remains unchanged. Is it any wonder that
some people mistrust measurements?
Serious loudspeaker manufacturers need to be able to see, in measurements,
anything that might result in
an audible defect. In terms of the measurement of frequency response, it
means expensive anechoic chambers,
facilities for gathering data over an entire sphere, and elaborate
post-processing of the data. Doing this quickly
and accurately is neither inexpensive nor simple.

THE AUDIBILITY OF RESONANCES
Why is it that resonances are so important? Because they are the fundamental
building blocks of almost
all of the sounds we are interested in hearing. High-Q resonances define the
pitches of voices and instruments.
Medium- and low-Q resonances define the timbres of sounds, allowing us to
distinguish between different voices
and instruments. It is subtle differences in the resonant structure of
sounds that are responsible for the nuances
and shading of tone in musical sounds. Our ears are very highly attuned to
the detection and evaluation of
resonances, and it is therefore no surprise that listeners zero in on them
as unwanted "editorializing" when they
appear in loudspeakers.
In order to be effective, design engineers need to know when a resonance is
present, and if it is audible.
PINK NOISEQ=50
Q=10
Q=1
The techniques described above, help to identify the
presence of resonances. Reference 9 describes, in
detail, how the audibility of resonances is related to
measurements.
FIGURE 11: The amplitude response measured in an
otherwise perfect system, after a single resonance has
been added at the threshold of detection, when listening
to the most revealing signal, pink noise.
dB
In that study, resonances of different Q, at
different frequencies, were added to an otherwise
excellent system, and the amplitudes at which they
100 200 500 1K 2K 5K 10K were just detected by listeners were determined by
FREQUENCY (Hz)experimentation. Different kinds of program material
yielded different thresholds, as did different listening
environments.
Resonances reveal themselves in both the frequency-domain (amplitude and
phase vs. frequency) and the
time domain (impulse response / transient response). As defined by the
Fourier Transform, if there is
misbehavior in one domain, there will be misbehavior in the other, so we
have two ways to look for problems.
It is a matter of fact that high-Q resonances exhibit prolonged ringing in
the time domain, and that low-Q
resonances exhibit little ringing. The irony of this finding is that, as
represented in conventional steady-state
frequency response measurements, Figure 11, the low-Q resonances were
detectable at much lower amplitudes
than resonances of higher Q. If this is so, it means that a treasured belief
is in jeopardy. The popular belief is
that prolonged ringing, by itself, is a reliable indicator of an audible
problem. To test this, we performed the
measurements in the time domain, with the following results.
INPUT SIGNAL FIGURE 12: Pulse responses of an otherwise perfect
system (top curve) to which resonances have been added at
the thresholds of detection.
Q = 50 Looking at Figure 12, it is important to note that, in
terms of audible changes in timbre, these responses are all
equal. To the eyes, though, the long tail of the Q=50
resonance is quite alarming, and the Q=1 response appears
Q = 10 almost perfect. How can this be so?
One important factor would appear to be related to
the portion of the spectrum influenced by a resonance.

High-Q resonances are very narrow-band phenomena. In order for one of these
to be energized by music, a
sound would need to be closely centered on the frequency, and remain there
long enough to impart significant
energy to the resonant system. The higher the Q, the longer is the
"build-up" period. In music, pitches are
constantly changing, and voices and instrumental sounds frequently have
vibrato, a fluctuating pitch. Such
sounds would more often drive a low or medium Q resonance to maximum output,
than they would a high-Q
resonance. Statistically, therefore, with music and speech, lower-Q
resonances would be heard more frequently
than those of higher Q.
The apparent contradiction between perception and measurement, then, begins
with the observation that,
as conventionally measured, the frequency response is a "steady-state"
measurement, showing the resonance
outputs at their maximum amplitude. With music, high-Q resonances are rarely
driven to their maximum
outputs, and so are less audible than the measurement indicates. The problem
is not that the measurements are
wrong, or irrelevant, it is that they are non-linearly related to the
perceptual mechanism in humans, and therefore
must be interpreted. Time-domain measurements are similarly problematic,
since they suggest audibility in the
prolongation of ringing. Such phenomena are wonderfully visible in pulse
responses, tone-burst responses and
the highly ornamental "waterfall" diagrams that digital measurement systems
permit. Truth is that, without
careful interpretation, these are just as misleading as the frequency
responses. The conclusion is that, in practical
situations, if a resonance does not make itself apparent in an accurate,
high-resolution spatially-averaged,
frequency-response measurement, then it is probably not audible. If a
resonance is visible in a frequency
response measurement, its audibility must be assessed by comparison with
data from Figure 11, or better, from
the detailed analysis in reference 9.
An interesting fact now emerges: that the conventional method of specifying
the excellence of frequency
response -± x dB - is almost useless unless the tolerance is very, very
small. For equal audibility, high-Q
phenomena could be ± 5 dB, while moderate-Q resonances could be ± 3 dB and
low-Q and other broadband
deviations could be ± 0.5 dB. Clearly, frequency response curves must be
interpreted, there is no simple "catchall"
kind of tolerance specification that is truly meaningful. Such is life.
Having identified the presence of a problematic resonance, it is the task of
the design engineer to
diagnose the cause, and to prescribe a remedy. All the while, it is
essential to keep a close eye on costs. In this
task, other specialized tools are needed, since the origins can be both
acoustical and mechanical, and they can be
associated with either the drivers or the enclosure. It is a curious
phenomenon that the perception of "boxiness"
in sound, may have nothing to do with the box itself. It is not uncommon for
the offending resonance to have
another origin.
In the analysis of resonances, a laser interferometer/vibrometer is a
powerful ally, in that it can show the
complete vibratory behavior of a surface, such as a loudspeaker diaphragm or
an entire surface of a loudspeaker
enclosure. Vibration is important only if sound is radiated. Surfaces do not
move uniformly, as a piston, at all
frequencies, so measurement at a single point can be misleading. In
practice, portions of the surface move in
opposite directions, simultaneously. Consequently, some vibratory modes
radiate sound very effectively, others
less so, and some not at all. It is important not to go chasing after modes
that simply cannot be audible. The
combination of finite element analysis, in the design of diaphragms and
enclosures, and modal analysis, after the
prototype is built, allow engineers to be intelligent about the way they use
shapes, materials, braces, etc. in
reducing the audibility of resonances. The traditional method of playing
safe, has been to build enclosures from
massive, dense and stiff materials. If cost, size and weight are not
considerations this method works well.
However, enclosures with excellent acoustical performance can be built from
mundane materials, at much lower
cost, with due regard to modal prediction and analysis. In lower cost wooden
enclosures, and especially with
plastic enclosures, this becomes normal engineering practice.

LOUDSPEAKERS AND ROOMS
As important as the enclosures behind loudspeaker diaphragms are, it is the
ones in front that, as often as
not, give us serious problems. The listening room is the final audio
component, and it is the one over which the
loudspeaker manufacturer has little or no control. The first important step
is to design loudspeakers so that they
have a reasonable chance of sounding good in a room.
In a room, listeners hear the direct sound first, followed quickly by early
reflections from floor, ceiling
and sidewalls. Then arrive the multitudes of sounds from many reflecting
surfaces after several reflections - the
reverberation. Ideally, all of these should reinforce a similar timbral
signature in the mind of a listener. This can
only happen if the loudspeakers are designed to radiate similar sounds in
all of the relevant directions. In
technical terms, this reduces to a requirement for constant directivity as a
function of frequency.
The directivity itself can take different forms for different applications,
but is should not change with
frequency. This is a difficult requirement to fill, and it requires special
measurement techniques to allow
engineers to judge their success as products are developed. In brief, it
requires that we know:
.. the nature of sounds radiated in the direction of the listener (the
on-axis/listening-window performance),
.. the nature of sounds that will be reflected from adjacent room surfaces
(vertical and lateral "early-reflection"
performance), and
.. the nature of sounds that will generate the diffuse reverberant sound
field (the sound power).
From these the directivity index can be calculated. It is the combination of
these that matter; any one by
itself is insufficient data.
Loudspeakers that meet these requirements also tend to be the ones that win
listening tests, and there can
be no better reinforcement for a methodology than that. One of the factors
in listening tests is imaging, and that
raises an issue for which there is not a complete answer at the moment: what
is the ideal directivity?
LOUDSPEAKERS FOR STEREO
As discussed earlier, 40-some years with two-channel stereo have yielded
nothing in the way of a clear
direction. Although the majority of loudspeakers sold are traditional
forward-facing "cone and dome" designs, it
is also true that the majority of listeners are not very critical about the
imaging of their systems. Among those
who are, the "high end audiophiles", those designs figure strongly in their
preferences. However, so do designs
of very different kinds, like, dipole, bipole and directional horns. The
perceptual consequences of speakers this
diverse are not subtle. Bipole designs have approximately omnidirectional
sound radiation properties, and
therefore produce energetic reflected sound fields in listening rooms.
Conventional forward-facing systems, and
horn-loaded systems will place the listener in a sound field in which the
direct sound is more prominent. The
more directional the system, the more dominant will be the direct sound.
It is probably correct to say that the majority of listeners find stereo to
be pleasantly embellished if the
room reflections are energetic. The sound tends to be open and spacious,
with a good sense of depth, but the
specific images can be rather vague - in other words, not unlike real
concerts. A positive effect of this vagueness
is that the stereo listening region is enlarged.
However, there is also a category of listeners who respond unfavorably to
this kind of reproduction, and
prefer to have a very specific, almost pinpoint, sense of image position.
Interestingly, this category includes
many recording engineers who, in their studios, require that they be able to
hear, very precisely, the results of
their manipulations. Consequently, recording studios are often acoustically
rather dead, and the loudspeakers
directional (often horn loaded), or placed very close (so-called near-field
listening). However, these same
people, at home, frequently revert to the more spacious version of stereo.
So, go figure.
+ -

LOW BASS MID FREQUENCIES HIGH TREBLE
+ +
FIGURE 13: Forward-firing (left), bidirectional-in-phase - Bipole (center)
and bidirectional out-of-phase -
Dipole (right), are just some of the very different directional patterns
that are used in loudspeakers for stereo
systems. The differences in imaging precision, spaciousness, and soundstage
depth are not subtle. Once
selected, though, the characteristics are applied to all kinds of music,
whether it is appropriate or not.
LOUDSPEAKERS FOR MULTICHANNEL AUDIO
With the introduction of multichannel audio, things are no less complicated.
Multichannel sound should
mean that things become more controlled, that listeners stand a better
chance of hearing the senses of direction
and space that the artists created. In film applications, this is actually
attempted. The film industry, has had
basic standards for playback systems and environments for many years. THX
improved on this, and attempted to
translate it into the home. For the Dolby Surround films of that era, it was
moderately successful. However,
things now are much more confused, with Dolby Digital and DTS sound tracks
and music recordings, and the
Audio DVD around the corner. One senses an impending free-for-all in which
anything goes.
If it were approached logically, however, it seems that there is a scheme
that makes sense. The purpose
of multiple channels with loudspeakers located around the listeners, is to
allow for a large variety of predictable
localization and spatial effects. Since one of these is a sense of intimacy,
wherein the sound from the front
loudspeakers does not energize the listening room reverberant field, there
is a requirement for front loudspeakers
that are predominantly forward firing, with directional control in both
vertical and horizontal planes. The wellknown
THX requirement has been for some directional control in the vertical plane
only. Many of the
implementations have been somewhat less than ideal; simple vertical arrays
of drivers cause severe lobing at the
off-axis angles at which floor and ceiling bounces occur. If we are to
address this issue properly, we need to
incorporate horizontal directional control as well - our ears are in the
horizontal plane, and it is horizontal
reflections that are primarily responsible for the impressions of
spaciousness [10]. We also need to focus on how
well the speakers behave at the off-axis angles of importance - the adjacent
boundary reflections and sound
power.
Predictable directional control is possible with horns, waveguide-loaded
tweeters or complex twodimensional
arrays, all of which can deliver excellent sound, using today's technology.
The alternative,
unattractive in most practical situations, is to move quantities of sound
absorbing material into the room and
cover large areas with it. The lack of attraction has two components, visual
and acoustical. Visually, areas of
sound absorber run contrary to popular themes of interior décor.
Acoustically, absorbers dissipate sound energy
that one has paid good money to create, thus making the speakers work even
harder.
In a multichannel system, the impressions of space and envelopment are
provided by loudspeakers
positioned to the sides of the listeners. Here we enter hostile territory,
with monopoles, dipoles, tripoles and
quadrapoles attempting to be the perfect solutions. Truth is that, while all
of these names have meaning in the
physics of sound, the products bearing them are only crude approximations to
these forms of acoustic behavior.
What really is at issue here is the matter of whether the listener should be
in a predominantly direct or

predominantly reverberant sound field from the surround loudspeakers. There
cannot be a single correct answer
for the loudspeaker configuration until there is agreement at the production
end of the process.
From the perceptual point of view, impressions of ambiguous localization and
spaciousness exist when
sounds arriving at the two ears are uncorrelated, containing many
reflections. If the decorrelation is in the
recording, or is added electronically, spaciousness can be perceived in
headphone listening. It can also be
convincing through conventional forward-facing surround speakers. However,
much (most?) existing recorded
material is deficient in this respect, and additional decorrelation tends to
be beneficial. Using multidirectional (or
multiple) surround speakers is a good method of adding decorrelation through
multiple reflected sounds. The
actual performance, however, is dominated by the geometry and reflective
properties of the room.
At present, movies are still made assuming that audiences will experience a
multiplicity of speakers down
the sides of the cinema, even though the digital discrete surround channels
are all wideband and full range. At
home, this argues for multidirectional surround speakers, strong electronic
decorrelation, or multiple directradiating
monopoles - i.e. business as usual. However, the notion of five identical
loudspeakers is gaining
favor. Multichannel music, still very much in its infancy, also supports the
idea of five identical loudspeakers.
DTS, in its multichannel music demos, goes a step further, and has been
promoting the idea of moving the
surround speakers to the rear of the room, mirroring the left and right
fronts, and delivering sounds of individual
musicians to these locations.
At this point the audience divides into two camps - those who like having
musicians behind them, and
those who do not . . . and the feelings can be very strong. Some see this as
a cause to criticize multichannel
audio. This is where we must invoke the old adage: if you don't like the
message, don't shoot the messenger.
Multichannel audio systems are just delivery systems [11].
SPEAKERS FOR MUSIC AND SPEAKERS FOR MOVIES
Are there fundamental differences between loudspeakers designed for movies
and music? This oftrepeated
concern is really the wrong question. The real question is: are there
fundamental differences between
loudspeakers designed for stereo and those designed for multichannel
systems? The answer is that there may be.
Can multichannel delivery systems work equally well for music and films?
They must or we will have
consumers up in arms, and for very good reasons. A good loudspeaker is a
good loudspeaker, and there is no
practical reason for different standards of performance in loudspeakers
intended for films and music. There are
reasons for differing directivities and, depending on how the loudspeakers
are used, listeners will experience
different imaging and spatial effects depending on the manner in which
loudspeakers radiate their sound into
rooms. That has always been true for stereo, and it remains so for
multichannel systems. Thus, the choice of
loudspeaker directivity should not alter ones expectations of inherent sound
quality.
GOOD BASS IN ROOMS - A BASIC PROBLEM
At low frequencies the room interactions take on a special flavor because of
the way acoustical standing
waves (resonances) dominate what we hear. The quantity and quality of bass
sound is as much or more
determined by the room and how it is set up, as it is the speakers
themselves [12,13,14,15].
This is an enormous frustration for speaker manufacturers. Once the product
is in a box, on a truck, we
have lost control of how the speaker will sound at low frequencies. Gaining
some control, has been the objective
of several projects and products over the years. Short of hiring an
acoustical consultant, and being willing to
rearrange the furniture and possibly the walls, what can be done?
The short answer is "equalize". At this point, some people's hackles are
rising, I am sure. Equalization
has acquired a bad reputation over the years. To some proponents, it is a
"cure all"; set up a microphone, follow
the dancing lights of a "real-time analyzer", reach for the trusty
multifilter equalizer and create a pretty curve.
Such an exercise is almost certainly doomed to disappoint. Steady-state room
curves are "dumb" measurements,
in that the microphone simply adds up all of the incoming sounds, from
whatever direction, after whatever time.

One-third octave analysis, which is typical of these devices, is a crude
measurement. Two ears and a brain are
much more sensitive and analytical, directionally, temporally and
spectrally.
To be successful, equalization should address the problems it has the
potential to remedy. There are some
things equalization can and cannot do:
1. It cannot make a poor speaker sound good. With laboratory measurements to
work from it might make a
good speaker sound better. However, there is nothing that an average
consumer can measure in a listening
room that would reveal problems in a speaker that can be repaired by
equalization. If the speaker has been
competently designed, it should probably be left alone at frequencies above
about 300 to 500 Hz, whatever
the room-curves look like.
2. At frequencies below about 300 to 500 Hz, the system performance is
dictated by the shape and size of the
room, and the position of the speaker and listener within it. Three factors
are interactively operational he
.. solid-angle gains (the proximity of the speaker and listener to adjacent
room boundaries: walls and floor.)
Addressing this requires a broadband "tone-control" kind of equalization.
.. room resonances, which can cause strong peaks and dips, some with quite
high "Q". Before these can be
addressed, the resonances must be identified within a complicated confusion
of peaks and dips caused by
acoustical interference. It is always unwise to try to fill dips -
acoustical cancellations can be
"bottomless pits". Prominent peaks can be individually addressed with
parametric filters set to the
appropriate center frequency, Q, and gain/loss. Identifying those parameters
accurately requires highresolution
measurements; much more detail than is revealed by the traditional
1/3-octave "real-time
analyzers".
.. Acoustical interference caused by the interaction of many reflected sounds
within the room. These are
non-minimum-phase phenomena, and they cannot be addressed with equalization.
Fortunately, our ears
are less sensitive to their effects than our measurement systems, so we
really need to find a measurement
process that diminishes their visibility.
Successful equalization begins with good measurements of what is happening
in the room. Broadband
spectral trends are easily identified even in elementary real-time analyzer
(RTA) measurements. Separating
resonances from acoustical interference requires spatial averaging, just as
it does in speaker design. For this one
needs to make measurements at several different locations throughout the
listening area, and they need to be
averaged. This requires data acquisition and post processing, it cannot be
done with several microphones
plugged into a simple mixer. The measurements need to have high resolution
in the frequency domain, if there is
to be any hope of identifying the key parameters of problematic resonances.
For this job, the RTA fails.
Successful equalization requires a good equalizer. Ideally, this would be a
multiple parametric-filter type.
The JBL Synthesis home theater systems incorporate all of the required
characteristics for successful
equalization. The dedicated digital controller has 95
individually-configurable parametric filters distributed
among the 5.1 channels. In the laboratory, based on high-resolution
spatially-averaged anechoic measurements,
some filters are preset to address small residual problems in the speakers
themselves. The equalizer has helped to
create better speakers. Once the system is installed in the customer's home,
a trained installer arrives with a
custom measurement system to adapt the system to match the room at low
frequencies. The system employs five
microphones connected to a multiplexer, coupled to a laptop computer which,
in turn, is coupled to the digital
processor. In a carefully controlled sequence, the appropriate test signals
are sent through each of the channels,
the measurements are compared to predetermined "target functions", and a set
of filters is automatically designed
that will allow the system performance to approach the target with minimum
error. Built into the system are
safeguards that attempt to prevent it from trying to equalize the
unequalizable. Manual override is always an
option, so human intervention can modify a bad decision by the computer, or
accommodate customer preferences
in spectral balance.
At present, this is an expensive and cumbersome system. However, we are
learning from our experiences
in the real world. We are learning what the optimum target functions are,
how best to instruct the adaptive

equalization process, and what we may be able to leave out of a cost-reduced
version of the system. Ideally, this
should become a standard feature in popularly-priced active woofers and
subwoofers.
A LITTLE FOURIER ANALYSIS
Before leaving this topic, let us address one of the common
misunderstandings of equalization. It is
frequently asserted that equalizers, because they are filters, add ringing,
in the time domain. That is a fact. The
assertion continues that, in addressing a frequency response problem, one
may damage the transient behavior of
the system. That may or may not be the case.
Resonances in speakers and rooms also ring. If the filter is used to correct
a resonance, both the problem
and solution are minimum-phase phenomena. If the measurement of the problem
has been made accurately, and
the parametric filter solution has been designed to match the problem, then
the ringing of the filter is equal and
opposite to the ringing of the problem resonance. The ringing is gone, just
as the measured peak in the frequency
response has been eliminated.
The real trick in this is trying to ensure that the filters are not used to
"fix" something that cannot, and
perhaps need not, be fixed by this means. In the days of RTA's and
1/3-octave multifilter equalizers, success of
this kind was simply not possible. With today's elegant technology, it is
possible, but difficult.
SUBJECTIVISM vs. OBJECTIVISM - IN CONCLUSION
In this lengthy summary we have covered a lot of topics. Much of it was
matter-of-factly technical,
driven by data and the need to measure, and much of it was subjective,
driven by the desire to understand what
we can hear. All of it was oriented towards creating loudspeakers that sound
better.
The literature of audio continues to be sprinkled with letters and articles
debating the merits of science in
audio. The subjectivist stance is that "to hear is to believe", and that is
all that matters. Some of the arguments
conjure images of white-coated engineers with putty in their ears, designing
audio equipment, and not caring
how it sounds, only how it measures. I have never met such a person in my 30
years in audio science and
engineering.
The simple fact is that, without science, there would be no audio as we know
it. Without extensive and
meticulous subjective evaluation, there would be no audio science as we know
it. Without audio science, audio
engineering reverts to trial and error. So, where does this leave us?
Clearly, to be successful in this business,
one must be actively involved with both of the objective and subjective
sides.
A faith in the scientific method is not a blind faith. It is a faith built
on a growing trust that measurements
can guide us to produce better sounding products at every price level, for
every application. The proof, as
always, is in the listening, and one MUST listen.
The Harman International loudspeaker companies, JBL, Infinity, and Revel
have invested heavily in
measurement facilities that allow them to take the fullest advantage of
existing audio science. They have
invested in talented engineers who understand and respect the scientific
method, good sound and great music.
They have invested in elaborate listening rooms where they can enjoy and
criticize the fruits of their labors.
There are people on staff with many years of experience in successfully
probing the frontiers of knowledge in
product design and audio science, and they are equipped to continue those
investigations, to push those frontiers.
The arrival of multichannel audio for films required some adjustments in the
performance objectives of
speakers, certainly at the high end. Multichannel music is another, as yet
ill-defined, challenge. More speakers
in rooms, means less consumer tolerance for large boxes. Merging
loudspeakers with rooms is not easy, and it is
the one remaining large challenge for our industry. We are working on all of
these fronts. Stay tuned.

REFERENCES
1. "Listening Tests, Turning Opinion Into Fact", F.E. Toole, J. Audio Eng.
Soc., vol. 30, pp. 431-445 (1982 June).
2. "Listening Tests - Identifying and Controlling the Variables", F.E.
Toole, Proceedings of the 8th International Conference, Audio Eng, Soc.
(1990
May).
3. "Subjective Evaluation", F.E. Toole, in J. Borwick, ed. "Loudspeaker and
Headphone Handbook - Second Edition", chap. 11 (Focal Press,
London, 1994).
4. "Subjective Measurements of Loudspeaker Sound Quality and Listener
Performance", F.E. Toole, J. Audio Eng. Soc., vol 33, pp. 2-32 (1985
January/February)
5. "A Method for Training of Listeners and Selecting Program Material for
Listening Tests", S. E. Olive, 97th Convention, Audio Eng. Soc.,
Preprint No. 3893 (1994 November).
6. "Hearing is Believing vs. Believing is Hearing: Blind vs. Sighted
Listening Tests and Other Interesting Things", F.E. Toole and S.E. Olive,
97th
Convention, Audio Eng. Soc., Preprint No. 3894 (1994 Nov.).
7. "Loudspeaker Measurements and Their Relationship to Listener Preferences",
F.E. Toole, J. Audio Eng, Soc., vol. 34, pt.1 pp.227-235 (1986
April), pt. 2, pp. 323-348 (1986 May).
8. "Loudspeakers and Rooms for Stereophonic Sound Reproduction", F.E. Toole,
Proceedings of the 8th International Conference, Audio Eng, Soc.
(1990 May).
9. "The Modification of Timbre by Resonances: Perception and Measurement",
F.E. Toole and S.E. Olive, J. Audio Eng, Soc., vol. 36, pp. 122-142
(1988 March).
10. "The Detection of Reflections in Typical Rooms", S.E. Olive and F.E.
Toole, J. Audio Eng, Soc., vol. 37, pp. 539-553 (1989 July/August).
11. "The Future of Stereo", F.E. Toole, Part 1, Audio, vol 81, no.5, pp.
126-142 (1997 May), Part 2, Audio, vol.8, no.6, pp. 34-39 (1997 June).
12. "Perception of Reproduced Sound in Rooms: Some Results from the Athena
Project", P. L. Schuck, S. Olive, J. Ryan, F. E. Toole, S Sally, M.
Bonneville, E. Verreault, Kathy Momtahan, pp.49-73, Proceedings of the 12th
International Conference, Audio Eng. Soc. (1993 June).
13. "The Detection Thresholds of Resonances at Low Frequencies", S.E. Olive,
P. Schuck, J. Ryan, S. Sally, M. Bonneville, J. Audio Eng. Soc. Vol.
45, No. 3 (1997 March.)
14. "The Variability of Loudspeaker Sound Quality Among Four Domestic-Sized
Rooms", S.E. Olive, P. Schuck, J. Ryan, S. Sally, M. Bonneville,
presented at the 99th AES Convention, preprint 4092 K-1 (1995 October).
15. "The Effects of Loudspeaker Placement on Listener Preference Ratings",
S.E. Olive, P. Schuck, S. Sally, M. Bonneville, J. Audio Eng. Soc., Vol.
42, pp. 651-669 (1994 September)
5/8/98 rev. 8/19/99


Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
A New Laboratory for Evaluating
Multichannel Audio Components and Systems
SEAN E. OLIVE, AES Fellow, BRIAN CASTRO, AES Member, AND
FLOYD E. TOOLE, AES Fellow
R&D Group, Harman International Industries Inc., 8500 Balboa Blvd.,
Northridge, CA, 91329, USA
Email:
ABSTRACT
The design criteria, features and acoustic measurements of a new listening
laboratory
designed specifically for listening tests on multichannel loudspeakers and
components
are described. Among its features is a novel automated speaker shuffler that
eliminates
loudspeaker position effects or allows the variable to be efficiently
tested. Other features
include complete computer control of experimental design, control and
collection of
listener data, making listening tests more reliable and efficient.
1.0 INTRODUCTION
Listening tests are the final arbiter for determining whether an audio
product sounds
good, and they play a critical role in the research and development of new
products.
Designing and conducting listening tests that produce reliable and accurate
data is,
however, no simple task. There are many variables other than those under
test that unless
removed or controlled can seriously bias the results [1-9]. Two of the more
difficult
variables to control are the listening room [5],[7],[9] and the position(s)
of the
loudspeakers under test [5],[9] both of which can significantly influence
the sounds that
arrive at listeners' ears and listeners' perceptions of them.
Recently we had the opportunity to design and construct a new
state-of-the-art listening
laboratory to be used for developing and subjectively testing multichannel
loudspeakers
and other components. The goal from the outset was to build and equip a
listening
laboratory that could generate subjective measurements as accurate,
efficient and free of
1

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
bias as possible. To meet these goals, a large effort went into developing
hardware and
software that would automate the design and control of experiments,
including the
collection, storage and statistical analysis of listener data. Included in
the design is a
novel automated speaker shuffler that performs positional substitution of 9
loudspeakers
so that positional biases can be eliminated or efficiently tested. By
eliminating position as
a variable, the speaker shuffler has reduced the length of a typical
multiple loudspeaker
listening test by a factor of 24:1 making product development faster and
less costly.
Another notable feature of the room is that the acoustics can be easily
varied from almost
hemi-anechoic to semi-reverberant by adding removable reflective panels to
the walls
and ceiling.
This paper describes the rationale, features and measurements of the new
listening
facility, which we call the Multichannel Listening Laboratory (MLL).
Finally, the results
are compared with several current international standards that recommend
performance
criteria for listening rooms intended for critical listening.
1.1 Listening Room Standards
Several standards recommend values for various acoustic parameters that
define listening
room performance. The goal of these standards is to facilitate the
replication of listening
evaluations in different rooms under the same test conditions. This is
particularly
important for radio and television broadcast corporations, audio production
facilities,
large audio equipment manufacturers, and international standards and
research
organizations, all of whom have multiple facilities in which critical
judgments are made
on the same program material or equipment. Ideally, if the listening rooms
and test
conditions in which these judgments are made are sufficiently similar, and
the listeners
have normal hearing are properly trained, then a consensus in opinion should
be possible.
If not, then there is likely something wrong with the test procedure itself.
In reviewing these various standards, a serious problem common to many is
that while
they define tolerances for specific acoustic parameters, they do not
adequately define
how the parameter is to be measured. For example, IEC is the only standard
that specifies
how reverberation time should be measured, even though it has been shown
that RT60
can vary widely depending on the technique used. Unfortunately, this rather
defeats the
purpose of defining a standard in the first place! It is conceivable that
one measurement
method may show the room meets the standard, while another measurement
method may
not. Added to this is the belief, held by some authorities, that in small
rooms,
reverberation time is a parameter of little or no value.
A very good discussion and summary of standards as they relate to the design
of
multichannel listening room intended for loudspeaker listening tests are
given by
Jarvinen et al in [9].
The current standards that recommend listening room performance include:
2

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
1. IEC Publication 268-13: Sound System Equipment, part 13. Listening Tests
on
Loudspeakers (1985) [10 ]
2. NR-12 A, Technical Recommendation: Sound Control Rooms and Listening
Rooms.
2nd Edition, The Nordic Public Broadcasting Corporation, (1992) [11]
3. ITU-R Recommendation BS.1116: Methods for Subjective Evaluation of Small
Impairments in audio systems including multichannel sound systems, 2nd
Edition
(1997) [12]
4. ITU-R Recommendation BS.775: Multichannel stereophonic sound system with
and
without accompanying picture (1994) [13]
5. EBU Tech 3276 (2nd Edition, 1997 ) [ 14]
6. AES20-1996: Recommended Practice for Professional Audio - Subjective
Evaluation
of Loudspeakers (1996) [15 ]
The standards can be classified according to the intended application of the
listening
room and can be generally classified into two groups. The AES and IEC
standards were
intended for monophonic and stereophonic testing of loudspeakers in typical
domestic
listening rooms. Both these standards are now quite old and the recommended
room sizes
are too small to allow multiple comparison of multichannel systems.
The EBU, ITU and NR standards were drafted primarily by broadcasters and
allow for
much larger control rooms that can accommodate several listeners at a time.
Only the
AES, IEC and ITU standards include recommendations for listening test
methodology.
At the design stage, we did not intentionally set out to meet any of the
above standards.
However, in post-hoc examination have found that our listening room meets
both ITU
and EBU standards in its current configuration in which we have added
reflective and
diffractive surfaces to both the ceiling and walls.
In the following sections we show measurements made in the MLL and compare
these
with various acoustical properties recommended in the above standards. These
properties
include dimensions, floor area, volume, proportions, reverberation time and
background
noise. The values measured for the MLL are compared with the recommended
values in
Table 1 for each standard, and shows that the MLL meets both ITU and EBU
recommendations.
2.0 MULTICHANNEL LISTENING LAB (MLL)
2.1 Room Dimensions
The listening room itself consists of double-wall constructed shell built by
Industrial
Acoustics Corporation (IAC). The dimensions of the MLL were largely dictated
by our
requirements to be able to evaluate up to 3 different 5.1 or 7.1 channel
systems at a time
and accommodate 1-6 listeners. The room also had to be sufficiently large to
accommodate our automated 9-loudspeaker shuffler that requires a space of
approximately 9 m (L) x 1.5 m (W) x 1 m (D). This resulted the following
dimensions:
3

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
MLL Dimensions
Length 9.14 m
Width 6.58 m
Height 2.59 m
Floor Area 60.20 m 2
Volume 155.92 m 3
As shown in Table 1, the MLL satisfies the recommended volume and floor area
values
specified in ITU, EBU, and N12 standards. The MLL's volume exceeds the IEC
and AES
recommended limits of 110 m 3 and 120 m 3 respectively because the standards
were
intended for small domestic stereo listening rooms.
2.2 Room Proportions
The most problematic performance issue in small listening rooms is
non-uniform low
frequency reproduction caused by standing waves that produce large pressure
peaks and
nulls in the lower 3-4 octaves of the audio range. The distribution and
frequencies at
which these peaks and notches occur are directly related to the geometry of
the room. If
the ratio of the room dimensions is carefully chosen, a more uniform
response is possible.
Walker from the BBC [16] has created a room geometry criterion that has been
adopted
by both the EBU and ITU standards. The "Walker" criterion defines the limits
of the
ratios for length (l), width (w) and height (h) as:
1.1 w = l = 5.4 w -4 (1)
hh h
As shown in Table 1, the ratio of dimensions for the MLL meet the "Walker"
criterion
and therefore satisfies the EBU and ITU standards. The relatively large size
of the MLL
also benefit uniform frequency response in the lower octaves since the first
order width
and length modes are below 25 Hz.
2.3 Background Noise
Accurate and repeatable subjective measurements require a listening room
with low
background noise so those listeners are able to reliably judge the quality
of low-level
signals. Perception of timbre, nonlinear distortion, loudness and spatial
qualities are all
influenced by the presence and masking effects of background noise.
Minimizing background noise in the MLL was carefully considered during the
design and
construction. The IAC double-wall shell itself is located in a large room
that has limited
access to both people and noisy equipment. No part of the shell touches the
structural
walls of the building except the floor, which is mechanically floated.
4

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
The inner walls and ceiling of the double-wall IAC shell are made of heavy
gauge steel
panels separated 10 cm and filled with fiberglass. The inner surfaces are
perforated with
2.34-mm openings to provide substantial sound absorption inside the room.
The inner
walls are entirely floated and separated from the outer wall of the shell by
a 10 cm space
to minimize mechanical and acoustic transmission of noise.
The room has its own dedicated HVAC system with ventilation silencers and
acoustically
lined ducts that create a comfortable and quiet environment. For experiments
that require
extremely low background noise the room can be cooled and the HVAC can be
completely shut off during the test. The room requires minimal lighting
during the test
itself (i.e. 1 Halogen light) which means that noise from lights is not an
issue. All audio
equipment, other than the required amplifiers, is located outside the room,
and this also
helps to minimize electrical noise as well.
In an effort to simulate the construction of floors found in many homes, a
carpeted
"squeak-free" plywood floor was laid on 5 cm x 15 cm wooden joists separated
41 cm
apart. The joists are mounted on 6.4 mm neoprene pads for isolation from the
concrete
floor beneath. The rationale for constructing this floor is to allow
transmission of low
bass from the loudspeaker through the floor to the listeners' feet, since
the perception of
bass depends on what is felt, as well as what is heard. The front and middle
sections of
the floor can be removed to allow easy access of audio, video and data
cables that run
underneath the floor to access panels both inside and outside the room.
In reviewing the various listening room standards there is a wide range of
recommended
levels for background noise. The most stringent requirements are specified
by the EBU
and ITU standards, which call for minimum level of NR10, not exceeding NR15.
These
rather demanding requirements are likely justified in broadcast environments
where
listeners are frequently required to evaluate small signal linearity, for
example in relation
to CODECS.
At the other extreme, the AES and IEC standards both have rather liberal
recommended
background noise limit of 35 dBA measured using a slow time constant. The
AES
standard has an additional limit of 50 dB C-weighted for low frequency
noise. The less
stringent requirements are likely justified on the basis that they are aimed
at loudspeaker
evaluations in typical domestic environments where background noise levels
are typically
higher.
Figure 1 shows the background noise measured in the MLL with the air
conditioner
turned both on and off. Also plotted are the NR curves 0 through 15. The MLL
noise
curves each represent an average of four measurements take at 4 different
locations
around the listening area. The time over which each measurement was averaged
was 64 s.
The measurement was taken using a Bruel & Kjaer 4179, 1 inch microphone, a
Bruel &
Kjaer preamp Type 2660, and a Bruel & Kjaer real-time analyzer. The low
noise
microphone and preamp allow accurate measurement of sound pressure levels
below the
threshold of hearing, which is necessary at higher frequencies for measuring
rooms below
NR20. Figure 1 shows that with the air conditioning turned off, the MLL
meets NR5,
5

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
thus meeting the requirements of the EBU and ITU specification. With the air
conditioning turned on the noise increases to NR15.
2.4 Reverberation Time
The reflected sounds and reverberation time in a room have been shown to
have an
important influence on the perception of loudness, timbre and spatial
qualities and speech
intelligibility in both live and reproduced sound. While this is a complex
phenomena, the
acoustic community sees fit to summarize it all in a T60 measurement.
Both the EBU and the ITU standards specify values for the average
reverberation time in
the room. ITU and EBU recommend the value (within a tolerance of ± 0.05 s)
be
determined using the following equation:
1/3
25.0
V
T
=
m
s
(2)
V
ref
where Tm is the average reverberation time between 200 Hz to 4 kHz, V is the
volume of
the room, and Vref is the reference volume of 100 3 m. The EBU also put
limits on the
range of values specifying that the value should lie between 0.2 Tm 0.4
s.
The IEC standard specifies a Tm of 0.3 - 0.6 seconds which is very similar
to the AES
standard that recommends 0.45 s ( ±0.05 s). The N 12-A standard specifies Tm
be
measured in 1/3 octaves between 200 Hz to 2.5 kHz and be determined as a
function of
the floor area using the following equation:
35.0
S
T
=
m
±s 05.0
(3)
S ref
where S is the floor area of the room and S ref is the reference area of 60
2 m.
In addition to specifying the average reverberation time, most of the
standards
recommend that Tm be relatively independent of frequency within a certain
bandwidth
and tolerance. For ITU and EBU standards, the Tm value for each octave band
between
200 Hz - 3.5 kHz should vary no more than ±0.05 s from the calculated
optimum value.
Below 200 Hz, Tm is allowed to increase monotonically with frequency to 0.3
s above the
optimum value. Above 3.5 kHz, the tolerance is increased to ±0.1 s from the
optimal
value.
By substituting the volume of the MLL (155.92 3 m) into equation (2), we
calculate that
Tm should be 0.29 s to meet ITU and EBU standards. According to N 12-A, the
Tm for the
MLL should be 0.35 s.
6

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
The Tm of the MLL was measured using a MLSSA system from DRA laboratory. The
microphone was a Bruel & Kjaer 4134 microphone. The sound source consisted
of four
JBL Synthesis satellite loudspeakers crossed at 80 Hz over to a JBL
Synthesis Two
subwoofer located in the corner of the room. Each of the four satellites was
located
approximately 2 m apart and aimed at a different corner in an attempt to
create a diffuse
sound field. The measurement shown in Figure 2 represents a spatial average
of four
microphone locations. The average Tm value for the MLL is about 0.23 s,
which is
slightly below the calculated ITU and EBU optimal value of 0.29 s. However,
the curve
falls within the minimum recommended value, and is quite uniform with
frequency, only
rising slightly below 125 Hz.
2.5 Control of Early Reflections
With the advent of 5.1 and 7.1 multichannel and 3D audio playback systems,
there is a
trend among professional and home theater listening room designs towards
lower
reverberation times and the control of early reflections. There are sound
scientific reasons
for doing this, since strong early reflections are known to influence the
perceived spatial
and timbral qualities of reproduced sound [7], [17]. In the new generation
of
multichannel recordings and video disks, the additional center and surround
channels
allow the producer and recording artist to create much more realistic and
spatiallyenriched
environments than ever before. There is less need to use the room's
boundaries
and the loudspeakers' directional characteristics to compensate for the
obvious spatial
deficiencies inherent to stereo.
The EBU standard recommends that all reflections within the first 15 ms
after the arrival
of sound be no greater than 10 dB in level relative to the direct sound from
each sound
source. With multichannel setups the early sound field is rather complex
given that there
are between 5-7 loudspeakers and several boundaries. For example with 5
loudspeakers
and 6 boundaries there are 30 first order reflections and 150 second order
reflections.
Measuring and separating out these reflections is no trivial task. The
reflections from the
floor are particularly problematic to treat since in most facilities, the
floor surfaces must
be hard and reflective to facilitate the movement of people and equipment.
Nonetheless,
several organizations [18], [19] are building such rooms that meet this
reflection-free part
of the specification with the exception of the floor bounce.
In the MLL room, the only significant first order reflections are from the
floor, and these
are attenuated at higher frequencies by the carpet. At listener-loudspeaker
distances
greater than 2 m any reflection with a path length greater than 6.34 m will
be attenuated
10 dB by spreading loss [18]. This effectively eliminates all second order
reflections
since their path length exceeds this value. For front channel sources, first
order
reflections from the side walls will also be sufficiently delayed beyond the
15 ms time
gap. The main culprits are reflections from the front and back walls, and
the ceiling.
Fortunately these surfaces can be made absorptive by simply removing the
reflective
panels so that the absorptive surface is exposed. To reduce flutter echoes
from reflective
surfaces and to increase reverberation, 120 RPG Skylines, an omnidirectional
primitive
root number theory 2D diffusor, are placed on the reflective panels located
on the walls,
7

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
as well as on the ceiling and areas behind the loudspeaker as shown in
Figures 4 and 5.
These light-weight diffusors are easily removed or relocated, and help
reduce any other
specular reflections that may arrive after the direct sound.
2.6 Automated Speaker Mover
The position of a loudspeaker in a room has a significant impact on its
perceived sound
quality. Changing its position affects the way it couples to the standing
wave modes of
the room, and alters the physical characteristics of broadband reflections
that arrive at the
listener. In listening tests that involve multiple comparisons among
loudspeakers the
positional effects on listeners' ratings can be larger than the differences
between the
loudspeakers under test [8]. Unless these positional effects are controlled,
the results may
be contaminated by a nuisance variable.
For multiple comparison loudspeaker tests, asking human beings to sit behind
a doubleblind
screen and quickly and smoothly substitute the positions of 2-9 loudspeakers
(some
weighing upwards to 100 kg) on command presents an obvious logistical
problem.
Clearly the problem of positional substitution calls for an automated
solution. This
realization led to the development of our own custom-built speaker shuffler.
Prior to
having a speaker shuffler, the positional effects in loudspeaker tests had
been balanced by
testing each loudspeaker in each position. Any position-related bias would
be equally
distributed or balanced across each loudspeaker. More scientifically
rigorous designs go
even further and test all possible loudspeaker-position permutations so that
any possible
context effects between loudspeaker and position are also balanced.
The disadvantage of not having a speaker mover is that an additional number
of trials are
required to balance the variable position. This relationship in illustrated
in Figures 3(a)-
(b), which compare the number of trials required to balance the variable
position in
multiple comparison tests, with and without a speaker mover. The number of
trials is
calculated using the following equation:
Trials of Number = N Positions Speaker ! × N Programs × N Repeats (4)
Where N Speaker Positions equals the number of speaker positions in the
test, N Programs equals
the number of program selections being used and N Repeats is the number of
repeats. In
Figure 3 we, the experimental design shows no repeats, that is N Repeat = 1.
The graphs clearly shows that an automated speaker mover can drastically
reduce the
length of the experiment because the variable N Speaker Positions always
equals 1, regardless
of how many loudspeakers are compared. In comparing the two graphs we see
that there
is a 2:1 advantage for paired comparisons, a 6:1 advantage for triple
comparisons, and a
24:1 advantage for comparisons among four loudspeakers. When you multiply
these
ratios by the number of programs and repeats used in the experimental
design, the
number of trials quickly escalates. For multiple comparisons between four
loudspeakers
using 4 programs with no repeats, a total of 96 trials are required without
a speaker
mover. Having a speaker mover reduces the experiment to 4 trials. This
enormous
8

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
difference provided the justification to design and build a custom speaker
shuffler, since
over the long-term, it could afford considerable savings in person-listening
hours and
product development time.
A custom-built floor at the front of the room allows us to perform
positional substitution
of up to 9 different loudspeakers. A photograph of the speaker mover set up
for an A/B
stereo loudspeaker comparison is shown in Figure 4. Figure 5 shows a
photograph of the
speaker mover set up for a single comparison of a 5.1 loudspeaker system.
For the
purposes of the photograph the front, side and rear listening curtains have
been retraced
out of the way. Each loudspeaker is attached to one of nine pallets that
move in 1-inch
increments over a range of 4 feet forwards and backwards while the entire
array moves 4
feet to the left and right of the listener. The movement of the floor can be
controlled
manually from a programmable logic controller (PLC), or from a computer that
is linked
serially to the PLC via RS232. This allows all positions of loudspeakers to
be
programmed, stored and recalled quickly. The movement of the floor is
extremely quiet,
repeatable to within 1 inch, and fast. Transit time between positions is no
greater than 3 s,
and most positional changes are under 2 s. The transit speed is also
programmable and
can be decreased or increased if desired. As a safety measure, a light fence
is installed in
front of the moving floor so that if anyone crosses the light beam the
speaker mover
automatically stops.
The speaker shuffler allows position-controlled loudspeaker comparisons in
mono (up to
4 different systems), stereo (4 different systems) or three different
left/center/right
channel loudspeakers. At this time, positional substitution of surround and
rear channel
speakers must be done manually for multichannel experiments. The speakers
can be
placed away from the side and rear boundaries on stands, or placed on
adjustable shelves
that are mounted on baffles made of high-density board, that slide in a
track along the
perimeter of the room.
The moving floor gives us an efficient means to eliminate the effects of
loudspeaker
position, or it can do the reverse, and allow us to test the interaction
effects between
loudspeaker and position. By statistically-averaging a loudspeaker's
performance over a
number of different positions we can assess its off-axis performance, and a
number of
other parameters that are position dependent. All of this becomes essential
as we aim to
design loudspeakers that are 'room friendly' and develop digital room
equalization
systems.
Finally, the speaker mover also allows us to efficiently randomize between
each trial,
how the loudspeaker is identified to the listener (e.g. "A,B,C..). This
ensures that
listeners' judgments in each trial are statistically independent between
program
selections. Without a speaker mover, experimenters normally do not move the
loudspeakers behind the screen until a complete block of programs has been
rated. These
are not independent judgments since the listener knows they are rating the
same
loudspeaker(s) within each block. The extent to which this biases the
results has not yet
been reported.
9

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
2.7 Blind versus Sighted Listening Tests
It is generally accepted among scientists that psychometric experiments must
be
performed double blind. For audio tests, this means the identities of the
components
under test cannot be made known to the listener, and the experimenter cannot
not directly
control or administer the actual test.
In 1996 Toole and Olive in [2] conducted some blind versus sighted
loudspeaker tests
that showed both experienced and inexperienced listeners' judgments were
significantly
influenced by factors such as price, brand name, size and cosmetics. In
fact, the effect of
these biases in the sighted tests were larger than any other significant
factors found in the
blind tests, including loudspeaker, position and program interactions. These
experiments
clearly show that an accurate and unbiased measurement of sound quality
requires that
the tests be done blind.
To remove these biases from listening tests in the MLL an acoustically
transparent
curtain that is visually opaque is placed between the products and the
listeners so that
they do not know the identities of the products under test. All other
associated equipment
in the signal path is also out-of-sight and locked in an equipment rack,
since the
performance and paranoia of some listeners can be affected by simply having
knowledge
that a certain brand of interconnect or CD player is in the signal path.
The front screen consists of a black open knit polyester knit cloth chosen
for its acoustic
transparency and used as grille clothe in many of our loudspeakers. The
material is
attached to a large automated curtain roller so it can be easily lifted down
and up with an
infrared remote control. Weights are attached to a seam in the bottom so the
cloth retains
its tautness when in use. Retractable curtains made of the same material
surround the
listeners to hide the identities of loudspeakers located at the sides and
rear of the listening
room. Figures 4 and 5 show the front, side and rear curtains fully retracted
when not in
use, and Figure 8 shows the curtains in place during an actual listening
test.
2.8 Video Playback
Video and audio are increasingly becoming recorded, processed and
distributed together.
There is a growing interest among researchers in studying how the perceived
quality of
one affects the perception of the other. Although much research still needs
to be done,
evidence suggests there are bimodal interactions between the two that
influence listeners'
expectations and judgments of the quality of the audio, and vice versa.
Keeping this in
mind, we were careful in selecting a video playback system within our budget
that had
sufficient quality, so that it would not negatively impact listeners'
opinions of the sound
quality.
We selected a three gun front projection CRT made by Audio Video Source for
its above-
average picture quality and the additional advantage that is has no fan. The
picture is
projected on a 100 inch Stewart Microperf screen that is retractable so it
can be removed
10

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
for audio-only listening tests. The acoustical effect of the screen is
another factor that is
not completely understood, and will be a subject of investigation.
2.9 Automated Control, Collection and Analysis of Data
In designing the MLL, we wanted to automate as much as possible the design
and
running of experiments including the collection, storage and analysis of
data, in order to
reduce the time and costs of performing listening tests. Automation of
experiments has
the additional benefit of making listening tests more reproducible, largely
because it
reduces the risk of human errors and biases introduced by the experimenter.
Considerable
ongoing effort in software development is helping us to fulfill these goals.
Automation begins at the experimental design stage where all important
experimental
parameters and details are defined by the experimenter as a "*.exp" file
that is stored in a
database that resides on the Windows NT server.
The experiment file contains the following information:
.. The name of the experiment and a brief description
.. Detailed information related to the experimental design and protocol
including
definition of scales and randomization of variables. Protocol choices
include single or
multiple comparisons, ABX, ABC(with hidden reference) and different
threshold
measurement protocols.
.. Instructions to the listeners
.. Equipment control information and operational parameters required by the
audio
switcher for level matching, switching and overall output level.
.. The file names or track information for each program selection. This
information is
sent to the appropriate signal source device.
.. Information related to the position and movement of loudspeakers
.. A list of trials which the software randomly selects
The Windows NT server controls the running of the experiment including
control of all
associated equipment in the signal path. A block diagram of the equipment
and signal
path for the MLL is shown in Appendix 1. The lines that connect each block
as well as
the signal paths are color coded and typed according to whether the signals
are audio
(either analog or digital), video, infrared or RF control, computer data,
MIDI control or
sent over PCI or serial buss. The signal sources are the blocks on the top
left of Appendix
1. They currently include DVD and Laser Disk player, an 8-channel PCM
digital
recorder, and an 8-channel PC-based hard disk recorder (Lexicon Studio) and
its
associated A/D and D/A I/O cards. The audio and video outputs of the DVD and
LD
players are sent to the Lexicon DC-1 which provides AC-3 and DTS decoding
when
required. The analog outputs are sent to the Spirit 328 digital mixer which
provides signal
switching and level matching (within 0.03 dB) for up to 16 analog or digital
inputs. The 8
channel sources are sent digitally to the Spirit mixer and remain digital up
to the power
amplifier before they are converted by the Studer D/A's.
All operational parameters of the Spirit mixer can be viewed, stored and
recalled from the
NT Server via MIDI control.
11

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
The input of listener data, feedback and status information is done using
laptops
connected to the NT Server through a LAN. For single listener experiments,
the listener
can control switching of the stimulae remotely from their laptop. A
photograph of a
listener entering data on the laptop connected to the NT Server is shown in
Figure 6. For
multiple listener experiments, the NT Server controls the switching either
manually or
through software automation. During the experiment, all changes in listener
response data
can be viewed in real-time on the NT Server which performs running
statistical averages
and graphs of the results.
Remote access to the NT Server and control of the equipment from inside the
listening
room is also possible through a wireless RF mouse, keyboard and a flat panel
display, all
of which are connected to the Server. This might be required during set up
or for
informal listening sessions or product demonstrations. The flat panel
display also shows
status information to the listener(s) indicating what stimulus (i.e. A, B,
C.) is currently
selected, and any other necessary information.
Finally, all experimental data and information related to listeners (date
and time, name,
seat position, age) is stored in a relational data base which can be
formatted and imported
into various statistical packages we use for analysis of results.
Not shown in the block diagram is a video camera used for monitoring
subjects and to
detect and hopefully deter possible cheating. Also not shown is a two-way
intercom that
allows communication between the subject(s) and the experimenter.
3.0 CONTROL ROOM AND LISTENER TRAINING LAB
Outside the MLL is a lab area dedicated for audio and test equipment used
during the set
up, running and monitoring of listening experiments. Here a space is also
dedicated for
the training of listeners, which is done over headphones at computer audio
workstations.
Bech in [20] has shown that 6 trained listeners can provide data that is as
statistically
reliable as data gathered from 18 untrained listeners. Clearly, considerable
cost-savings in
time and money can be realized if listeners are trained before they
participate in formal
listening experiments. At Harman, listeners with normal hearing undergo a
listener
training program, which self-administered through a computer and custom
software
developed in-house [21]. The software teaches listeners to identify and rate
using
different scales, frequency response irregularities according to the center
frequency,
amplitude and Q of the distortion. The graphical user interface of the
training software is
shown in Figure 8.
The training focuses on frequency-related problems since these are the
common and most
serious audible problems found in most loudspeaker-related listening tests,
which many
untrained listeners find difficult to describe. The training solves this
problem by teaching
12

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
listeners to describe these phenomena in technical terms that design
engineers can
understand and use to correct any problematic audible artifacts in product
designs.
The training software has proved to be a valuable tool for teaching
listeners how to
describe and scale the various dimensions of sound quality in meaningful
terms, and
allows their performance to be quantified in terms that allow us to
discriminate good
listeners from bad ones. An additional, indirect, benefit accrued from
training is that we
have learned which program selections are most revealing of typical
frequency-related
artifacts introduced during the training exercises, and we now use these in
our product
evaluations.
4.0 CONCLUSIONS
In summary, we have described a new facility designed to test multichannel
components
efficiently and as bias-free as possible. The facility includes acoustically
transparent
listening screens that hide the identities of all multichannel loudspeakers
and equipment
within the audio path. Particular attention has been taken to address the
two of the most
problematic variables in listening tests: the listening room and the
position(s) of the
loudspeaker. Through the use of a computer automated speaker shuffler, we
have greatly
reduced the amount of time and effort required to set up and test multiple
comparisons
between loudspeakers by reducing the factor position to a one-dimension or
level
variable. Typical loudspeaker evaluations should be reduced in length by a
factor of 24:1.
The listening room itself is capable of testing up to three different 5.1 or
7.1 channel
systems and accommodate 1-6 listeners at a time. The measurements we have
shown in
this paper indicate its performance in its current form meets the very
highest standards set
out by the ITU and EBU recommendations, in terms of volume, geometry,
reverberation
time, and the control of early reflections. The acoustics of the room can be
easily altered
from hemi-anechoic to more typical domestic room conditions by adding
reflective
panels to the room's boundaries.
Finally, the experimental design, set up and control are computer-automated
so that
experiments can be easily repeated, and are less prone to human error. The
more timeconsuming
and mundane tasks such as collection and analysis of data have also been
computer-automated, so that experiment report writing becomes a simple
cut-and-paste
operation.
5.0 ACKNOWLEDGEMENTS
The authors would like to thank Tom Roberts of Bruel & Kjaer for his
assistance and
loan of the equipment used to make the background noise measurements shown
in this
paper.
13

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
6.0 REFERENCES
[1] F.E. Toole, "Listening Tests - Identifying and Controlling the
Variables", Proceedings
of the 8th International Conference, Audio Eng., Soc. (1990 May).
[2] F.E. Toole and S.E. Olive, "Hearing is Believing vs. Believing is
Hearing: Blind vs.
Sighted Listening Tests and Other Interesting Things", 97th Convention,
Audio Eng.
Soc., Preprint No. 3894 (1994 Nov.)
[3] F.E. Toole, "Listening Tests, Turning Opinion Into Fact", J. Audio Eng.
Soc., vol. 30,
pp. 431-445 (1982 June).
[4] F.E. Toole, "Subjective Measurements of Loudspeaker Sound Quality and
Listener
Performance", J. Audio Eng. Soc., vol. 33, pp. 2-32 (1985 January/February).
[5] Soren Bech, " Perception of Timbre of Reproduced Sound in Small Rooms:
Influence
of Room and Loudspeaker Position J AES, Vol. 42, Number 12 pp. 999 (1994).
[6] S.E. Olive, P. Schuck, J. Ryan, S. Sally, M. Bonneville, "The
Variability of
Loudspeaker Sound Quality Among Four Domestic-Sized Rooms", presented at the
99th
AES Convention, preprint 4092 K-1 (1995 October).
[7] F.E. Toole, "Loudspeakers and Rooms for Stereophonic Sound Reproduction",
Proceedings of the 8th International Conference, Audio Eng., Soc. (1990
May).
[8] S.E. Olive, P. Schuck, S. Sally, M. Bonneville, "The Effects of
Loudspeaker
Placement on Listener Preference Ratings", J. Audio Eng. Soc., Vol. 42, pp.
651-669
(1994 September).
[9] Antti Jarvinen, Lauri Savioja, Henrik Moller, Veijo Ikonen, Anssi
Ruusuvuori,
"Design of a Reference Listening Room - A Case Study", AES 103rd Convention,
New
York, Preprint 4559, September 26-29, 1997.
[10] IEC Publication 268-13: Sound System Equipment, part 13. Listening
Tests on
Loudspeakers (1985)
[11 NR-12 A, Technical Recommendation: Sound Control Rooms and Listening
Rooms.
2nd Edition, The Nordic Public Broadcasting Corporation, 1992.
[12] ITU-R Recommendation BS.1116: Methods for Subjective Evaluation of
Small
Impairments in audio systems including multichannel sound systems, 2nd
Edition (1997)
[13] ITU-R Recommendation BS.775: Multichannel stereophonic sound system
with and
without accompanying picture (1994).
[14] EBU Tech 3276 (2nd Edition, 1997).
[15] AES20-1996: Recommended Practice for Professional Audio - Subjective
Evaluation of Loudspeakers (1996).
[16] Walker, R. "Optimum Dimension Ratios For Small Rooms". 100th AES
Convention.
Preprint 4191 (Copenhagen, Denmark, 1996).
[17] S.E. Olive and F.E. Toole, "The Detection of Reflections in Typical
Rooms", J.
Audio Eng., Soc., vol. 37, pp. 539-553 (1989 July/August).
[18] R.Walker," A controlled-reflection listening room for multichannel
sound", AES
104th Convention Amsterdam, The Netherlands, Preprint #4645, May 16-19, 1998
14

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
[19] E. Arató Borsi, T. Póth, and A. Fürjes," New Reference Listening Room
for Two-
Channel and Multichannel Stereophonic" AES 104th Convention Amsterdam, The
Netherlands, Preprint #4732, May 16-19, 1998.
[20]Soren Bech,"Selection and Training of Subjects for Listening Tests on
Sound-
Reproducing Equipment" Vol. 40, Number 7 pp. 590 (1992).
[21] S. E. Olive, "A Method for Training of Listeners and Selecting Program
Material for
Listening Tests", 97th Convention, Audio Eng. Soc., Preprint No. 3893 (1994
November).
TABLE 1
Parameter Harman ITU EBU N12-A IEC AES
MLL
Volume 155.92 60-110 50-120
( m 3 ) (80)
Floor area 60.20 20-70 40 60 ± 10 20
( m 2 )
Height 2.59 2.3 - 3.0 rec. 2.1
h (m) 2.8
Length 9.14 = 6
l (m) rec. 6.7
Width 6.58 = 4
w (m) rec. 4.2
(1.1 w / h) 2.80
( l / h) 3.53
( 4.5w / h - 4 ) 7.44
T m (s) 0.23 0.29 0.29 0.35 0.3 -0.6 0.45 ± 0.15
± 0.05 0.4 ± 0.05
T 63 Hz Max .34 Tm(s) 0.2 - 0.4 0.35 0.8
(s)
Noise Level NR 5 NR10; NR10;NR 10 L pA L pA 35 dB
abs. max abs. max or L pA 35 dB andNR 15 NR 15 15 dB L pC 50 dB
Table 1: Dimensions and Acoustic Parameters of Harman MLL versus
Recommendations of Various Standards
15

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
0
10
20
30
40
50
60
70
32 63 125 250 500 1000 2000 4000 8000
)
SPL ( )
-10
Frequency (Hz
dBAC ON
AC OFF
NR0
NR5
NR10
NR15
Figure 1 A spatially-averaged measurement showing the background noise in
the MLL
with the air conditioning off (dotted) and turned on (dashed) compared to
the NR curves:
0,5,10 and 15.
0
63 250 500 2000
()
T60 (seconds)
in
0.1
0.2
0.3
0.4
0.5
0.6
0.7
125 1000 4000 8000
FrequencyHzEBU & ITU OPT.
EBU & ITU Max
EBU & ITU MMLL
Figure 2 The Tm (RT60) values measured in the MLL compared to the optimal,
maximum and minimum values recommended by the EBU and ITU standards.
16

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
()
0
20
40
60
80
100
120
1 2 3 4
/o
/o
/o
/o
Minimum Number of Trials Without Speaker MoverNumber of Loudspeaker
Positions
Compared
1 program w2 program s w3 program s w4 program s w
Figure 3(A) The above graph shows the number of trials required for a
multiple
comparison loudspeaker experiment as a function of the number loudspeaker
positions
compared. The lines represent experiments in which 1-4 programs are used.
The design
balances all position and context effects and has no repeats.
()
0
2
4
6
8
10
12
14
16
18
20
1 2 3 4
Minimum Number of Trials With Speaker MoverNumber of Loudspeaker Positions
Compared
1 program w.
2 programs w.
3 programs w.
4 programs w.
Figure 3(B) The same experiment is shown as in Figure 3(A) above except here
a
speaker mover is used.
17

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
Figure 4 Shown is the automated speaker shuffler of the MLL set up for A/B
stereo testing of two stereo
loudspeakers. Here the front listening screen is pulled up.
Figure 5 A front-left wide-angle shot of the MLL with the listening screens
pulled back.
The automated speaker shuffler is in the foreground setup for 5.1 playback.
Note the side
and rear channel speaker baffles in the background, and the audio and
computer data
control box on the back wall. The video projector is mounted on the ceiling
with a
retractable screen in front of the speaker mover.
18

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
Figure 6 A listener performing a test by entering their data on a laptop
computer that is
networked to the NT Server. In this test, video is displayed and both front,
side and rear
curtains are drawn to hide the identifies of the 5.1 loudspeaker systems
under test.
Figure 7 Shown is the control room area outside the listening room where all
audio
equipment,experimental control and monitoring takes place. Shown here is the
NT Server
on the left, and two listener training workstations on the right.
19

Harman International Industries, Incorporated 8500 Balboa Blvd., PO Box
2200, Northridge, CA 91329 (818) 893-8411
Figure 8: The GUI of the listener training software. The listeners' task is
to match the 4 different
equalizations indicates by their frequency response curves that are randomly
assigned to Buttons A-D
Feedback is given on their responses. The "FLAT" button allows listeners to
audition the program
without any equalization added.
Figure 9: The GUI of the software used for a typical listening test or
training exercise. Listeners
enter their preference ratings for sounds A-D relative to a given reference
("REF"). Ratings are also
given on spectral balance and distortion. Relevant comments are optional.
20

Appendix 1: Block Diagram of Harman Multichannel Listening Laboratory (MLL)
showing key features, equipment and path for audio,
video, data and control signals.
Speaker Mover ateProceed
(Controlled via Amplifier
RS-232) (16 channels)
2200, N Studer 8411
D/A Converter
(16 channels)
Sony PCM800
Curtain Stewart
DVD
Video Screen
Flat Panel
Display
Laser Disk
Wireless
Infrared
Analog Audio Mouse
+
Digital Audio Keyboard
Lexicon DC-1
Video
Listeners'
Computer Data
MIDI Data
Laptops
Computer Card
Spirit Digital
328
Faroudja Lexicon
Front Line I/O Box
Projector
Doubler
MIDI Card
Ethernet WINDOWS
HUB NT SERVER
Lexicon Studio
Core PCI Card
RS232/IR /RF
21
Controller

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net wrote
in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



If you don't like what I write, don't read it.

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net wrote
in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.

"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

On Mon, 14 Nov 2005 00:53:45 GMT, wrote:


"George M. Middius" cmndr [underscore] george [at] comcast [dot] net wrote
in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.

I've got a modern newsreader that has the number of lines listed right
next to the subject in the message list pane (and George has that same
newreader). You should try such a newsreader sometime. It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution to
the "The case for ABX..." thread. I thought you might be unwell or on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT medical
research before any of you heard the name. Medical DBT research can be
validated by objective data (psyche is an exception) 2) DBT is DBT,
ABX is ABX 3) I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind. 5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB. But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

wrote in message
oups.com...

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution to
the "The case for ABX..." thread. I thought you might be unwell or on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT medical
research before any of you heard the name. Medical DBT research can be
validated by objective data (psyche is an exception) 2) DBT is DBT,
ABX is ABX 3)

ABX is a double blind protocol used by people doing audio research from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and on
and on.


I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind.

Take whatever you want you still haven't a clue about ABX for audio.

5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB.

You mean you've finally accepted that ABX is a valid protocol used to test
for subtle audio differences?

But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

I know that ABX is a form of DBT, something you don't seem to get.
It is also a widely used and valid protocol, something else you don't seem
to get.

"dave weil" wrote in message
...
On Mon, 14 Nov 2005 00:53:45 GMT, wrote:


"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote
in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.

I've got a modern newsreader that has the number of lines listed right
next to the subject in the message list pane (and George has that same
newreader). You should try such a newsreader sometime.

Why? I don't care about how many lines in a post.


It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Yawn.

"Arny Krueger" wrote in message
...
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the difference between
audio knowlege and posturing, Middius.

Odd since he's the one doing so much of the posturing.

On Mon, 14 Nov 2005 07:58:02 GMT, wrote:


"dave weil" wrote in message
.. .
On Mon, 14 Nov 2005 00:53:45 GMT, wrote:


"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote
in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.

I've got a modern newsreader that has the number of lines listed right
next to the subject in the message list pane (and George has that same
newreader). You should try such a newsreader sometime.

Why? I don't care about how many lines in a post.

It would stop you from making stupid statements about "wasting time
counting the number of lines".

It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Yawn.

I guess having your ignorance exposed again makes you sleepy.

Cool.

The Krooborg puffs up its scrawny chest.

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing

Speaking of showing, Turdy, how about you post that picture of your "system"
again so the newest generation of RAO spectators can establish an accurate frame
of reference for your vast "audio knowedge"?



..
..

wrote in message
nk.net...
Since we all know you are in desparate need of more information on how and
why DBT's are used by Sean Olive and Floyd Toole I include the following
for your edification.

They make crap.
It has been proven that ABX is used by these companies to determine the
poorest quality that the public will accept.
ABX has been proven not to work in hifi applications.

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



If you don't like what I write, don't read it.

You are wasting usenet bandwidth. You are a bad usenet netizen.

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.

Mikey, you are being repetitious. We covered this last month. We all know
what the other side thinks.

wrote in message
ink.net...

"dave weil" wrote in message
...
On Mon, 14 Nov 2005 00:53:45 GMT, wrote:


"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote
in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.

I've got a modern newsreader that has the number of lines listed right
next to the subject in the message list pane (and George has that same
newreader). You should try such a newsreader sometime.

Why? I don't care about how many lines in a post.


It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Yawn.
Don't do this Mikey. You are a bad person, as well as stupid.

wrote in message
ink.net...

wrote in message
oups.com...

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution to
the "The case for ABX..." thread. I thought you might be unwell or on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT medical
research before any of you heard the name. Medical DBT research can be
validated by objective data (psyche is an exception) 2) DBT is DBT,
ABX is ABX 3)

ABX is a double blind protocol used by people doing audio research from
the BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies,
and on and on.


I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind.

Take whatever you want you still haven't a clue about ABX for audio.

5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB.

You mean you've finally accepted that ABX is a valid protocol used to test
for subtle audio differences?

But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

I know that ABX is a form of DBT, something you don't seem to get.
It is also a widely used and valid protocol, something else you don't seem
to get.
We know what it is, and what it is used for. Not for hifi.

wrote in message
nk.net...

"Arny Krueger" wrote in message
...
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the difference between
audio knowlege and posturing, Middius.

Odd since he's the one doing so much of the posturing.
We know who Harmon is, and what they make, which is barely acceptable
equipment.
Ludovic has shown you that ABX is an invalid protocol for hifi.

Robert Morein said:

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

What the **** are doing wasting time counting the number of lines, get a
hobby already.

It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Yawn.

Don't do this Mikey. You are a bad person, as well as stupid.

Mickey is following his and Arnii's religious rituals. Stupidity is one of their
chief articles of faith. It ranks right up there with hypocrisy. A primo example
of both the stupidity and hypocrisy of the 'borgs is their fanatical devotion to
the depraved aBxism torture rituals. Their mindless, monotonous flogging of this
intellectually bankrupt "testing" regimen is stupid because all the extant
evidence shows it's worthless for consumers. In fact it's actually worse than
worthless because any consumer who undertakes the blinding rituals not only
doesn't learn anything useful, he also wastes his own time and/or money.

The hypocrisy is even more salient: None of the 'borgs has EVER done an aBxism
ritual before making a purchase for their own use. And yet they yammer on
endlessly about how perfect the rituals are, how indispensable for "saving
money". Hypocrisy of this intensity is worthy of the most two-faced politicians
or potentates of organized religion. It reminds me of Jim Bakker and Tom Delay.


..
..
..

"dave weil" wrote in message
...
On Mon, 14 Nov 2005 07:58:02 GMT, wrote:


"dave weil" wrote in message
. ..
On Mon, 14 Nov 2005 00:53:45 GMT, wrote:


"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote
in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.

I've got a modern newsreader that has the number of lines listed right
next to the subject in the message list pane (and George has that same
newreader). You should try such a newsreader sometime.

Why? I don't care about how many lines in a post.

It would stop you from making stupid statements about "wasting time
counting the number of lines".

It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Yawn.

I guess having your ignorance exposed again makes you sleepy.

Cool.

Talking about how many lines in a post and the software that does it make me
bored.

No surprise you guessed wrong, again.

"George Middius" wrote in message
...


Robert Morein said:

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

What the **** are doing wasting time counting the number of lines, get
a
hobby already.

It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Yawn.

Don't do this Mikey. You are a bad person, as well as stupid.

Mickey is following his and Arnii's religious rituals. Stupidity is one of
their
chief articles of faith. It ranks right up there with hypocrisy. A primo
example
of both the stupidity and hypocrisy of the 'borgs is their fanatical
devotion to
the depraved aBxism torture rituals. Their mindless, monotonous flogging
of this
intellectually bankrupt "testing" regimen is stupid because all the extant
evidence shows it's worthless for consumers. In fact it's actually worse
than
worthless because any consumer who undertakes the blinding rituals not
only
doesn't learn anything useful, he also wastes his own time and/or money.

To that, Ludovic cites considerable evidence that ABX, as currently
practiced, is useless for research in the "high end". He cites examples of
amps known to be mediocre, which were indistinguishable in an ABX "test"
from amps known to be superior.

As chocolate dulls and confuses the palate, so does ABX dull and confuse the
ear.

"Arny Krueger" wrote in message
...
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the difference between
audio knowlege and posturing, Middius.
Krueger replies with #27, "Thanks for showing".

It appears that Arny's entire phraseology fits neatly into five bits.
The other three bits are presumably used by Krueger to define various Borg
power-up states, which are strikingly similar to ACPI.

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



If you don't like what I write, don't read it.

How would he know if he liked it, until
after he read it? Oh, I get your message,
just look at the specs such as line length, rather than
actually read (and listen to) the contents.
Its the Mikey way!

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.


2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??

"George Middius" wrote in message
...


Robert Morein said:

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

What the **** are doing wasting time counting the number of lines, get
a
hobby already.

It took me
about a nanosecond to see the number of lines in the post that Mr.
Middius is referencing, although he rounded down.

Yawn.

Don't do this Mikey. You are a bad person, as well as stupid.

Mickey is following his and Arnii's religious rituals. Stupidity is one of
their
chief articles of faith. It ranks right up there with hypocrisy. A primo
example
of both the stupidity and hypocrisy of the 'borgs is their fanatical
devotion to
the depraved aBxism torture rituals. Their mindless, monotonous flogging
of this
intellectually bankrupt "testing" regimen is stupid because all the extant
evidence shows it's worthless for consumers. In fact it's actually worse
than
worthless because any consumer who undertakes the blinding rituals not
only
doesn't learn anything useful, he also wastes his own time and/or money.

The hypocrisy is even more salient: None of the 'borgs has EVER done an
aBxism
ritual before making a purchase for their own use. And yet they yammer on
endlessly about how perfect the rituals are, how indispensable for "saving
money". Hypocrisy of this intensity is worthy of the most two-faced
politicians
or potentates of organized religion. It reminds me of Jim Bakker and Tom
Delay.

and John Kerry.

"George Middius" wrote in message
...



The Krooborg puffs up its scrawny chest.

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing

Speaking of showing, Turdy, how about you post that picture of your
"system"
again so the newest generation of RAO spectators can establish an accurate
frame
of reference for your vast "audio knowedge"?


he can't.
His wife threw it out in last years spring cleaning

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.


2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??
Just don't care. Is there some reason I should?

wrote in message
nk.net...

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get a
hobby already.


2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??
Just don't care. Is there some reason I should?


Yes indeed, you were mouthing off about them,
without knowing anything about them.

wrote:
wrote in message
oups.com...

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution to
the "The case for ABX..." thread. I thought you might be unwell or on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT medical
research before any of you heard the name. Medical DBT research can be
validated by objective data (psyche is an exception) 2) DBT is DBT,
ABX is ABX 3)

ABX is a double blind protocol used by people doing audio research from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and on
and on.


I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind.

Take whatever you want you still haven't a clue about ABX for audio.

5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB.

You mean you've finally accepted that ABX is a valid protocol used to test
for subtle audio differences?

But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

I know that ABX is a form of DBT, something you don't seem to get.
It is also a widely used and valid protocol, something else you don't seem
to get.
__________________________________________________ _________

ABX is a double blind protocol used by people doing audio research
from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and on
and on.

BBC-- no. Quote documentation and show me for a liar or you for a
....what?
B&O- no. Ditto, Ditto, Ditto
Nokia that famous audio company uses it for comparing cellphones or
what? Documantation?
Harman -no. Ditto, Ditto, Ditto
Have you got the distinction between DBT and ABX clear in your head
yet?
NYOB, NYOB, you make my day.
Ludovic Mirabel

wrote in message
oups.com...

wrote:
wrote in message
oups.com...

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution to
the "The case for ABX..." thread. I thought you might be unwell or on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT medical
research before any of you heard the name. Medical DBT research can be
validated by objective data (psyche is an exception) 2) DBT is DBT,
ABX is ABX 3)

ABX is a double blind protocol used by people doing audio research from
the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and
on
and on.


I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind.

Take whatever you want you still haven't a clue about ABX for audio.

5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB.

You mean you've finally accepted that ABX is a valid protocol used to
test
for subtle audio differences?

But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

I know that ABX is a form of DBT, something you don't seem to get.
It is also a widely used and valid protocol, something else you don't
seem
to get.
__________________________________________________ _________

ABX is a double blind protocol used by people doing audio research
from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and
on
and on.

BBC-- no. Quote documentation and show me for a liar or you for a
...what?

Audio Info
Audio Information
Audio Info | Search | Statistics | Contact Us

You are logged in as a guest. ( logon | register )


A bit from the BBC (Long)


Audio Info- Audio Opinion

Michael McKelvy
Posted Jan 27, 2005 21:53:45
Subject: A bit from the BBC (Long)

Guest





The selection of loudspeakers for BBC Radio & Music.


R. Walker, BBC Research and Development Department.



1 Introduction.

The monitoring loudspeaker is unlike any other part of an audio
production
chain. It is the means by which the audio signal is judged. Other
parts of
the production chain certainly have effects, but they can be judged,
altered
or adjusted based on the final sound as reproduced by the
loudspeakers.
Historically, the BBC satisfied its requirements for studio monitoring
loudspeakers by designing and developing its own range. The
requirements
could not generally be met at that time by contemporary commercial
products,
though some were used in special cases.



Apart from the obvious need for high quality reproduction, the large
number
of studios and control rooms (more than 600 in the period around 1980)
meant
that the quality and character of the audio reproduction had to be
maintained over a large number of units, over a range of loudspeaker
sizes
and for periods of many years. One of the main reasons for requiring
consistency was (and still is) production practices that require
operational
staff to move between rooms for different jobs and for programme audio
to be
processed consecutively in different facilities (especially in
television).
Both of those factors require the sound monitoring in different rooms
to be
as similar as possible, otherwise much time and effort can be spent in
doing
and re-doing changes to the recorded material. That is unlike a
commercial
recording studio or small radio/TV station, where the technical staff
work
almost all the time in the same few rooms and can become familiar with
their
eccentricities.



Those considerations led, over the years, to a succession of
BBC-designed
loudspeakers that also achieved substantial national and international
recognition. The BBC-designed loudspeakers introduced several new
developments, such as the use of polyethylene/ polypropylene (PE/PP)
co-polymers for the diaphragm and Kapton? voice coil formers. The unit
costs
of BBC-designed loudspeakers were also significantly lower than
equivalent
commercial products, at that time.



The last successful BBC-designed loudspeakers were the small LS3/5A
(c.
1975), the large LS5/8 (c. 1982) and the medium LS5/9 (c. 1984). Those
loudspeakers were the main ones in use from the dates of their
original
designs to the present. Very many are still in use at the present time
(2004). In design, they may be well past their time and are, in any
case,
now irreplaceable. This author suggested in about 1992 that work
should be
started on their successors. At that time, the loudspeaker designs
were
between 8 and 17 years old. They are now between 20 and 29 years old.



Three main factors led to the demise of the loudspeaker development
work.
With the coming of a more commercial imperative to the BBC as a whole,
it
was no longer considered worthwhile to develop loudspeakers
internally. At
that time (around 1994), the individual Directorates were made
autonomous
and responsible for managing themselves. That led to a lack of support
for
the sort of centralised activities needed to support loudspeaker
development. Also, each business unit was isolated and indeed had been
deliberately set up to be in internal competition with other units.
The
choice of loudspeakers was seen as an individual selection, based on
each
unit's own activities or cost considerations. That policy has led,
over a
period of about the last 10 years, to significant fragmentation, with
many
different types of loudspeakers being used in different areas. That
has in
turn led to the recognition (at least in some places) of the value of
that
earlier requirement for consistency.



A second factor has been the availability of improved commercial
loudspeakers. Over the period of about the last 40 years there is no
doubt
that commercial loudspeakers have improved dramatically. Even over the
last
20 years since the design of the LS5/9, developments in commercial
loudspeakers have led to improved properties, perhaps most of all in
consistency.



The third factor is the cost and complexity of modern loudspeakers.
Years
ago, a loudspeaker consisted of a few relatively simple components. Of
course, the sound quality, reliability and other parameters depended
on the
choice of materials and their physical forms, but the principles were
straightforward enough for a relatively small team to succeed in the
design.
Now, the additional refinements, the improvement in achievable quality
and
the complex design techniques make it less practicable for a small
team to
embark upon the development with any realistic hope of success.
Designing
high quality loudspeakers is an activity that can only be contemplated
by a
well-resourced team [1]. The addition of modern complex electronics,
for
room equalisation, for allowance for component variations or for the
addition of digital control or input interfaces, further adds to the
potential scale of the task.



BBC Radio and Music (BBC R&M) considered that the scale of the project
made
it necessary to carry out a study to select a range of loudspeakers
with the
objective of setting up a Framework Agreement for the procurement of
'standard' loudspeakers. They invited News and World Service to take
part,
so that they could take advantage of the results if they wished. The
size of
the requirement made it necessary to go through a formal tendering
process.
It also exceeded the EU threshold, which meant that the EU Procedure
(Restricted) process had to be applied.



In order to include the quality of the loudspeaker as a selection
parameter
it was necessary to carry out formal listening tests. Even without
that
potential legal requirement, it was thought desirable to have a
uniform
loudspeaker installations throughout the new development. That
required all
potential user groups to support the selection process and to accept
the
outcome. It was also hoped that other Directorates might accept the
idea of
a new set of 'standard' BBC loudspeakers for future developments. Of
course,
under the existing separations between Directorates, there is no
guarantee
that will be so, but it would clearly be beneficial to the Corporation
as a
whole if it helped to keep down costs and contribute to uniformity of
output.





2 The tendering process.

In outline, the 'Restricted' EU tendering process consists of two
stages.
The first is an advertisement for potentially interested parties. The
responses to the advertisement were studied and used to create a short
list
for the subsequent 'Invitations to Tender'.



The Invitation called for proposals for three different sizes of
loudspeaker
to replace the LS5/8, LS5/9 and LS3/5A. It was made clear that the
audio
quality selection would be on the basis of subjective assessment
alone[2].
However, a brief technical specification was included in the
Invitation as
guide to the general performance requirements. The subjective
assessment
was to be made on the basis first of absolute audio quality for each
loudspeaker size and then on the 'family' resemblances between the
three
different sizes[3]. The set of three loudspeakers was considered as a
family
of products needed to fulfil the BBC requirements. They did not
necessarily
all have to be provided by a single manufacturer.



The first level of assessment was the response to the initial
advertisement.
The accompanying questionnaire covered aspects of the supply of
loudspeakers
not directly related to their acoustic performance, such as company
size and
financial controls, staff health and welfare and company history. That
was
intended to eliminate commercially inappropriate proposals. At that
stage,
there was no exclusion of any potential supplier who might otherwise
provide
suitable loudspeakers. All respondents who met the commercial
requirements
were included, even though that resulted in more loudspeakers to be
included
in the assessments than had been thought practicable at the outset.
Some of
the proposals included the same loudspeakers in different
combinations. It
was clear that the final total of 19 different proposals was too many
to
assess. By removing the repetitions and the impractically expensive
loudspeakers, the total number of proposals was reduced to 10. Each of
the
proposals included submissions for all three of the target sizes, so
the
total number of loudspeaker pairs actually to be assessed subjectively
was
nine large, ten medium and nine small (28 in total).





3 Test methodology.

3.1 The Parameters of an Ideal Test.

A rigorous subjective test should consist of double blind, individual
assessments using a suitable number of descriptive axes by a suitable
number
of test subjects. Probably five to ten subjective axes would be used.
That
is likely to be higher than the true number of different quality
attributes,
but would allow some scope for subsequent statistical factor
reduction.
Other tests doing similar things have commonly used around 20
descriptors.
If the set of descriptors is too limited then test subjects feel
constrained
whilst having too many leads to confusion and overload of the test
subjects.



It was thought that not less than 10 test subjects would be needed, if
they
were highly skilled and could demonstrate a reasonable degree of
consistency. For less able subjects, at least two or three times that
number
would have been necessary. The subjects were also being asked to
assess many
different aspects of loudspeaker quality at the same time. Less expert
subjects would have found that difficult. Because of the background to
the
tests, it was thought that the test subjects should also be, in some
sense,
'chosen' by their colleagues, so that the tests would be seen as
reasonable
and the final results more readily accepted by the majority of
production
staff who had not taken part directly.



The total number of loudspeakers was also an issue. At the outset, it
had
been anticipated that as many as eight suppliers might provide samples
of
each of the three sizes, resulting in 24 loudspeakers (stereo pairs)
to be
assessed. In the event, the final number was 28 pairs. Some dummy or
repeat
assessments also had to be included. In particular, the first
assessment
each day had to be rejected because there was no way of indicating to
the
test subjects the likely range of quality. Subjective tests, at least
absolute assessments, usually include 'anchor points' to guide the
subjects
in the range of responses to be expected. That was not feasible in
this case
because the loudspeakers had to be moved between presentations. It had
also
been decided at the outset that the existing range of BBC loudspeakers
would
not be included as references because that would unfairly bias the
results
towards the existing and familiar products. It was felt that the
proposals
should be assessed entirely on their own merits, not in comparison
with
existing products. With one dummy test added to each set, the final
total
number of quality assessments was 31.



The time allowed for each person's assessment also has to be
reasonable. It
was thought unreasonable to ask subjects to begin immediately with
completely unknown material so some time was allowed for
familiarisation
using the subject's own material. It was estimated that the
familiarisation
and assessment could not be done in less than 30 minutes.



In loudspeaker listening, the acoustics of the room have a significant
influence on the perceived sound. That, at least, would be the same
for all
loudspeakers. However, more significantly, the position of the
loudspeaker
in the room also has a profound effect. That meant the assessments had
to be
done with all of the loudspeakers in the same positions in the room
and that
each test had to include changing the physical positions of the
loudspeakers. In addition, some people think that having another
loudspeaker
present anywhere in the room affects the perceived sound[4]. That
meant not
only did the loudspeakers have to be changed in position but actually
removed from the room when not being assessed. All of that had to take
place
without the subject knowing what was happening. In practice, the test
subjects would have to be moved to a separate waiting area while the
loudspeakers were changed over and the new ones connected, tested and
their
levels set. All of that was thought likely to add at least 30 minutes
to
each assessment.



So, it was thought that a full set of tests would consist of about 240
assessments. With 60 minutes each that would total about 240 hours of
testing, or about 30 working days.



There were two possibilities :-

1) One test could be carried out per day with one loudspeaker and all
ten
subjects. With the final number of loudspeakers, that would have taken
a
total of 28 days to test all of the loudspeakers. It would have meant
all
ten subjects attending the test venue on 28 separate occasions for one
hour
on each occasion. Taking into account the significant travelling time,
even
from central London never mind from the Regions, that would have been
logistically impossible.

2) Alternatively, each of the test subjects could be present at the
test
venue for an extended period and the loudspeakers changed over. About
eight
tests per day might have been achieved. That would have made a total
of
about four days. The total time for the testing would therefore have
been
about 40 working days. However, it would have been utterly exhausting
for
the project team, who would have to arrange the loudspeakers in the
room a
total of 310 times and probably too exhausting for the subjects (who
would
have to concentrate very hard for most of the four days, admittedly
with
reasonable breaks every 30 minutes or so).



Both of these ideal options were considered to be unworkable. Option
'1' had
the serious problem that no dummy or 'ranging' tests could be sensibly
included. Each test subject would listen to only one loudspeaker each
day
and would have to carry mental recollections of the quality over a
period of
more than a month. That was clearly not sensible. Option '2' had the
advantage of minimising the commitments for the test subjects, but
involved
an unacceptable amount of work for the project team. Both of the
options
would have taken too much time overall. It was clear that the
procedure was
going to have to be compromised in some way in order to make the tests
practicable.





3.2 Possible Simplifications and Their Defects.

A number of considerations applied to the simplification of the test
procedu-

1. Whatever else happened, there had be some randomisation of the
order of
presentation to reduce the effects of order as a factor in the
results. If
that were not done, the results would depend very heavily on the
presentation sequence. Even expert listeners would find it difficult
to
avoid their assessment being relative to the previous samples rather
than
independent of them. That would be impossible with scheme '1' but
possible
to do with a test programme based on scheme '2'.



2. More than one loudspeaker pair could be put in the room at the same
time
in as near to the optimum positions as possible (restricted
significantly in
some cases because of the size of some of the loudspeakers). That
would
simplify the changeover. However, that would unfairly influence the
results
because of room position effects, apart from any questions over
interactions
between the loudspeakers.



3. The obvious simplification was to allow several subjects to assess
the
loudspeakers at the same time. If that were done without allowing them
to
change seating positions then the results would again be heavily
biased by
the test subject's seat position. If they were allowed to move around
during
the session then there would inevitably be some risk of collusion.
Collusion
leads to effectively fewer test subjects, especially if there are one
or two
dominant personalities in the room. However, the test subjects were
all
experienced operators, with independent opinions. It was thought
likely that
they could form their own views, independent of their colleagues. (In
the
event, the results showed that most test subjects did form reasonably
individual opinions.) That simplification would not have been
acceptable
with inexperienced test subjects.



4. Given the time-scale and scope of the testing, the only practicable
test
procedure would be to have all the sample loudspeakers on site and to
invite
all of the test subjects to be at the test venue for an intensive
period of
three or four days.



5. It was not possible to carry out the tests entirely 'double-blind'.
Inevitably, the project team would know which loudspeakers were being
assessed. However, only one member of the project team remained in the
room
to supervise the tests - to ensure that the test subjects did not
confer or
try to look behind the screen to identify the loudspeakers. That
person was
often actually unaware of the loudspeaker manufacturer because code
letters
were used to identify the loudspeakers. In any case, under the
pressure of
physically moving the loudspeakers and setting the levels, the members
of
the project team had little time (or inclination) to take note of the
identity. The supervisor was also under strict instruction not to
exhibit
any reactions.





3.3 The Final Test Procedure - Absolute Quality.

The final procedure for the quality tests consisted of one day's
testing for
each of the three sizes of loudspeaker. Each day consisted of 10 or 11
assessments (depending on the number of loudspeakers in the group). On
occasions, errors in set up procedure or defects in equipment caused
some
assessments to be re-started. When the fault was clearly noticeable to
the
subjects and made it unrealistic to continue, for example one channel
not
working or a 'ringing' loudspeaker stand [5], the test was re-started
immediately. When the fault was not directly evident, for example if a
mistake had been made in the overall level setting, the whole test was
repeated without the test subjects being aware and the first results
discarded. The final programme allowed 45 minutes for each assessment,
30
minutes for the actual assessment and 15 minutes for the technical
changeover. Longer breaks were included for refreshments and meals.
Overall,
the test programme was very demanding of both the project team and the
subjects, with additional breaks having to be added sometimes.



The test panel was nominally 12 subjects, but not all were present all
of
the time. Eleven test subjects attended all of the tests. Because it
was
thought that 12 at once were too many for the room, the test panel was
split
into two groups. The whole test procedure for quality assessment was
therefore carried out twice, once for each group. The two test
sessions were
held in mid-December 2003 and early January 2004. The orders of
presentation
were different for the two groups.



The reproduction level was controlled to within ?0.5 dB using a
pre-recorded
pink noise track on the test disk and a Bruel and Kj?r 2231 Precision
Sound
Level meter set to 'flat', located at the central front seat position.
The
pink noise recording was band-limited to 200 Hz - 8 kHz, removing the
lowest
frequencies to provide fair comparison signals for the smaller
loudspeakers.
The individual tracks of the test disk had been set subjectively to
appropriate levels for the type of material by members of the project
team.
During the tests it was not necessary to adjust the levels. The test
disk
was simply allowed to play from beginning to end, with 13 tracks of
about 1
minute each. A period of about 10 minutes was allowed before the
formal test
for the test subjects to use their own material for familiarisation.



The quality assessments were carried out using two-channel
stereophonic
material, though some tracks were presented in left or right channels
only.





3.4 The Test Material.

The test material was selected to cover a range of genres. It included
male
and female speech, drama and several musical items. Not all of the
material
was the highest quality. Monitoring loudspeakers have to be able to
present
defective material accurately in order for the defects to be
identified.
Each extract lasted for about 1 minute. Thirteen tracks were finally
selected for presentation. The recording medium was audio CD. Replay
was by
Studer A730 CD player (digital output) with a Prism 'Dream DA-1'
digital-to
analogue decoder. Most of the loudspeakers included built-in
amplifiers. For
the remainder, analogue power amplifiers were supplied by the system
proposers. All of the systems tested used analogue audio inputs to the
loudspeakers/amplifiers.





3.5 Absolute Quality Assement Parameters.

The absolute quality was assessed on a total of 19 individual
parameters
divided into five main attributes and an overall score. The parameters
were
:-

SPECTRAL UNIFORMITY - uncoloured tonal reproduction of the audio
material,
evenness of frequency response, granularity of frequency response,
adequate,
but non-emphasised low frequency response, adequate, but
non-emphasised high
frequency response.

SOUND-STAGE IMAGING - Correctly located sound stage, continuous, even
and
wide left-right sound stage, localisation and separation of individual
sound
images, stable and open sound stage.

AMBIENCE REPRODUCTION - spacious and diffuse reverberant sound,
rendition of
direct-to-reverberant sound ratio, rendition of recording sound space,
uncoloured reproduction of reverberation.

DYNAMICS AND DISTORTION - consistency of performance at reduced level,

adequate rendition of material recorded at low level, level, purity
and
impact of loud sounds, lack of range compression and frequency
intermodulation.

OFF-AXIS PERFORMANCE - performance off the centre line, loudspeaker
plane or
with head movements.

SUMMARY - Overall Impression.

The 19 parameters were to be given a numerical score between 1 and 5,
with
'1' representing 'poor' and '5' representing 'good'. A space was left
at the
end for free-form comments.





3.6 Test Procedure - Family Resemblances.

The second stage of the tests, assessing family resemblances was
thought to
be technically less problematical. It was arranged as a follow-on
test,
after the loudspeakers had been put in order of preference by the
quality
assessments.



It was inescapable that all three of the loudspeakers being assessed
for
similarity should be in the room at the same time, to allow rapid
switching
between them. It was considered quite impossible for test subjects to
assess
or remember differences or similarities over longer periods of time.
The
absolute quality had already been decided, so that the effects of the
room
and the other loudspeakers on quality were less important. Because it
was
considered less critically dependent on the test conditions, all 11 of
the
test subjects were present at the same time. The three loudspeakers
being
compared were placed close together, so that room position effects
would be
minimised. The tests were carried out with monophonic signals, the
source
being the same test recordings with the two channels bridged together.



The scoring sheet for the assessment of family resemblance consisted
simply
of three boxes for the two-way comparisons between the large, medium
and
small loudspeakers. The points were awarded as +2 for "very good
match", 1
for "fairly good match", -1 for "slightly different" and -2 for "very
different". The instructions were to assess and score each of the
three
comparisons. There was no fixed time limit; the test subjects were
allowed
to continue until all were satisfied that they had produced a fair
assessment.



The selection of groups for the family resemblance tests was guided by
two
main factors. Firstly, the best individual loudspeakers should be
selected.
Secondly, if the first consideration did not lead to an optimum
selection
for family resemblance then loudspeakers not scoring the highest in
absolute
quality might have to be included. A third and minor consideration, to
be
used to resolve otherwise closely-matched selections, was that
preference
should be given to loudspeakers from the same manufacturer.



The family matching was considered to be a high priority. It was not
so high
that the absolute quality should be seriously compromised but high
enough to
allow for the rejection of the best individual loudspeakers. This was
expected to be a difficult compromise to achieve, and so it proved to
be.
The technique adopted was first to try a few obvious groupings and
then
discuss with the test subjects other possible groupings. The test
subjects
remained unaware of the identity of the loudspeakers throughout. All
discussion was carried out using the coded identifications.





4 Room acoustics and the test room.

The room to be used for the tests was the subject of much discussion
during
the preparations for the tests. In the past, there had been a
reasonably
clear view of what a typical BBC control room was like, acoustically.
Usually, they were (and many still are) heavily treated to produce
much
shorter reverberation times than most audience environments and to
control
early reflections by absorption. Many people have felt for some time
that
there should be a change in that basic room design. Indeed this
author, in
around 1991, began a process for change in the development of room
acoustics
that allowed much more 'lively' rooms, whilst still maintaining
control of
early reflections1,2,3,4,5. The new studio developments, and in
particular,
the development for which this loudspeaker selection process was being
carried out, were (allegedly) to include much less acoustic treatment.
It
was thought best if a room more like the proposed new rooms could be
used
for the loudspeaker selection.



In the event, that proved to be impossible. Firstly, it was difficult
to
find a room of any reasonable sort that was free for the necessary
period.
The use of reference listening rooms at BBC R&D had been ruled out on
the
grounds that they were too unrepresentative. Secondly, the messages
coming
from the design teams for new buildings suggested that the higher
quality
rooms at least would not be quite as different to present rooms as at
first
thought. That was more or less inevitable. Critical assessment of
sound
quality cannot be carried out in rooms with large areas of
specularly-reflecting surfaces and long and uneven reverberations
times,
however attractive they might be architecturally.



A room was eventually found that was available, centrally located and
that
had adjacent storage areas for the test loudspeakers[6]. The room had
been a
control cubicle for radio drama and so had the 'standard' BBC acoustic
treatment already installed, though it had been disused for some time
and
was visually unattractive. To modify the room acoustically, it was
made
acoustically symmetrical by the addition of similar acoustic treatment
over
the existing observation window and some of the wall treatment was
removed
where it would not result in discrete early reflections in the
listening
area. Some patches of high frequency absorption were added to areas
where
equipment had been removed and the resulting bare patches would have
caused
reflections. To hide the wall surfaces, to make the room somewhat more
attractive and thus not cause too much distraction for the test
subjects,
the entire inner wall surface was lined with lightweight fabric
drapes. The
final mid-band reverberation time was about 0.2 s.



To provide for blind testing, an 'acoustically transparent' screen was
erected across one end of the room, as an enclosure for the
loudspeakers and
amplifiers under test. The screen was made of lightweight black net
fabric
and lit by downlights so that the test subject could not see what was
behind. The acoustic transmission properties of several different
samples of
material were measured and the one showing the least acoustic effect
was
selected. It showed a uniform progressive fall in transmitted sound
level
with increasing frequency, amounting to about 1 dB at 20 kHz. That
would
probably have been detectable in a direct comparison but it was the
same for
all of the loudspeakers tested and was considered to be acceptable.





5 Results.

5.1 Absolute Quality.

The quality scores from the test subjects were averaged. That took
into
account the varying numbers of test subjects and their varying numbers
of
responses. It had not been insisted on that all subjects should score
all
attributes. Sometimes, subjects were unable to give numerical scores
for all
of the 19 attributes. Sometimes, they just gave an overall score for a
group
of attributes. In fact, there were only a very few instances of no
score
being given at all but there were perhaps 20% of cases where an
aggregate
score had been given for two or more attributes. The use of averages
allowed
the data processing to take account of those different numbers. The
scores
were averaged for the five quality attributes individually and
overall. The
overall scores were also calculated for the two groups of test
subjects
separately.



The results from the quality assessments are shown in Figs. 1 to 4.
Fig. 1
shows the overall average results for each of the loudspeakers for all
three
sizes. Figs. 2, 3 and 4 show the average results by attribute for each
of
the three sizes of loudspeaker. The 10 proposals were given code
numbers 1 -
10 and the three sizes were given code letters 'a' for large, 'b' for
medium
and 'c' for small. In those cases where the same loudspeaker had been
included in more than one proposal, the data was duplicated (e.g.
a4/a8 and
c4/c8). The loudspeakers themselves had not been assessed twice (as
discussed above).



Because the scale for scoring quality was from 1 to 5, an 'average'
loudspeaker would score 3. There was a strong tendency for the test
panel to
avoid scoring either very high or very low. Individual scores of 1
were not
given often, most test subjects probably thinking that they would
reserve '1'
for really poor loudspeakers, which did not in fact appear very often.
At
the other end of the range, scores of '5' were also comparatively
rare,
individuals probably reserving that for better loudspeakers than
actual
appeared. Certainly, there were many instances of loudspeakers being
given
scores of '1' (or even 0 in extreme cases) or '5' by individual test
subjects who particularly disliked or liked a specific loudspeaker.
However,
those very polarised results tended to be swamped in the averaging, so
that
the overall results tended to be compressed into the middle range. An
average of '2' would have actually indicated a poor quality
loudspeaker and
'4' an exceptionally good one [7]. Fig. 1 shows that, in fact, the
entire
range of average results for all three sizes was from 2.42 to 3.72.
For the
individual attributes, the total range of values was 2.08 to 4.0. That
illustrates the compression of the results into a reduced range.



In Fig. 1, some of the proposals scored either all poor (e.g. No. 7)
or all
good (e.g. Nos. 8, 9) over the range of sizes but in the majority of
cases,
the three sizes scored quite differently. That suggested that it might
be
difficult to select the best loudspeakers to make up family groups.



In Figs. 2 to 4, the individual attributes were more consistent.
Generally,
those loudspeakers with poor overall results also scored poorly in all
attributes, and vice versa. That was encouraging, as it showed
loudspeakers
could not be selected for particular attributes. Thereafter, only the
overall scores were taken into account. The converse of that would
have been
very difficult to manage. If it really had been the case that some
loudspeakers were especially suitable for some applications then
selection
of a 'standard' set of loudspeakers might have been impossible.





5.2 Family Resemblance.

The Table 1 shows the results for the family resemblance tests,
ordered by
the overall family resemblance score. The total possible range for the
score
was -2 to +2, showing once again a compression of the actual results
into a
restricted range.





Loudspeakers in test . . .
Loudspeaker quality scores . . .

Test #
Overall

average score
Large
Medium
Small
Large
Medium
Small
Total-9

6
0.93
a5
b4
c5
3.55
3.21
3.61
1.37

9
0.67
a5
b5
c5
3.55
2.95
3.61
1.11

8
0.44
a2
b2
c5
3.49
3.35
3.61
1.45

3
0.23
a2
b2
c2
3.49
3.35
3.09
0.93

5
0.17
a9
b9
c9
3.28
3.34
3.42
1.04

7
0.13
a6
b4
c4
3.72
3.21
3.21
1.14

2
-0.30
a4
b4
c4
3.15
3.21
3.21
0.57

4
-0.87
a6
b3
c3
3.72
3.56
3.48
1.76

1
-1.13
a5
b3
c3
3.55
3.56
3.48
1.59


Table 1. Family resemblance results sorted by overall score.



The columns are arranged as 'Number of the test' followed by the
'Score',
followed by three columns listing the actual loudspeakers included in
that
test. Columns 6 to 8 give the overall average scores for those
loudspeakers
from the quality tests. The final column gives the total average
quality
score. In the final column, the differences have been emphasised by
subtracting the average quality score for three loudspeakers of 9.



The order of preference for family resemblance depended on which of
the
comparisons - large/medium, medium/small, large/small or overall - was
used
to order the list. The table shows the list ordered by the overall
result.
Other tables were generated (not shown) with ordering by the three
individual two-way comparisons. The top four family test results for
each of
the four orders gave 6,9,3,8 / 6,5,9,8 / 8,6,9,5 and 6,9,8,3 as the
preferences, for respectively the large/medium, medium/small,
large/small
comparisons. The shaded section in Table 1 indicates those best five
candidates. The others scored too low on either the absolute quality
or the
family resemblance tests to be considered as contenders.



Test 6 was obviously a good combination. It came top (three times) or
second
(once) in the family resemblance test results and had a reasonably
high
quality score, coming fourth overall. However, it did include a mix of
manufacturers (the middle size was different). Test 9 produced lower
results
from the quality tests. It was thought to be too far down the quality
preference order to be a first choice, even though the family
resemblance
positions were 2nd, 3rd, 3rd, and 2nd. Test 8 had very good results
from the
quality tests (3rd place overall and top out of the five best results
for
family resemblance). However, it produced lower results from the
family
resemblance tests, coming 4th, 4th, 1st and 3rd respectively. Test 3
only
featured in two of the top four family resemblance results (sixth and
seventh in the other two). It also had low results for quality. Test 5
produced poor result from quality tests, although all sizes got about
the
same score. It also featured in only two of the top four family
resemblance
test orders, coming 5th and 8th in the others.



Of these results, Test 8 appeared to be a reasonable first choice. It
achieved 3rd place overall in the quality tests and 3rd in the overall
family resemblance results, but was only 4th in the important
Large-Medium
ordered list.



Tests 6 and 9 were both though to be possibilities for a second
choice. They
differed only in the selection for the medium sized loudspeaker. They
both
came high in the results for family resemblance. Test 9 did have lower
quality results but had the advantage of being a single manufacturer.





6 Final selection.

The loudspeakers in Test #8 of the family resemblance tests were a2,
b2 and
c5, i.e. a mixed manufacturer group. The small loudspeaker from the
same
manufacturer as the two larger sizes, c2, only obtained an average
score in
the quality assessments and the family resemblance score was not high
(Test
#3). However, a mixed set was not considered unacceptable. The large
and
medium sizes, a2 and b2, were certainly very well liked. Taken
together,
they scored higher than any other pair of large/medium loudspeakers
(Fig.
1). The small loudspeaker, c5, was the highest scoring one in its size
category. Overall, that should have been a fairly clear winning
combination,
with perhaps a bias towards absolute quality.



However, there were some serious reservations about the maximum sound
level
capabilities of the large loudspeaker. It was physically much smaller
than
others in the 'large' category. The original guide specification had
indicated a maximum output level of 108 dB spl, which it was easily
capable
of at mid and high frequencies. However, even the manufacturer's own
data
showed very limited low frequency output - with measured indications
of
stress and distortion by 90/92 dB spl at 50/63 Hz. Listening tests
confirmed
that the limited low frequency output would be an operational
restriction in
many areas, though adequate for some others.



One option would have been to select that loudspeaker as the standard
for
the large size and to specify something else in areas that required
higher
output. However, it was decided that would lead to too many special
cases,
maybe even the majority of 'large' loudspeaker installations, and
would
partially defeat the objective of selecting a standard set of
loudspeakers.
Therefore the selection was rejected.



The next reasonable option was either Tests #6 or Test #9. They were
the
same except for the choice of the middle-sized loudspeaker. They both
came
high on the list of family resemblance results, but their results for
absolute quality were substantially different. That led to an
investigation
of some very different results obtained for the 'b5' loudspeaker from
the
two groups of test subjects. The first group had given the loudspeaker
an
overall score of 3.84 and the second group 1.91. The first group
thought the
loudspeaker better than any other by a large margin. The second
highest-placed medium loudspeaker by that group scored 3.50. The
second
group thought the loudspeaker worse than any other, though by a
smaller
margin. The next lowest placed medium loudspeaker by that group scored
2.17
[8]. That seemed to be a perverse anomaly that required further
investigation.



The discrepancy needed to be resolved, though the issue actually only
involved the selection of the medium-sized loudspeaker, whether it was
to be
'b4' or 'b5'. The two samples of loudspeaker 'b5' were listened to
extensively by the project team in the test environment. It became
clear
that the two samples supplied were significantly different in sound
quality.
When placed together, the difference was striking. Of course, there
was no
indication of which one was more 'correct'. It was decided that
further
samples of the type should be obtained and assessed. An additional
pair was
acquired and assessed by the project team. That pair was more
consistent and
also matched one of the original pair fairly well. It was concluded
that one
of the original pair had been somewhat different. It was still not
clear
whether that represented normal manufacturing tolerances or a
defective
sample. The effect of the difference in the responses of the original
pair
might well have been to give a 'spread' in the stereo image quality,
corresponding to a "pulling apart" or "stretching out" of the
stereophonic
images reported by some members of the project team. Such an effect
might
have been picked up by one group of test subjects but not by the
other,
especially if their attention was also being taken up by a long list
of
other factors. Further assessments, and some measurements, were
carried out
at BBC R&D Department, using another listening room and some other
test
subjects, comparing all four of the samples.



The conclusion at the end of all of the supplementary tests was that
the
anomalous score given by the second group of test subjects was
probably
correct but that it had been based on a defective or atypical
loudspeaker
sample.



The family of loudspeakers represented by 'a5', 'b5' and 'c5' had very
closely matched manufacturing processes. Indeed, the project team were
quite
convinced that some of the actual drivers were the same in some cases,
though the manufacturers did state that they were not precisely the
same. In
that context, it seemed unlikely that the loudspeakers should score
significantly differently in the quality tests. In the end, it was
recognised and accepted that the final choice for the medium-sized
loudspeaker was based on somewhat uncertain evidence. The balance of
probability, taking all the factors into account, was felt to be in
favour
of loudspeaker 'b5' as the preferred choice over 'b4'.



It was also the case that loudspeaker 'b4' was already in use in
significant
numbers in some parts of Radio. Therefore, the existing loudspeakers
could
be re-deployed without conflicting with the requirement for family
resemblance as it provided a good match and the existing stock would
remain
of some value. Thus, the final preferred selection for new purchases
was 'a5'
for the large sized loudspeaker, 'b5' for the medium sized loudspeaker
and
'c5' for the small sized loudspeaker.





7 Summary and conclusions.

This report has presented the background to the requirements for a
range of
new, standard loudspeakers for BBC Radio & Music. That required a set
of
three sizes, to replace the BBC designed LS5/8, LS5/9 and LS3/5A still
in
use in many areas. Because of the size of the potential order, there
was in
any case a requirement to pursue a formal tendering and selection
process
for the supply of the loudspeakers.



A project was set up to carry out the tendering process and to
organise
formal subjective tests on potential candidates as part of the
selection
process. The selection was to be made not only on the absolute quality
of
the loudspeakers but also on their family resemblance so that a more
uniform
sound quality could be achieved in a range of applications. The tests
involved operational studio managers from many areas of radio as test
subjects, proposed by their colleagues as being expert and credible
judges
of sound quality. The intention was to try to ensure that all staff
potentially affected would have contributed in some way to the process
and
would, as a result, more readily accept the outcome. Loudspeakers are
a
sensitive topic on which many people have very different views and it
was
anticipated that reservations could be expressed by some. By trying to
involve as many people as possible, even only in a peripheral sense,
it was
hoped that the transition to new loudspeakers could be made as smooth
and
trouble-free as possible.



Inevitably, the final outcome was not entirely clear-cut. It would
have been
simple if a single set of loudspeakers from one manufacturer had stood
out
from the rest in quality and family matching. In the event, the best
loudspeakers in each size category were not selected because of the
family
matching requirements and other technical performance issues, but the
final
compromise was still close to the best. The final choice consisted of
three
different sizes of loudspeaker from the same manufacturer. That should
have
benefits for consistency in the future, especially if that
manufacturer were
to change the production method or introduce new models. It should be
noted
that many other factors were also included in the selection process.
They
included weight, size, maintainability and many others. The acoustic
differences were taken into consideration amongst those other factors
in the
overall selection.





8 Acknowledgements.

This work was managed and led by Stuart Collins of Radio Resources.
Many
other people from different departments contributed to the selection
process. The author acknowledges the substantial contributions of all
of
those people, whose work this paper mainly represents. The author
gratefully
acknowledges the permission granted by the BBC for publication of this
paper.





9 References.

1. Walker, R. A new approach to the design of control room
acoustics for stereophony. AES Preprint No. 3543 (G1-1), 94th AES
Convention, Berlin, March 16-19, 1993.

2. Walker, R. Early reflections in studio control rooms: the
results from the first Controlled Image design installations. AES
Preprint
No. 3853 (P12-6), 96th AES Convention, Amsterdam, February 26-March
01,
1994.

3. Walker, R. Controlled Image Designs: the management of
stereophonic image quality. BBC R&D Report No. 1995/4.

4. Walker, R. Controlled Image Designs: the measurement of
time-frequency responses. BBC R&D Report No. 1995/3.

5. Walker, R. Controlled Image Designs: results from the first
installations. BBC R&D Report No. 1995/5.



RW
19/08/04





1] It is true that there are still some individual designers working
in
the field. Generally, they produce loudspeakers with individualistic
characteristics, which may well be excellent in their field of
application,
but which do not satisfy the BBC's requirements for a widely used,
standard
loudspeaker.

[2] Other non-audio factors, such as handling, maintainability,
interchangeability, etc. were also important, as were business
considerations not specifically related to loudspeakers, but they were
not
part of the audio quality selection process.

[3] Throughout this selection process, the terms 'large', 'medium' and
'small' were used to identify the three broad categories of
loudspeaker. In
the design, it is inevitable that more compromises have to be made to
obtain
good performance from a smaller loudspeaker. The use of loudspeakers
is
frequently constrained by the space available and the accompanying
reductions in performance have to be accepted. For cost-sensitive
applications, smaller loudspeakers also usually cost less.

[4] The author does not subscribe to this view. Most loudspeakers are
so
inefficient electrically and are in any case normally driven from a
virtually zero impedance source that the possibility of received sound
being
re-radiated after being converted to electrical signals and back again
is
entirely negligible. The box itself is an object in the room, but no
more
significant than any other piece of furniture. Some loudspeakers have
open
resonant ports for low frequency tuning, but the damping/reverberation
time
of those resonators is entirely swamped by the room itself (in any
reasonable room and with any reasonable loudspeakers).

[5] The suppliers had been asked to provide stands if they thought it
necessary to do so. Otherwise, ordinary metal frame stands were used.
In
more than one case, the manufacturer's own stand was found to be
unacceptable.

[6] Maida Vale, Studio 7 control cubicle.

[7] All of the loudspeakers assessed in this trial were from the upper
end of high quality professional monitoring loudspeakers and that a
'poor'
score might still represent a loudspeaker that was good or excellent
in
terms of absolute performance for other applications.

[8] It was also true that the second group gave lower scores overall.
The overall average scores from the quality tests for the two groups
were
3.34 and 2.97 respectively. Given the compression of scores into a
relatively narrow range, that represented quite a marked difference.








Michael McKelvy
Posted Jan 28, 2005 17:18:21
Subject: A bit from the BBC (Long)

Guest




"Ruud Broens" |broensr([at]wanadoo.nl| said
41f98e49$0$54524$a344fe98([at]news.wanadoo.nl...
|
| "Michael McKelvy" |deskst49([at]peoplepc.com| said
| JpdKd.836$Pi.315([at]newsread3.news.atl.earthlink.net...
| :
| : The selection of loudspeakers for BBC Radio & Music.
| :
| :
| : R. Walker, BBC Research and Development Department.
|
.................................................. ..............................

I hope this e-mail address of yours works because I just sent some .pdf
files that were sent to by Sean Olive that clearly indicate that B&O, NASA,
and Nokia all use ABX.




wrote in message
oups.com...

wrote:
wrote in message
oups.com...

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution to
the "The case for ABX..." thread. I thought you might be unwell or on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT medical
research before any of you heard the name. Medical DBT research can be
validated by objective data (psyche is an exception) 2) DBT is DBT,
ABX is ABX 3)

ABX is a double blind protocol used by people doing audio research from
the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and
on
and on.


I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind.

Take whatever you want you still haven't a clue about ABX for audio.

5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB.

You mean you've finally accepted that ABX is a valid protocol used to
test
for subtle audio differences?

But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

I know that ABX is a form of DBT, something you don't seem to get.
It is also a widely used and valid protocol, something else you don't
seem
to get.
__________________________________________________ _________

ABX is a double blind protocol used by people doing audio research
from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and
on
and on.

BBC-- no. Quote documentation and show me for a liar or you for a
...what?
B&O- no. Ditto, Ditto, Ditto
Nokia that famous audio company uses it for comparing cellphones or
what? Documantation?
Harman -no. Ditto, Ditto, Ditto
Have you got the distinction between DBT and ABX clear in your head
yet?
NYOB, NYOB, you make my day.
Ludovic Mirabel

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get
a hobby already.


2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??
Just don't care. Is there some reason I should?


Yes indeed, you were mouthing off about them,
without knowing anything about them.

And?

The Bug Eater allows us to expand the Mickey McMickey Codex.

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

What the **** are doing wasting time counting the number of lines, get
a hobby already.

2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??

Just don't care. Is there some reason I should?

Yes indeed, you were mouthing off about them,
without knowing anything about them.

And?


Thanks Mr. McMickey for, divulging the correct pronunciation of "duh"
in Bug-Eaterese.



..
..

"George Middius" wrote in message
...


The Bug Eater allows us to expand the Mickey McMickey Codex.

2500 lines of posturing, fer chrissakes. Shut up, Mikey.

What the **** are doing wasting time counting the number of lines,
get
a hobby already.

2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??

Just don't care. Is there some reason I should?

Yes indeed, you were mouthing off about them,
without knowing anything about them.

And?


Thanks Mr. McMickey for, divulging the correct pronunciation of "duh"
in Bug-Eaterese.


Oooooh, that hurts.

You may now return to your turd sandwich.

wrote:

Audio Info
Audio Information
Audio Info | Search | Statistics | Contact Us

You are logged in as a guest. ( logon | register )


A bit from the BBC (Long)


Audio Info- Audio Opinion

Michael McKelvyas

if that's the real name. just making one up doesn't make me
believe it's really the right name

mk5000

"anyonewho ,istens to this music is a kid at heart. It brings yuo back
to when cartoons were the last bastion of good fun"--MF Doom

wrote in message
nk.net...


1831 lines and two useless "gif"ts

wrote in message
nk.net...

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot] net
wrote in message ...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines, get
a hobby already.


2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??
Just don't care. Is there some reason I should?


Yes indeed, you were mouthing off about them,
without knowing anything about them.

And?

Your incompetence was showing.
i just wanted to give you a chance to zip up your fly.

wrote:
I hope this e-mail address of yours works because I just sent some .pdf
files that were sent to by Sean Olive that clearly indicate that B&O, NASA,
and Nokia all use ABX.




wrote in message
oups.com...

wrote:
wrote in message
oups.com...

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution to
the "The case for ABX..." thread. I thought you might be unwell or on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT medical
research before any of you heard the name. Medical DBT research can be
validated by objective data (psyche is an exception) 2) DBT is DBT,
ABX is ABX 3)

ABX is a double blind protocol used by people doing audio research from
the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and
on
and on.


I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind.

Take whatever you want you still haven't a clue about ABX for audio.

5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB.

You mean you've finally accepted that ABX is a valid protocol used to
test
for subtle audio differences?

But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

I know that ABX is a form of DBT, something you don't seem to get.
It is also a widely used and valid protocol, something else you don't
seem
to get.
__________________________________________________ _________

ABX is a double blind protocol used by people doing audio research
from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and
on
and on.

BBC-- no. Quote documentation and show me for a liar or you for a
...what?
B&O- no. Ditto, Ditto, Ditto
Nokia that famous audio company uses it for comparing cellphones or
what? Documantation?
Harman -no. Ditto, Ditto, Ditto
Have you got the distinction between DBT and ABX clear in your head
yet?
NYOB, NYOB, you make my day.
Ludovic Mirabel

__________________________________________________ _______

..
This was the exchange on Nov 16th in this thread. There were two
postings by NYOB
The first one can be looked up in that thread just above this one.
NYOB:
ABX is a double blind protocol used by people doing audio research
from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies, and
on
and on.
I answered:
BBC-- no. Quote documentation and show me for a liar or you for a
...what?
B&O- no. Ditto, Ditto, Ditto
Nokia that famous audio company uses it for comparing cellphones or
what? Documantation?
Harman -no. Ditto, Ditto, Ditto
Have you got the distinction between DBT and ABX clear in your head
yet?
NYOB, NYOB, you make my day.
Ludovic Mirabel
In response I got 2 postings . The first one can be read entire in the
thread. Even with "hiddeb text" it would make this too long for one
posting. Below is my response to the two.

You are my one and only, inimitable, incredible, fatuous,
transsubstantial NYOB.
With a wave of your wand you transport me to the "Alice in Wonderland"
country. Whuite Rabbits, Mad Hatters all over the place.
So far you produced some 60 phony references to the good, old
Humpty-Dumpty ABX.
This is the latest one- 7 meters long.. Ther was nothing about ABX in
most of those , except for a few of them with a negative, "no
diference" component comparisons.There is not one word about ABX in
this one. The closest it gets is an incomplete DBT. Quote.:
" It was not possible to carry out the tests entirely 'double-blind.
Inevitably, the project team would know which loudspeakers were
being
assessed. However, only one member of the project team remained
in the
room to supervise the tests - to ensure that the test subjects did not
confer or try to look behind the screen to identify the loudspeakers.
That
person was often actually unaware of the loudspeaker manufacturer
because code
letters were used to identify the loudspeakers. In any case, under the
pressure of physically moving the loudspeakers and setting the levels,
the members
of the project team had little time (or inclination) to take note of
the
identity. The supervisor was also under strict instruction not to
exhibit any reactions."
End of quote.
In the 2nd posting wrote:
I hope this e-mail address of yours works because I just sent some .pdf
files that were sent to by Sean Olive that clearly indicate that B&O, NASA,
and Nokia all use ABX.

I got by email 3 Acrobat attachments about a research project. I
didn't read it in detail but it seemed to have nothing to do with
ABX-as usual. Who cares what it "indicates" to NYOB? All we need is
just one paragraph: "We used ABX as testing method with this
result:..."
NYOB- I get enough unasked for garbage by email as it is. You're
welcome to wish me a Merry Xmas but if you have a contribution to a
forum discussion publish it in the forum. Having to cope with offers of
Cialis, penis enlargement and young women pining to entertain me I get
enough tempting spam as it is.
Secondly: If you found something about ABX (or whatever interested you)
publish the quote with a reference not the whole bloody paper to waste
time on for no benefit. If you had ever tried to publish a scientific
paper you wouldn't get very far if you snowed the editor under with
your complete copies to wade through Soon you'll be copying books in
the forum.

It is difficult to fathom what motivates you. I doubt if you're
sophisticated enough to be just an ordinary liar. I'm considering
several hypotheses: You have problems with simple verbal comprehension-
you fall asleep before you get to the end of it- you think I'll fall
asleep- which is it?
No further response to further pointless mailings. Go in the corner and
mouth: "I'm not a liar. Ludovic is a liar if he says I'm a liar.
Ludovic Mirabel
:

"Clyde Slick" wrote in message
news

wrote in message
nk.net...

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"Clyde Slick" wrote in message
...

wrote in message
nk.net...

"George M. Middius" cmndr [underscore] george [at] comcast [dot]
net wrote in message
...


2500 lines of posturing, fer chrissakes. Shut up, Mikey.



What the **** are doing wasting time counting the number of lines,
get a hobby already.


2,560 exactly, and I didn't ahve to count them.
Having trouble reading usenet specs, Mikey??
Just don't care. Is there some reason I should?


Yes indeed, you were mouthing off about them,
without knowing anything about them.

And?

Your incompetence was showing.
i just wanted to give you a chance to zip up your fly.
I'm sorry, does this mean I can't join the usenet geeks club and compare how
many lines in each post?

wrote in message
oups.com...

wrote:
I hope this e-mail address of yours works because I just sent some .pdf
files that were sent to by Sean Olive that clearly indicate that B&O,
NASA,
and Nokia all use ABX.




wrote in message
oups.com...

wrote:
wrote in message
oups.com...

Arny Krueger wrote:
"George M. Middius" cmndr [underscore] george [at] comcast
[dot] net wrote in message


2500 lines of posturing, fer chrissakes. Shut up, Mikey.

Thanks for showing once again that you can't tell the
difference between audio knowlege and posturing, Middius.

Nice to hear from you, Arny. I've been looking for your contribution
to
the "The case for ABX..." thread. I thought you might be unwell or
on
leave but I see that you let your flock take the grapeshot. Well an
ArchPriest has to save his irreplaceable self.
Your answer to Middius allows two interpretations: 1) you do not
understand that "posturing" refers to Nyob's appropriating a serious
article 2) You're up to your debating trickery which I continue to
think is not worthy of you.
As for Ludovic needing lessons from fatuous NYOB in the value of
double-blind testing{ For the umpteenth time:1) I was doing DBT
medical
research before any of you heard the name. Medical DBT research can
be
validated by objective data (psyche is an exception) 2) DBT is
DBT,
ABX is ABX 3)

ABX is a double blind protocol used by people doing audio research
from
the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies,
and
on
and on.


I use blind technique sometimes myself when I listen to a
new component. Not yours but mine Left-Right method.4) But when it
comes to invalidable preferences in musical reproduction I'll take
eg.
Holt's opinion sighted any time over NYOBs nth power seventy times
blind.

Take whatever you want you still haven't a clue about ABX for audio.

5) Research is another matter entirely: double blind provides
uniformity. I need no convincing from NYOB.

You mean you've finally accepted that ABX is a valid protocol used to
test
for subtle audio differences?

But I would like him to
stop conflating DBT and ABX - assuming that he is capable of
understanding the difference.
Ludovic Mirabel

I know that ABX is a form of DBT, something you don't seem to get.
It is also a widely used and valid protocol, something else you don't
seem
to get.
__________________________________________________ _________

ABX is a double blind protocol used by people doing audio research
from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies,
and
on
and on.

BBC-- no. Quote documentation and show me for a liar or you for a
...what?
B&O- no. Ditto, Ditto, Ditto
Nokia that famous audio company uses it for comparing cellphones or
what? Documantation?
Harman -no. Ditto, Ditto, Ditto
Have you got the distinction between DBT and ABX clear in your head
yet?
NYOB, NYOB, you make my day.
Ludovic Mirabel

__________________________________________________ _______

.
This was the exchange on Nov 16th in this thread. There were two
postings by NYOB
The first one can be looked up in that thread just above this one.
NYOB:
ABX is a double blind protocol used by people doing audio research
from the
BBC to Nokia, to Bang&Olafson, to Harman, to hearing aid companies,
and
on
and on.
I answered:
BBC-- no. Quote documentation and show me for a liar or you for a
...what?
B&O- no. Ditto, Ditto, Ditto
Nokia that famous audio company uses it for comparing cellphones or
what? Documantation?
Harman -no. Ditto, Ditto, Ditto
Have you got the distinction between DBT and ABX clear in your head
yet?
NYOB, NYOB, you make my day.
Ludovic Mirabel
In response I got 2 postings . The first one can be read entire in the
thread. Even with "hiddeb text" it would make this too long for one
posting. Below is my response to the two.

You are my one and only, inimitable, incredible, fatuous,
transsubstantial NYOB.
With a wave of your wand you transport me to the "Alice in Wonderland"
country. Whuite Rabbits, Mad Hatters all over the place.
So far you produced some 60 phony references to the good, old
Humpty-Dumpty ABX.
This is the latest one- 7 meters long.. Ther was nothing about ABX in
most of those , except for a few of them with a negative, "no
diference" component comparisons.There is not one word about ABX in
this one. The closest it gets is an incomplete DBT. Quote.:
" It was not possible to carry out the tests entirely 'double-blind.
Inevitably, the project team would know which loudspeakers were
being
assessed. However, only one member of the project team remained
in the
room to supervise the tests - to ensure that the test subjects did not
confer or try to look behind the screen to identify the loudspeakers.
That
person was often actually unaware of the loudspeaker manufacturer
because code
letters were used to identify the loudspeakers. In any case, under the
pressure of physically moving the loudspeakers and setting the levels,
the members
of the project team had little time (or inclination) to take note of
the
identity. The supervisor was also under strict instruction not to
exhibit any reactions."
End of quote.
In the 2nd posting wrote:
I hope this e-mail address of yours works because I just sent some .pdf
files that were sent to by Sean Olive that clearly indicate that B&O,
NASA,
and Nokia all use ABX.

I got by email 3 Acrobat attachments about a research project. I
didn't read it in detail but it seemed to have nothing to do with
ABX-as usual.

Thanks for admitting you don't try to find the pertinent references.

Who cares what it "indicates" to NYOB? All we need is
just one paragraph: "We used ABX as testing method with this
result:..."

All of those attachments had something to do with ABX.

NYOB- I get enough unasked for garbage by email as it is. You're
welcome to wish me a Merry Xmas but if you have a contribution to a
forum discussion publish it in the forum. Having to cope with offers of
Cialis, penis enlargement and young women pining to entertain me I get
enough tempting spam as it is.
Secondly: If you found something about ABX (or whatever interested you)
publish the quote with a reference not the whole bloody paper to waste
time on for no benefit.

You won't have to worry, since you are completely dishonest on thesubject of
ABX and DBT, there's no reaosn to talk to you about it anymore. Yo already
know that it's used by nearly everyone sho does any kind of audio research,
denying it doesn't change that.

You could e-mail Sean Olive and find out far more than I can tell, you, but
you won't

If you had ever tried to publish a scientific
paper you wouldn't get very far if you snowed the editor under with
your complete copies to wade through Soon you'll be copying books in
the forum.

Instead of talking about papers I'm not going to write, you'd be better of
reading some that have to with the subject you claim to have some interest
in.


It is difficult to fathom what motivates you.

I like to show fools that they arn't fooling anybody.

I doubt if you're
sophisticated enough to be just an ordinary liar. I'm considering
several hypotheses: You have problems with simple verbal comprehension-
you fall asleep before you get to the end of it- you think I'll fall
asleep- which is it?
No further response to further pointless mailings. Go in the corner and
mouth: "I'm not a liar. Ludovic is a liar if he says I'm a liar.
Ludovic Mirabel

We all know what you are Ludo.

**** off.