[Audio Empire]

On Fri, 1 Apr 2011 16:40:13 -0700, Harry Lavo wrote
(in article ):

"Andrew Haley" wrote in message
...
Scott wrote:
On Apr 1, 4:59?am, Andrew Haley
wrote:

snip

So it is not unfair much less grossly unfair to make this
charcterization when Arny pulls out the science flag. It's only
better to have more people doing "science" so long as they are doing
it up to the standards set by the scientific community.

There, I agree totally. What matters is how well the experiment is
done. But it's a matter of degree: some experimental controls are
surely better than none, even if the experiment isn't perfect.

Not necesarily. If the controls that aren't there are crucial to the
validity of the test, or the design of the test itself is not valid
(stimulus, measurements, intervals, training, intervening technology, etc.)

Conventional ABX'ng has never been shown to be valid in evaluating MUSIC
differences that other approaches (the aforementioned Oohashi test) and even
the ABC/hr test have proven better at. Yet ABX is the test that Arny
developed a computerized version of, and has relied on.

If the construct of the test itself intereferes with the normal evaluative
process, you can almost be guaranteed that it will not produce valid
results. One of the principles of testing in any field of human endeavor is
to try to emulate as much as possible the conventional context of the
variable under test.


Well put. These are some of the things that bother me about the body of
conclusions that many of these tests produce. As I have indicated before, I
have participated in many DBT tests where we have worked hard to set up
correctly, with level matching to less than a quarter of a dB both electrical
and acoustical, set switch times, long samples, the switch operator in
another room, all indications of a switch taking place masked (input lights,
etc.), the AB box (where used) in an insulation-filled box so we can't hear
the relays, etc. and we have returned statistically positive results for amps
and DACs. . I have also been involved in DBTs where null results have been
returned.

In those tests where a positive result occurred, I found the differences to
be so trivial that only a very anal retentive audiophile could possibly not
be happy with any of the units under test! While they all sounded a little
different in some respect, they all sounded good. The only time we got a
gross difference was when, for fun, we pulled out our host's old Dynaco
ST-120 and ran it against a new, and very expensive Audio Research Hybrid
HD220 amp. The results made us all laugh. The ST-120 sounded dreadful while
the AR was very neutral sounding.