View Single Post
  #7   Report Post  
Steven Sullivan
 
Posts: n/a
Default Comments about Blind Testing

Mkuller wrote:
(Watch King) wrote:


snip


But for CD players and interconnect cables the testing can be very
straightforward. The "moderator" cannot alas partake of the test for
the sake of non-biased presentation. All lead-ins, song intros and
explanations have to be prerecorded and "played" to the listening
testers. Once the test starts it must finish or all results are
unusable. There are many many "control restrictions" needed and
requiring pre-test documentation of procedures. The musical program
material should be rotated throughout the program during different
tests to reduce the biases that "program material position" in the
test program can create. As often as possible the order which any item
is tested first should change. For high power switching of items like
amps, speaker cable and speakers make the loudness turndown steps
between test items pretty short on the order of .1 seconds from full
loudness to 0, then switch, then turn up in .1 seconds. By putting
time code onto a CD and having the switches time code driven this can
be accomplished. We used telephone touchtone signals to activate the
numerical display box. The switch shutdown/turnup can be programmed
right onto the CD material although duplicate disks would need to be
synchronized somehow if 2 CD players were being compared.

The most listening testers can seem to hold their sonic concentration
is betwen 20 minutes and 40 minutes. It is an intense experience. On
the other hand testers don't seem to be able to fully concentrate
until about 2 comparisons into the test or about 1-2 minutes. 20
minutes gives you barely enough time for one throwaway opener and then
6 bits of test material and 40 minutes can allow for 12 or so tests
passages but people start getting headaches and listening fatigue. If
need be, run the test a number of times with different program
material and with intervals of 30-75 minutes between tests. Don't
drink too many liquids before a test session. Getting up for the
bathroom ends any test with "No valid results". In other words no
distractions should be tolerated (no cellphones, no doorbells, no
chatting or physical communication between test listeners, sadly-no
crying babies and especially no "Just listen to this" kind of cueing.)
It's either done professionally or it's useless.


Once again you have brought up many important variables that may affect the
outcome and validity of any open-ended audio component comparison DBT using
music, particularly the amateur DIY variety that are strongly advocated by some
posters here.


Any of the provisos he's cited would *also* apply to sighted comparison,
of course...but they certainly don't seem to be applied in the sighted
comparisons I read about every month.

But then again, nothing he's written even remotely supports the idea that
*sighted*, 'open ended' comparison, using music (and please, feel free
to add whatever new conditions you can conjure up),
advocated and practiced by the most audiophiles, including the two
main audiophile magazines, is a good way to test for difference at all.

And that's because -- and this is the crucial thing -- it
can't *ever* be a good method, for verifying subtle differnces.
In other words, in contrast to scientific methods,
the method advocated by
the main 'voices' of audiophilila, and people like yourself, is
*fundamentally and essentially flawed*,
as all researchers in the field of perception acknowledge.

DBT for audible difference is 'perfectable' -- sighted listening simply
*isn't*.

--

-S.

"They've got God on their side. All we've got is science and reason."
-- Dawn Hulsey, Talent Director