|
|
|
|
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
Darryl Miyaguchi
wrote:
On 5 Jul 2003 20:25:01 GMT, (Nousaine) wrote:
Actually in the ABC/Hr protocol any listener reliably rating the hidden
reference worse than itself has demonstrated SOME form of test or subject
bias.
Ther eis no other explanation; either the protocol has some form of
non-sonic
confounding identifier OR the subject can truly hear the difference and is
purposefully responding in a backward manner.
Typically the listener doesn't always pull down the slider for the
hidden reference, but does it once or twice out of a total of 5 or 6
groups. If the listener were to pull the reference for every codec,
or if he were to pull the reference for the same codec every time he
repeated the entire comparison (i.e., if it was "reliable" in some
way), I would certainly become suspicious.
I'm all for rejecting biased data but any kind of significant "reverse"
results
shoyld be followed by an examination of the experiment to find whether it's
specific to a given subject(s). If the latter cannot be shown then the
entire
experiment can be ruled invalid.
In one particular experiment, a listener was reliably identifying two
particular codecs out of six (he was able to do this for every one of
the music samples being listened to). This was very strange because
these were among the best codecs being compared, and he never
identified the one which sounded just awful to everybody else!
It turned out that this listener was picking out the two codecs based
on a difference in time alignment of about 25 milliseconds. Needless
to say, such a result would invalidate an experiment.
Well that was my point and it's gratifying that you take the time to examine
this form of bias.
What I'm saying here in a roundabout way is that certain results raise
red flags, and usually there's a sensible explanation for what's going
on. A listener downrating the reference doesn't happen too often, but
it does, and I can't say that I'm all too surprised that some people
will do this, especially if they're relative novices at performing
this type of comparison. Unless somebody can come up with a better
explanation, I chalk it up to people thinking they hear a difference
when they really don't.
Sure, that's a common human trait. But if they do this reliably then you need
to examine further for some form of bias.
As I said earlier if you cannot determine that results were limited to a
given
subject(s) then the whole experiment must be considered suspect. At the very
least it should be repeated.
One option is to ask the individual in question to repeat his test,
and I suppose this could be done for future comparisons. However, I'm
a very practical person. Pointing out a specific bias is one thing;
pointing out vague concerns about a possible bias with no plausible
mechanism in mind is another. Yes, some people could be downrating
the reference because of some as of yet unguessed reason. But I think
it's far more likely that they're hearing a difference that just isn't
there.
Darryl Miyaguchi
People hear non-extant differences of human nature. That's one of the ideas
behind statistical analysis. However, if they are reliably making these choices
then some form of non-sonic bias has to exist.
|