[Audio_Empire[_2_]]

In article , KH
wrote:

On 9/27/2013 4:04 PM, Audio_Empire wrote:
In article ,
ScottW wrote:

snip

But all of this depends so much on how the tests are set-up and run, and
the
environment in which they are held, and who is participating (what are the
p
articipant's personal agendas, if any? There are people who post here who I
wouldn't
let participate in a DBT, because their personal agenda is to find NO
difference
between any two of anything). All of this makes DBTs of audio gear somewhat
suspect in my mind.

As has been stated often before, all that is required are "subjectivist"
audiophiles who *know* they hear a difference between two components,
sighted, to participate in DBT's of those two components. If they can,
with proper controls, reliably identify these, typically, obvious
differences, then voila! Done. If not, the DBT protocol has
successfully removed the subject bias.

Obviously, it easier to prove that subjectivists can hear no differences
in a properly set-up DBT than it is to prove that objectivists CAN hear
differences when said objectivists DON'T WANT TO hear differences. In
the former case, the DBT should be able to eliminate biases. You can't
pretend to hear differences when you don't know what you are listening to.
The results should prove you wrong every time (if you are merely determined
to hear differences whether they exist or not). OTOH, a subjectivist whose
agenda is to not hear differences can fool the system by merely reporting
that they heard no differences on any try and there is no way for the statistics
to trip them up. You can't prove a negative, after all.

And how does a test overcome such a refusal? You can't make a dishonest
subject honest when all they need do is fabricate random responses.

Yes that's true. As far as I'm concerned, it makes DBTs for audio somewhat
suspect as we're not dealing with concrete results. In medicine, DBTs are
used
routinely to test new drugs. There are usually two groups, one of which
gets
the real drug, and the other (called the control group) gets a placebo.

Actually, much more often the control group gets a currently marketed
drug, not a placebo.

The people
taking part in the test don't know which group they are in and the people
dispensing the drugs don't know which participants are getting the placebo
and
which are getting the real drug. Someone, way up the line knows which is
which,
but even they just know the participants by number - not by name. The
results of
these tests compare results with the control group to see if the new drug
is statistically
effective. IOW, either the drug taking group has a change in symptoms
compared
with the control group, or they don't. The results are pretty unambiguous

That, unfortunately, is very often *not* the case. Even when a placebo
is used. It's a function of the "power" of the ample size, and often
requires a very large population (hence the extremely high costs
associated with them) to demonstrate statistical differences. Most
results are not binary (i.e. responds/does not respond).

There are exceptions to everything. Obviously I was generalizing here.

In this respect, an audio DBT is much easier, actually than a drug
trial. As stated above, the control and tests groups are the same.
Subject to subject variability in response is completely eliminated
since you start with subjects already demonstrated to respond to the
difference in "tests" under sighted conditions. The statistical power
of the test population becomes moot.

But since the results are nowhere as clear or well defined, I'd say that audio
DBTs are FAR less reliable than drug tests.

as there
is simply no way to fake a result.

Assumes a simple binary response. Accurate for audio, not for drugs.


I would suggest that a person who doesn't think they can hear a
difference is

simply not a good subject for such a test.

Yes. That is true, but still, that's something that is hard to determine
beforehand.

It's hard to ask a person...do you think you hear a difference sighted?

Exactly.

Why is that hard? You either pick one of the ubiquitous audiophiles who
claim differences between *everything* they hear, sighted, and pick to
components and do the DBT. Why is that hard? Or, you perform the "DBT"
methodology, sans blinding, and use only the subjects that clearly hear
a difference in two components. In most cases, that will be all of
them. Then blind the study and repeat.

As I've posted here before, I know the physics involved in wires, and
*know* that wire is wire in audio situations. I have also been in
sighted wire demonstrations by AQ in which I clearly heard a difference
between zip cord and AQ wire. Single blind at home - zilch, as
expected. The brain is a wonderful pattern processor and difference
engine, and will always try to discern a difference between stimuli.
Sighted testing simply doesn't work to discriminate subtle differences.

And DBT is unreliable because the bias controls are only effective for a
positive result, not for a null result. So, what does one do?