S888Wheel wrote:
Any of the provisos he's cited would *also* apply to sighted comparison,
of course...but they certainly don't seem to be applied in the sighted
comparisons I read about every month.

If you don't like them why are you reading them?

Component reviews are typically a mix of fact -- how many
inputs/outputs does the device have, what formats does it play,
what materials and parts were used in its construction, how is the
remote laid out , how does it differ from previous or competing
models in terms of features , etc -- and possibly spurious
subjective impression about the sound . Guess which part I
find most valuable.


But then again, nothing he's written even remotely supports the idea that
*sighted*, 'open ended' comparison, using music (and please, feel free
to add whatever new conditions you can conjure up),
advocated and practiced by the most audiophiles, including the two
main audiophile magazines, is a good way to test for difference at all.


Equipment reviews are not tests for differences per se. They are subjective
reviews of equipment used by the reviewer in the likely manner that the
consumer would use the product.

Which does not make the conclusions drawn any more tenable...all it means
is that both reviewer and consumer tend to evaluate the sound of a
component in a manner that is unlikely to be particularly
reliable or accurate. But it does make them feel good.

And that's because -- and this is the crucial thing -- it
can't *ever* be a good method, for verifying subtle differnces.

Varification is not an issue in subjective review for the most part.


Indeed. But these 'subjective reviews' you talk about do *not* confine
themselves purely to statemenst such as, "I liked this one", "I didn't like
that one"; "This one made me feel good", or the like. They instead often
*do* involve comparisons (often to the reviewer's 'reference system')
and they *do* make *specific claims* about one 'throwing a better soundstage'
or 'sounding less harsh' or 'clarifying the bass'. Under thy typical
review conditions there is a distinct possibility that these
impressions are NOT the result of
actual sonic differences, but are *entirely* psychological in origin -- a
likelihood that science tells us cannot be confidently dismissed without more
controlled comparison. Reviewers pretend this isn't the case.



Using the
product as the consumer would use it is a reasonable way to evaluate equipment
if the consumer who reads the magazine evaluates equipment the same way.

Yes, the community of belief is a strong component of the hobby. But is the belief
warranted by the facts? At what point does the hobby become more
'faith based' than fact-based?


If you
read the reviews and don't like the fact that they are not scientifically
reliable, I suggest you read the disclaimer that suggests consumers shouldn't
rely on reviews alone and should audition equipment for themselves before
making any purchases.

Do these disclaimers -- where are they printed , btw, in the current issues
of the usual audio journals? -- further recommend that the personal auditions be
carried out in a scientifically reliable manner? Repeating the error of the
reviewer yourself isn't going to get you closer to reliable.

In other words, in contrast to scientific methods,
the method advocated by
the main 'voices' of audiophilila, and people like yourself, is
*fundamentally and essentially flawed*,

Yes they are.
As is the case for any subjective review. Stereophile is not
trying to be a scientific journal. Most journals that do subjective reviews of
hardware in any number of fields are every bit as unscientific.

I'll never understand why you think telling me that *lots* of journals
have poor standards of objective proof for their claims, is any sort of
argument *for* sighted comparison.

What sorts of products are you thinking of?

DBT for audible difference is 'perfectable' -- sighted listening simply
*isn't*.


I wouldn't expect such absolute claims from a scientist.

All it means is that you can 'refine' a sighted method forever, and the results will
always have a huge, intrinsic question mark over them. By contrast DBT can shrink the
size of that question mark to the point of *scientific* certainty...which is the
strongest kind of certainty we can achieve about the natural world.



--

-S.

"They've got God on their side. All we've got is science and reason."
-- Dawn Hulsey, Talent Director