In the "Olive and Toole" thread on Jan 29 Sullivan in his best
supercilious manner tells Dave Weil:
" Of course you're 'pretty sure' of that, since you rely on sighted
tests -- which scientist have considered to be pretty bad evidence
for many decades now"
This is the third time in a week he called on his graven images for
help. The second time around he spoke more specifically on behalf of
the National Academy of Sciences.* This is what I said to him in the
"Honest truth about stereo" thread on Jan 29:
"When exactly did the National Acadeny of Sciences say that binding a
few guys eyes, making them listen to one snippet after another and make
Vs on paper for "Same" or "Different" is their aproved research method
in comparing audio components. I doubt also that they'd approve of your
claiming to be their spokesman"
And on Jan29 in the "Honest truth abt. stereo" I asked for the nth.
time for a little scientific thing called EVIDENCE.
"Give reference to any report of a decent-sized , randomly selected
listeners' panel ((ie representative like it is done in any reputable
research) ), auditioning double blind any comparable audio components
whatsoever that concluded with a positive outcome: "Yes, we heard a
difference". Between whatever you choose. Speakers , cartridges- your
pick. Naturally as an aspiring scientist you would want a significant
statistical margin with laid down statistical criteria commonly
recognised in research, right? Something that would be publishable in a
serious professional journal like your JAES."
As usual there was no answer.
A little dissection of Sullivan's guide to the website:- "What do to
do when short of an answer"
A) Announce that you put the impertinent questioner "in
your kill-file"
B) Look for a personal attack handle*
C) If a and b fail I clam up wait 5-6 weeks and then
reemerge and repeat that "science" in general and the Nat. Acad. of
Sciences in particular is on your side. Little danger of the "science"
or Academy to question your credentials. .
He has chosen C when the going got hot, (as often as not in
arguments he himself had started), more times than I can be bothered to
search for and quote. These are just the two latest instances.
Ludovic Mirabel

* Ad B: Two years ago in RAHE he put out thick hints that I forged my
academic credentials. I led him by the nose to get himself deeper in.
When he finally realised that the academic world does not end at 49th
parallel he chose C. No apology, no retraction as usual.


:

I want to clarify my opinion of blind tests as applied to audio
component comparison. They can be a useful manoeuvre on *individual,
personal* basis. Closing one's eyes helps concentration. It also
prevents sighted bias which like every other individual characteristic
affects some more some less. But closing one's eyes while listening
doesn't a "test" make. Single blind, double blind or quadruple blind
we're still individuals with different genes, abilities, experiences ,
likes and dislikes ie. hundreds of "biases". People who like chamber
music will look for and hear different kind of differences from
car-rock devotees.
When it comes to wines I'll listen to Robert Parkers of
this world sighted rather than to thousands of Gallo drinkers blind.
When it comes to violins, pianos, flutes.... take your pick and
complete the sentence. No "test". Just experience and expertise.

As for the ABX protocol with its "Listen to A, next listen
to B, next listen to X and tell if it is same as A or B" it seems to
raise insuperable problems for most people in recognising any
differences at all.
It seems that is not how people listen. That is why Sean
Olive said about his loudspeaker listening session which is AFAIK about
the only existing "scientific" DBT audio component comparison*. (I
quote from memory- not word exact. "We seldom ask for difference..We
ask for preference"). And that is why he did not use the ABX protocol.
Of course psychometricians looking for simple well defined
tasks such as human threshold in recognising, phase difference,
recognition of phase distortion etc. will do it DBT. They also have
statististically significant number of testees, rigid protocol, rigid
statistical critreria and so on. The web efforts of the ABX groupies,
Pinkerton etc are nice, amateurish reports suitable for the
free-for-all internet only-not for acceptance by an editor of a
professional journal like eg. JAES. And gossip about what "industry"
does is just gossip.
Give it up the web forum "scientists. You had plenty of years
to prove that you had a "test" that worked and you failed. In the world
of reproduction of complex musical signals there ain't no "test".There
is only individual expertise and preference.
Ludovic Mirabel