This is excerpted from a thread on the diyaudio.com website

I thought Arny might wish to weigh in either here or there on the matter.


If a single person can detect a change reliably only 0.5% more often
than by chance alone, controlled testing (including possible ABX) can
absolutely be used to confirm that ability to a high degree of certainty.



Absolutely. HOWEVER, if I wish to demonstrate THE PROBABLE ABSENCE a small
difference (say 0.22/0.17) to a level of significance of .05 I require a
very large sample.

Moreover, if 32 People all find that they can identify a given change to a
..2 certainty level the whole group requires to be aggregated and requires to
now allow the audibility of the suggested effect. However, if I insist that
ALL 32 People show a .05 certainty level I could claim (and thius what is
happening quite frequently) that no-one could hear the effect with any
significant certainty, while I have in fact data that supports the position
that the effect is audible.

For those who have not studied Statistics and wish to get a reasonable idea
of the position and why I will continue to insist on it are invited to
consult the discussion previously published in Stereophile:

The Highs & Lows of Double-Blind Testing

Do not be alarmed by the source in a subjectivist audio magazine, I feel the
discussion presented there is completely fair and allows both sides to
represent their positions well. I leave it to the genteele reader to draw
their own conclusions. Here just one quote I absolutely love:

"When data are nonsignificant, one scientist may conclude that differences
are inaudible, another may conclude that it is wiser to withhold judgment
(because, for example, it is always possible that ancillary equipment used
in the listening test masked otherwise audible differences), another may
decide to issue challenges, and a fourth scientist may decide to have spare
ribs for dinner. These four scientists, having decided what interpretation
to make when listening data are nonsignificant, may be interested in the
probability that their significance test will label data as nonsignificant
when differences are audible, forcing them to make that interpretation
rather than correctly conclude that differences are audible.

For example:
..Scientist 1 wants to know the risk of concluding that differences are
inaudible when differences are, in fact, audible.
..Scientist 2 wants to know the risk that he will withhold judgment when
differences are, in fact, audible.
..Scientist 3 wants to know the risk that he will issue challenges when
differences are, in fact, audible.
..Scientist 4 is beneath contempt because he is eating spare ribs while I am
hungry and writing this damn letter!"

You may go there by using the following:
http://www.diyaudio.com/forums/showt...1&goto=newpost

"Michael McKelvy" wrote in message
ink.net

This is excerpted from a thread on the diyaudio.com website

I thought Arny might wish to weigh in either here or there on the
matter.

If a single person can detect a change reliably only 0.5% more
often than by chance alone, controlled testing (including possible
ABX) can absolutely be used to confirm that ability to a high degree
of certainty.

Absolutely. HOWEVER, if I wish to demonstrate THE PROBABLE ABSENCE a
small difference (say 0.22/0.17) to a level of significance of .05 I
require a very large sample.

I really don't care about what people can't hear, I care about what people
can hear and the experiences they go through in order to do that.

As far as what people don't hear, the most important and meaningful results
are obtained when people try to hear these vanishingly small differences for
themselves. That's why the www.pcabx.com web site exists - for people to try
to hear small differences for themselves.

For example, as you may know, several of us have reliably heard small
diffrences between good power amps. Knowing what we went through to
accomplish that, and what we actually heard in terms of an audible
difference, is the most personally important finding.

Would I even spit in a bucket over the difference I heard? No way!

Arny:

At the time the Stereophile article was written or at any point since, has
Dave Clark had any financial in ABX devices?

"Michael McKelvy" wrote in message
ink.net


At the time the Stereophile article was written or at any point
since, has Dave Clark had any financial (interest) in ABX devices?

Good question. I don't remember if we had sold ABX by 1985 or not. That's
when his financial interest in ABX ended.

"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on double-blind
testing. It is http://www.stereophile.com/features/141 .

John Atkinson
Editor, Stereophile

"John Atkinson" wrote in message
om
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on double-blind
testing. It is http://www.stereophile.com/features/141 .

Except this isn't an artible about double-blind testing. It's really true
confessions and misapprehensions of people who are pretty clueless about
even the basics of subjective testing.

"Arny Krueger" wrote in message

"John Atkinson" wrote in message
om
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on
double-blind testing. It is http://www.stereophile.com/features/141 .

Since the normals who support Atkinson love to make a big point out of
trivail spelling errors:

Correction:

Except this isn't an article about double-blind testing. It's really
true confessions and misapprehensions of people who are pretty
clueless about even the basics of subjective testing.

On Sat, 4 Sep 2004 12:50:14 -0400, "Arny Krueger"
wrote:

Since the normals who support Atkinson love to make a big point out of
trivail spelling errors:

As do you, of course.

dave weil said:

Since the normals who support Atkinson love to make a big point out of
trivail spelling errors:

As do you, of course.

According to Arny-logic, he would be an Atkinson supporter as well as
a "normal" (note quotation marks as to not confuse Arny any further).
Who said a negative can't be proven?

;-)

--
Sander deWaal
"SOA of a KT88? Sufficient."

In article ,
dave weil wrote:

On Sat, 4 Sep 2004 12:50:14 -0400, "Arny Krueger"
wrote:

Since the normals who support Atkinson love to make a big point out of
trivail spelling errors:

As do you, of course.

And provides further evidence for the proposition that any commentary on
spelling will itself contain a typo.

Stephen

"Arny Krueger" wrote in message
...
"John Atkinson" wrote in message
om
You didn't include the link to the Stereophile article on
double-blind testing. It is http://www.stereophile.com/features/141 .

Except this isn't an artible about double-blind testing. It's really
true confessions and misapprehensions of people who are pretty clueless
about even the basics of subjective testing.

It looks as if you didn't read it, Mr. Krueger. The meat of the reprint
is an article by Les Leventhal, of the University of Manitoba's Psychology
Department, based on his Audio Engineering Society paper, "How
Conventional Statistical Analyses Can Prevent Finding Audible Differences
In Listening Tests," Preprint 2275 (C-9), which had been presented at the
79th AES Convention in New York, October 1985 and later reprinted in the
JAES, meaning it had passed peer review.

In addition to Mr. Leventhal's contributions, the reprint included
comments from myself and from J. Gordon Holt, as well as from Tom
Nousaine, David Carlstrom, David Clark, and E. Brad Meyer. Are you
seriously suggesting that none of us have a clue about the "basics
of subjective testing"?

John Atkinson
Editor, Stereophile

"John Atkinson" wrote in message
om
"Arny Krueger" wrote in message
...
"John Atkinson" wrote in message
om

You didn't include the link to the Stereophile article on
double-blind testing. It is http://www.stereophile.com/features/141
..
Note that Atkinson quickly becomes totally confused and transfers my
comment on a Stereophile article to an AES paper

Except this isn't an artible about double-blind testing. It's really
true confessions and misapprehensions of people who are pretty
clueless about even the basics of subjective testing.

It looks as if you didn't read it, Mr. Krueger.

It looks to me like you are free-associating, Mr. Atkinson.

The meat of the
reprint is an article by Les Leventhal, of the University of
Manitoba's Psychology Department,

It's true that your article contains very little meat Atkinson, and what
little meat it does contain was not written by you.

based on his Audio Engineering
Society paper, "How Conventional Statistical Analyses Can Prevent
Finding Audible Differences In Listening Tests," Preprint 2275 (C-9),
which had been presented at the 79th AES Convention in New York,
October 1985 and later reprinted in the JAES, meaning it had passed
peer review.

My comments also seem apply to Levinthal's paper, as the subsequent
disembowelments of it in the JAES (cited below) show quite clearly.

Leventhal's little rant passed peer review just long enough for subsequent
disembowelment.

BTW Atkinson, you left out the two published corrections/partial retractions
by Levinthal.

Then there are the subsequent JAES papers that disemboweled Leventhal's
papers:

Comments on "Type 1 and Type 2 Errors in the Statistical Analysis of
Listening Tests" and Author's Replies 674942 bytes (CD aes4)
Author(s): Shanefield, Daniel; Clark, David; Nousaine, Tom; Leventhal, Les
Publication: Volume 35 Number 7/8 pp. 567·572; July 1987

Transformed Binomial Confidence Limits for Listening Tests 468821 bytes (CD
aes5)
Author(s): Burstein, Herman
Publication: Volume 37 Number 5 pp. 363·367; May 1989
Abstract: A simple transformation of classical binomial confidence limits
provides exact confidence limits for the results of a listening test, such
as the popular ABX test. These limits are for the proportion of known
correct responses, as distinguished from guessed correct responses.
Similarly, a point estimate is obtained for the proportion of known correct
responses. The transformed binomial limits differ, often markedly, from
those obtained by the Bayesian method.

Approximation Formulas for Error Risk and Sample Size in ABX Testing 442116
bytes (CD aes4)
Author(s): Burstein, Herman
Publication: Volume 36 Number 11 pp. 879·883; November 1988
Abstract: When sampling from a dichotomous population with an assumed
proportion p of events having a defined characteristic, the binomial
distribution is the appropriate statistical model for accurately
determining: type 1 error risk (symbol); type 2 error risk (symbol); sample
size n based on specified (symbol) and (symbol) and assumptions about p; and
critical c (minimum number of events to satisfy a specified [symbol]). Table
3 in [1] pre;sents such data for a limited number of sample sizes and p
values. To extend the scope of Table 3 to most n and p, we present
approximation formulas of substantial accuracy, based on the normal
distribution as an approximation of the binomial.

In addition to Mr. Leventhal's contributions, the reprint included
comments from myself and from J. Gordon Holt, as well as from Tom
Nousaine, David Carlstrom, David Clark, and E. Brad Meyer.

Inability to distinguish between the Stereophile article I mentioned, and an
AES paper I didn't mention noted.

Are you
seriously suggesting that none of us have a clue about the "basics
of subjective testing"?

Since "the none of us"were from an AES paper I wasn't commenting on, I'm not
commenting on them.

Are you totally confused or what, Atkinson? Why are you so hot to cite a
paper that was subsequently debunked so thoroughly?

"Arny Krueger" wrote in message
...
"John Atkinson" wrote in message
om
"Arny Krueger" wrote in message
...
"John Atkinson" wrote in message
om
You didn't include the link to the Stereophile article on
double-blind testing. It is http://www.stereophile.com/features/141

Note that Atkinson quickly becomes totally confused and transfers my
comment on a Stereophile article to an AES paper

As I said, Mr. Krueger, you didn't appear to read the article at the
link
I gave, which includes a summarized version of the text of Professor
Leventhal's AES paper. That's why I pointed this out to you.

Except this isn't an artible about double-blind testing. It's really
true confessions and misapprehensions of people who are pretty
clueless about even the basics of subjective testing.

It looks as if you didn't read it, Mr. Krueger...

snip of Mr. Kruger's opinoions of the Leventhal AES paper, which I
am sure he believes to be true but are irrelevant to the specific
point
I was making

In addition to Mr. Leventhal's contributions, the reprint included
comments from myself and from J. Gordon Holt, as well as from Tom
Nousaine, David Carlstrom, David Clark, and E. Brad Meyer.

Inability to distinguish between the Stereophile article I mentioned,
and an AES paper I didn't mention noted.

As I pointed out, the article and the AES paper are essentially the
same.
However, the Stsreophile article includes comments form several
authors,
not just Leventhal, which you would have known had you read past the
first
page. :-)

Are you seriously suggesting that none of us have a clue about the
"basics of subjective testing"?

Since "the none of us" were from an AES paper I wasn't commenting on,
I'm not commenting on them.

The authors mentioned contributed to the Stereophile article to which
you were referring, not the AES paper that was also summarized in the
article. So it is fair to assume that your comment "It's really true
confessions and misapprehensions of people who are pretty clueless
about
even the basics of subjective testing" refers to me, to Gordon Holt,
to Les Leventhal, and to Tom Nousaine, David Carlstrom, David Clark,
and
E. Brad Meyer, all of whom contribued to the Stereophile reprint to
which you were referring.

As I said, had you actually read past the first page of the article at
http://www.stereophile.com/features/141, you would have known this. As
it now stands, you are on record as stating that your associates Tom
Nousaine, David Clark, and David Carlstrom "are pretty clueless about
even the basics of subjective testing." :-)

I am sure they will explain to you that you are wrong about this. :-)

John Atkinson
Editor, Stereophile

In article , "Arny Krueger"
wrote:


Leventhal's little rant passed peer review just long enough for subsequent
disembowelment.


Arny, are you really stupid enough to belive what you claim?




Then there are the subsequent JAES papers that disemboweled Leventhal's
papers:

Comments on "Type 1 and Type 2 Errors in the Statistical Analysis of
Listening Tests" and Author's Replies 674942 bytes (CD aes4)
Author(s): Shanefield, Daniel; Clark, David; Nousaine, Tom; Leventhal, Les
Publication: Volume 35 Number 7/8 pp. 567·572; July 1987


This is a letter to the editor, and not a regular journal article.
See below for detailed comments on this, after we check the only real
journal articles you cited.




Transformed Binomial Confidence Limits for Listening Tests 468821 bytes (CD
aes5)
Author(s): Burstein, Herman
Publication: Volume 37 Number 5 pp. 363·367; May 1989
Abstract: A simple transformation of classical binomial confidence limits
provides exact confidence limits for the results of a listening test, such
as the popular ABX test. These limits are for the proportion of known
correct responses, as distinguished from guessed correct responses.
Similarly, a point estimate is obtained for the proportion of known correct
responses. The transformed binomial limits differ, often markedly, from
those obtained by the Bayesian method.


There is nothing in this article that debunks Leventhal's 1986 JAES
article "Type 1 and Type 2 Errors in the Statistical Analysis of Listening
Tests" which is the final version of the preprint.

The only places that Burstein even mentions Leventhal's JAES article are
in footnotes:

"Leventhal in [3] also justly suggests that c/n be evaluated in terms
of the probability of type 2 error."

and

"The relationship between p_k and p_c in Eq. (2) also applies to other
instances where it is desired to differentiate between the known response
rate and the correct response rate. For example, Leventhal [3] in his
Table 3 lists several effect sizes, namely, hypothesized correct-response
rates in the population. These can be converted to known-response rates
by Eq. (2). Thus an effect size of 0.75 becomes a known-response rate of
(2 X 0.75) - 1 = 0.50. It is assumed here, as in Leventhal's Table 3, that
a subject is choosing between two components. If more than two components
are involved, it would be necessary to use Eq. (2a) below."

Other than a bibliography entry, that is it.

Burstein does mention a different Leventhal paper (in the BAS Speaker) and
he indicates that he prefers p_k to p_c, but these are equivalent ways to
express effect size. There is no debunking of Leventhal in this paper.

Arny, you claimed this disemboweled Leventhal's paper, but
you can't find anything in this paper to support your claim.

Put up or shut up.







Approximation Formulas for Error Risk and Sample Size in ABX Testing 442116
bytes (CD aes4)
Author(s): Burstein, Herman
Publication: Volume 36 Number 11 pp. 879·883; November 1988
Abstract: When sampling from a dichotomous population with an assumed
proportion p of events having a defined characteristic, the binomial
distribution is the appropriate statistical model for accurately
determining: type 1 error risk (symbol); type 2 error risk (symbol); sample
size n based on specified (symbol) and (symbol) and assumptions about p; and
critical c (minimum number of events to satisfy a specified [symbol]). Table
3 in [1] pre;sents such data for a limited number of sample sizes and p
values. To extend the scope of Table 3 to most n and p, we present
approximation formulas of substantial accuracy, based on the normal
distribution as an approximation of the binomial.


Far from debunking Leventhal's paper, this is a whole-hearted endorsement
of it.

It starts as follows.

"This paper is principally an extension of Table 3 in the paper by
Leventhal [1], which correctly stresses the frequent importance of
considering type 2 error risk as well as type 1 in testing a sample from a
dischotomous population."

Burstein goes on to write:

"Our purpose is to provide approximation formulas that extend the scope
of Leventhal's Table 3 to most values of sample size n and proportion p."



Later, as he presents his formulas, he gives worked examples and shows
that their numerical values agree with Leventhal's.

Arny, you claimed this disemboweled Leventhal's paper, but
you can't find anything in this paper to support your claim.

Put up or shut up.




There remains the "letter to the editor" that you cited.

Shanefield observes that Leventhal provides some mathematical precision,
and he cautions that statistical and practical significance are not the
same.
In short, there is no serious conflict with Leventhal's paper.

Leventhal, of course does not disembowel his own paper.

As another poster observed, Nousaine is confused about probability.
(Were he correct in his view, he'd think that when a subject is guessing
with p = .50, only half the trials were real trials!)

Clark's letter is a rant, but it mostly shows that Clark doesn't seem to
understand elementary statistics at all, but that is not new; it has been
the case as far back as his 1982 JAES paper introducing ABX.

Also, in this letter to the editor, Clark says he never claimed that
differences were inaudible. Perhaps he forgot to read his own JAES paper,
where he wrote

"A 12-bit companded digital delay line was just audible. A 16-bit
linear system was not."

By the way, when I went back to Clark's original JAES paper, I noticed
several problems.

The overall impression is that Clark simply didn't really understand how
using statistical methods works in science, but he knew enough to look
scientific to an untrained reader.

Clark appears to make the common mistake about what the p-value computed
in a standard statistical test is. (It is NOT the "probability that a
particular score from an A/B/X test is due to chance".)

When Clark mentions a JAES paper by Plenge et al as an example of how
Clark can use statistical methods to increase sensitivity, he commits a
gaffe comparable to using an AC voltmeter to measure the (DC) voltage of a
battery.
Plenge's experiment involved Same/Different trials where each subject was
presented with 10 Same and 10 Different pairs of filters. Since the
design of that experiment fixed the numbers of same and different trials,
it was not inherently appropriate to apply a binomial distribution.
Clark's conclusion that some filters were audible is based on Clark's
bogus misapplication of the binomial distribution formula; Plenge does not
do that. Ironically, the design of ABX cleverly avoids that particular
problem.

Statistical science is more than simple calculations---it requires
understanding of how and when to use the techniques. Clark's rant about
Leventhal's JAES paper has to be seen as the work of someone who simply
does not grasp the issues involved.


Are you totally confused or what, Atkinson? Why are you so hot to cite a
paper that was subsequently debunked so thoroughly?

Arny, why are you so hot to cite references that you either haven't read,
or don't understand? The only debunking here is of your claims.

"John Atkinson" wrote in message
om...
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on double-blind
testing. It is http://www.stereophile.com/features/141 .

John Atkinson
Editor, Stereophile

John,
Good article.
I've subscribed now for a year, and IMHO, the magazine has lost it's
spice. It's still a good magazine, but, IMHO, a bit too much "The Village of
the Happy Nice People." Magnets for controversy do better in the long term.

"Robert Morein" wrote in message

"John Atkinson" wrote in message
om...
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on
double-blind testing. It is http://www.stereophile.com/features/141 .

John Atkinson
Editor, Stereophile

John,
Good article.

It's a biased misleading POS designed to appeal to people who want to
believe in magic, not science.

On Sun, 5 Sep 2004 05:08:52 -0400, "Arny Krueger"
wrote:

"Robert Morein" wrote in message

"John Atkinson" wrote in message
om...
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on
double-blind testing. It is http://www.stereophile.com/features/141 .

John Atkinson
Editor, Stereophile

John,
Good article.

It's a biased misleading POS designed to appeal to people who want to
believe in magic, not science.

From the article:

The ABX Comparator system, which I helped develop, has been refined
during the 10 years of its existence by the suggestions of many
audiophiles and scientists. Some hardware improvements of this system
can even be traced to the pages of this magazine (Vol.5 No.5). Other
inputs have resulted in the development of double-blind listening
tests which require no switching. The reason for perfecting listening
tests is to develop the ability to hear sonic improvements when they
exist as sound, rather than as mere claims. To quote the esteemed J.
Gordon Holt on the subject of double-blind testing, "The losers will
be the dissemblers, the frauds, and those skilled in the art of
autohype. The winners, ultimately, will be music and the rest of us
who are interested in the maximal fidelity of reproduced music."—David
Clark
--------------------
Does Mr. Clark now consider you a turncoat, since you've in essence
called him a POS?

Then of course, there's this bit of POS:

"The human brain is best at making sense out of nonsense. Humans tend
to find differences and distinctions whether they exist or not. The
research of Richard M. Warren, Diana Deutsch, and others, confirms
that humans can decipher a word obscured by noise as much as a minute
after a sentence was spoken (footnote 8). This same potential can, at
times, create the wrong word. We can misunderstand and still believe
firmly we heard a word different than the one spoken. This is not a
defect. It allowed our ancestors to survive by detecting threats
through noise. Sometimes they overreacted and called out defences when
no mastodon approached. This did no harm".
---------------------

Totally absurd, innit, Arnold? Taling about prehistoric mammals and
all. What does *that* have to do with audio? Plus, everyone knows how
short the auditory memory is. Who does this guy think he is, going
against conventional wisdom like that?

Once again, RAO should applaud you for alerting it to the horribly
misleading POS that Mr. Atkinson has attempted to foist upon it.

"dave weil" wrote in message

On Sun, 5 Sep 2004 05:08:52 -0400, "Arny Krueger"
wrote:

"Robert Morein" wrote in message

"John Atkinson" wrote in message
om...
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on
double-blind testing. It is
http://www.stereophile.com/features/141 .

John Atkinson
Editor, Stereophile

John,
Good article.

It's a biased misleading POS designed to appeal to people who want to
believe in magic, not science.

From the article:

The ABX Comparator system, which I helped develop, has been refined
during the 10 years of its existence by the suggestions of many
audiophiles and scientists. Some hardware improvements of this system
can even be traced to the pages of this magazine (Vol.5 No.5). Other
inputs have resulted in the development of double-blind listening
tests which require no switching. The reason for perfecting listening
tests is to develop the ability to hear sonic improvements when they
exist as sound, rather than as mere claims. To quote the esteemed J.
Gordon Holt on the subject of double-blind testing, "The losers will
be the dissemblers, the frauds, and those skilled in the art of
autohype. The winners, ultimately, will be music and the rest of us
who are interested in the maximal fidelity of reproduced music."-David
Clark

--------------------
Does Mr. Clark now consider you a turncoat, since you've in essence
called him a POS?

The logical flaw here is that the presence of one or even a few true facts
and well-considered opinions in an article does not prevent the entire
article from still being on balance, a POS. I suspect that Weil would be
able to detect this flaw in someone else's writing, but being highly
challenged in the self-awareness department, he missed this critical point
in his own writing.

Note that neither Weil nor Atkinson quoted the following paragraph from the
Stereophile aritcle on RAO:

"Les Leventhal's critique of the statistical analysis commonly used in blind
subjective testing is misleading, erroneous, and borders on the incompetent.
His letter is written in a style that prompts the casual reader to think
"Someone has finally figured out what's wrong with all those blind tests
where they don't hear anything." Not only has Leventhal failed to prove his
case; he has demonstrated his own lack of understanding of how the
audiophile benefits from double-blind testing. "

Dave was being a gentleman at this point and didn't call Leventhal's
weirdness a POS. But he pretty well communicated the same basic idea.

Apparently, the absence of a snapply comeback from Atkinson means that he
reread page 5 of his own article.

(John Atkinson) wrote in message . com...
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on double-blind
testing. It is http://www.stereophile.com/features/141 .

John Atkinson
Editor, Stereophile

Thank you for posting this--it is very informative. I wish I had read
this a long time ago.

There are three problems here that I had suspected for a while. The
first, which isn't the main focus of the article, is generalizability,
which Clark conceded at the outset. Results are only valid for the
two pieces of equipment under test. Period.

The second is the conclusion reached about equipment from a small
number of trials. The clearest example of this was the conclusion
that PASS and Yamaha sounded the same based on a handful of trials at
Sunshine Stereo.

The third is the difficulty of identifying subtle differences in an
ABX test.

A further important fact emerged in this exchange, which it is
interesting to see has been of such longstanding duration. And that
is the tendency of this same small group of men to impugn the
objectivity and integrity of those who criticize ABX, to claim
criticisms are the result of not taking such tests, and to expound the
virtues of ABX in whatever space available no matter what the topic at
hand. This has clearly been an ongoing multimedia gesamtkunstwerk
since at least 1986. There is also a weird tendency to use
nonsensical logic and nonsequiter in response to basic logical
objections. For instance Nousaine's interpretation of the probability
of a difference being heard as a probability of a difference being
present.

"Jacob Kramer" wrote in message
om
(John Atkinson) wrote in message
. com...
"Michael McKelvy" wrote in message
link.net...
This is excerpted from a thread on the diyaudio.com website
I thought Arny might wish to weigh in either here or there on
the matter.

You didn't include the link to the Stereophile article on
double-blind testing. It is http://www.stereophile.com/features/141 .

John Atkinson
Editor, Stereophile

Thank you for posting this--it is very informative. I wish I had read
this a long time ago.

There are three problems here that I had suspected for a while. The
first, which isn't the main focus of the article, is generalizability,
which Clark conceded at the outset. Results are only valid for the
two pieces of equipment under test. Period.

I'm glad that you finally figured this out Kramer after only how many years
of uselessly pontificating about DBTs?

The second is the conclusion reached about equipment from a small
number of trials. The clearest example of this was the conclusion
that PAS and Yamaha sounded the same based on a handful of trials at
Sunshine Stereo.

Zippy repeatedly claimed that there were mind-blowing differences between
these amps. All of a sudden you're haggling over the number of trials in the
test Kramer?

The third is the difficulty of identifying subtle differences in an
ABX test.

It's no problem at all, if the subtle differences are in fact audible.

A further important fact emerged in this exchange, which it is
interesting to see has been of such longstanding duration. And that
is the tendency of this same small group of men to impugn the
objectivity and integrity of those who criticize ABX, to claim
criticisms are the result of not taking such tests, and to expound the
virtues of ABX in whatever space available no matter what the topic at
hand.


It's not our fault that ABX critics are generally either greatly mistaken or
just plain frauds.

This has clearly been an ongoing multimedia gesamtkunstwerk
since at least 1986.

Ignorance of Clark, David L., "High-Resolution Subjective Testing Using a
Double-Blind Comparator", Journal of the Audio Engineering Society, Vol. 30
No. 5, May 1982, pp. 330-338. noted.

There is also a weird tendency to use
nonsensical logic and nonsequiter in response to basic logical
objections.

Please see the collected writing of a certain Ludivic Mirabel.

For instance Nousaine's interpretation of the probability
of a difference being heard as a probability of a difference being
present.

Double-talk from Kramer.