In article , "Harry Lavo"
wrote (re
http://www.hfm-detmold.de/eti/projek...paper_6086.pdf):


It is certainly a test that probably is as good an abx test as could be
done.

Huh?


The authors are obviously believers in abx. Accordingly they downplay the
fact that there are differences as evidenced by four responders.

Four listeners getting at least 15 of 20 correct in 145 attempts is hardly
strong evidence. Is there any other evidence to support this "fact" that
there are differences?

Consider these facts:

For a single run of 20 trials, the probability that someone would get at
least 15 correct just by guessing is .0207, which is about 1 chance in 48.

In an experiment with 145 runs (of 20 trials each) the expected number of
apparently significant results (at the .05 level) is 3 if subjects are
just guessing. That is, over all such experiments, the average number of
20-trial runs with "significant" results is 3 per 145-attempt experiment.

The probability that just guessing in an experiment with 145 attempts to
get 15 or more correct in 20 trials would yield at least 4 successes is
about .3529, so seeing four apparently successful results is not unusual
enough to rule out chance.