Reply
 
Thread Tools Display Modes
  #1   Report Post  
 
Posts: n/a
Default Stereophile feels the heat.

There have been three articles on the web pages recently about the place,
or not, of blind testing. We had a double rehash of the editor's epiphany
that if you make yourself subject to all the standard conditions known to
produce subjective effects based on false perceptions existing only in the
brain, you will have those false perceptions. This was the basis of his
"debate" offering reported before in print and again in the web pieces.

This week we have another's jousting at blind testing. Nothing new
really, a redo of old tired arguments that boil down to saying even if the
gear really really has a different enough sound, the testing gets in the
way of a blind test discovering it, all in answer to why the mag doesn't
do blind testing to the benefit of it's readers and with a tactical
"disgruntled" thrown in here and there to make sure the point is not
missed.. Sneak preview, the real motivation of the testing folk is
revealed, saying more about the writer then his targets.

"The Blind Leading the Blind?"

http://www.stereophile.com/asweseeit/805awsi

Is it a coincidence that the "debate" and this spate of articles follows
the rough treatment the editor recieved here and at the hands of james
randi as he tried to defend his mag's attitude toward subjective
"reviewing and support of dubious technical claims"? I think the answer
is yes and they will now want to tell us that all is well in the
subjective enterprise now that this small blip has been met and defeated.
  #2   Report Post  
Steven Sullivan
 
Posts: n/a
Default

wrote:
There have been three articles on the web pages recently about the place,
or not, of blind testing. We had a double rehash of the editor's epiphany
that if you make yourself subject to all the standard conditions known to
produce subjective effects based on false perceptions existing only in the
brain, you will have those false perceptions. This was the basis of his
"debate" offering reported before in print and again in the web pieces.


This week we have another's jousting at blind testing. Nothing new
really, a redo of old tired arguments that boil down to saying even if the
gear really really has a different enough sound, the testing gets in the
way of a blind test discovering it, all in answer to why the mag doesn't
do blind testing to the benefit of it's readers and with a tactical
"disgruntled" thrown in here and there to make sure the point is not
missed.. Sneak preview, the real motivation of the testing folk is
revealed, saying more about the writer then his targets.


"The Blind Leading the Blind?"


http://www.stereophile.com/asweseeit/805awsi

Is it a coincidence that the "debate" and this spate of articles follows
the rough treatment the editor recieved here and at the hands of james
randi as he tried to defend his mag's attitude toward subjective
"reviewing and support of dubious technical claims"? I think the answer
is yes and they will now want to tell us that all is well in the
subjective enterprise now that this small blip has been met and defeated.


Of course, Mr. Iverson erects a straw man. Blind tests *can* indeed
be used to test the subject's hearing acuity. But they *can* indeed also
be used to test a difference one claims to hear --
including, of course, those perceived *during the sighted
part of the test*. 'Acuity' is not an issue in that case.
A difference can already be 'heard'. The question is whether
that individual actually heard that difference.

The anecdote in question about the 1989 test
is the same one Atkinson unspooled during the HE2005 debate.
There, as Mr. Iverson does, he drew an illogical conclusion
about DBTs, from his experience.

Moreover Stereophile harps on this one public DBT as if it somehow
invalidates DBTs. Iverson even goes so far as to claim,

"The fact that any two of us in the same room taking the same test end up
scoring different results is all you need to know about the nature of
blind audio testing."

Menahile, the details of the test, as written up by Atkinson himself,
offer several possible explanations for a small difference:

http://www.stereophile.com/features/113/index5.html

the amps in question, btw were a tube vs solid state:

Adcom GFA-555 vs VTL 300W monoblocks when used to drive B&W 801
loudspeakers

One might also note ,a s JA did:


"Despite the visitors to the show being keen audiophiles, over half those
who took part in these listening tests were unable to reliably hear the
small differences between the two power amplifiers. Part of the reason
must be the high level of sound breakthrough from the adjacent room, as
well as the overcrowded conditions€”it would be hard to imagine someone
either sitting on the floor to the side of one of the speakers, or
standing next to the rear wall behind eight rows of seats, getting much of
a good sound. But in my opinion, the main reason for the lack of
identification was that even keen audiophiles rarely perform the kind of
concentrated listening that I was asking from them in these tests. I was
not surprised to note that some of the high scorers were in fact active in
the high-end industry. Tony Di Chiro of Kinergetics, for example, scored 6
correct out of 7, as did Jon Iverson, the retailer whose comments on the
test appear in this month's "Letters" section [and now webmaster for the
Stereophile website€”JA"

Atkinson, like Iverson, offers his *opinion* that it was the differences
between the listeners, and not the conditions of the test, that mainly
account for the variation in performance.

(And jsut for giggles, how many low-scorers, one wonders, were active in
the high-end industry?)




--

-S
"God is an asshole!" -- Ruth Fisher, 'Six Feet Under'
  #3   Report Post  
 
Posts: n/a
Default

wrote in message ...
There have been three articles on the web pages recently about the place,
or not, of blind testing. We had a double rehash of the editor's epiphany
that if you make yourself subject to all the standard conditions known to
produce subjective effects based on false perceptions existing only in the
brain, you will have those false perceptions. This was the basis of his
"debate" offering reported before in print and again in the web pieces.

This week we have another's jousting at blind testing. Nothing new
really, a redo of old tired arguments that boil down to saying even if the
gear really really has a different enough sound, the testing gets in the
way of a blind test discovering it, all in answer to why the mag doesn't
do blind testing to the benefit of it's readers and with a tactical
"disgruntled" thrown in here and there to make sure the point is not
missed.. Sneak preview, the real motivation of the testing folk is
revealed, saying more about the writer then his targets.

"The Blind Leading the Blind?"

http://www.stereophile.com/asweseeit/805awsi


Let's imagine this hypothetical situation:

You have a room full of audiophiles taking a blind test of amplifiers, a
respected solid state amp selling for $700, and a high end tube amp selling
for $4900. There are 30 people taking this test, and there is a 50% chance
of being right by guesswork alone. There are 7 chances, and when the test
is over the average of the entire group is 3.5--exactly what you would
expect from flipping coins.

But wait! Here is some guy by the name of Iverson who got it right 6 out of
7 tries, only a 6% chance of doing that well by guesswork. We must have a
golden ear here--right?

While we're thinking about this, we notice that there's a guy that was WRONG
6 out of 7 times. What will we say about him? Cloth ear? Hardly. How do
you "earn" a score less than chance? And there's the overall score of 50%
to contend with. Since we have a golden ear in the group, how do we arrange
for the balance of the testees to do worse than chance, a necessity if the
final score is to be average?

I'm sure that any of you, faced with those results would say that the guy
that was right only once out of seven tries was just plain unlucky (unless
you want to insinuate that he was purposely trying to do poorly.) If that's
the case, why not say that Iverson was just plain lucky?

Norm Strong

  #5   Report Post  
 
Posts: n/a
Default

Gene Poon wrote:
wrote:

Let's imagine this hypothetical situation:

You have a room full of audiophiles taking a blind test of amplifiers, a
respected solid state amp selling for $700, and a high end tube amp selling
for $4900. There are 30 people taking this test, and there is a 50% chance
of being right by guesswork alone. There are 7 chances, and when the test
is over the average of the entire group is 3.5--exactly what you would
expect from flipping coins.

But wait! Here is some guy by the name of Iverson who got it right 6 out of
7 tries, only a 6% chance of doing that well by guesswork. We must have a
golden ear here--right?

While we're thinking about this, we notice that there's a guy that was WRONG
6 out of 7 times. What will we say about him?...


I'm sure that any of you, faced with those results would say that the guy
that was right only once out of seven tries was just plain unlucky (unless
you want to insinuate that he was purposely trying to do poorly.)



No, I would say he CONSISTENTLY HEARD a difference and that he just
guessed wrong as to which amp was which.


And you would be wrong. This was a same-different test, and didn't
require subjects to guess which amp was which. They merely had to guess
whether B was the same as A. Norm's statistical analysis is correct, as
usual.

With others, I'm mystified by S-phile's continued fixation on this
issue. None their readers give a hoot about it (at least not the ones
the advertisers care about). Okay, so Atkinson had his fun with Arny.
But Iverson's column appears to have been unprompted, and demonstrates
ignorance on a heroic scale. What's the point?

bob


  #6   Report Post  
Stewart Pinkerton
 
Posts: n/a
Default

On 18 Aug 2005 23:31:08 GMT, Gene Poon wrote:

wrote:

Let's imagine this hypothetical situation:

You have a room full of audiophiles taking a blind test of amplifiers, a
respected solid state amp selling for $700, and a high end tube amp selling
for $4900. There are 30 people taking this test, and there is a 50% chance
of being right by guesswork alone. There are 7 chances, and when the test
is over the average of the entire group is 3.5--exactly what you would
expect from flipping coins.

But wait! Here is some guy by the name of Iverson who got it right 6 out of
7 tries, only a 6% chance of doing that well by guesswork. We must have a
golden ear here--right?

While we're thinking about this, we notice that there's a guy that was WRONG
6 out of 7 times. What will we say about him?...


I'm sure that any of you, faced with those results would say that the guy
that was right only once out of seven tries was just plain unlucky (unless
you want to insinuate that he was purposely trying to do poorly.)



No, I would say he CONSISTENTLY HEARD a difference and that he just
guessed wrong as to which amp was which. His ear is just as golden as
Iverson's because he was able to tell one from the other just as reliably.


You miss the point, which is that if the same number of people in the
same room flipped a coin seven times, you'd get the same spread of
results. Basically, that's why casinos make money................
--

Stewart Pinkerton | Music is Art - Audio is Engineering
  #7   Report Post  
BEAR
 
Posts: n/a
Default

wrote:

There have been three articles on the web pages recently about the place,
or not, of blind testing. We had a double rehash of the editor's epiphany
that if you make yourself subject to all the standard conditions known to
produce subjective effects based on false perceptions existing only in the
brain, you will have those false perceptions. This was the basis of his
"debate" offering reported before in print and again in the web pieces.


The "discussion" - thus far - imho is all nonsense on both "sides."

I, personally, am neither for nor "against" DBTs.

I am, however entirely against unwarranted conclusions, poorly set up
"test" conditions, as well as against "snake oil" and "golden ear
pronouncements."

It is my opinion that it is possible to set up a DBT that *might* yield
results that have meaning, not just statistical significance. This has
yet to be done, afaik, and I don't have the time nor funding to do the
job. If you do, contact me. We can become well known in the field. :- )

It is rather obvious to me why the erstwhile Mr. Atkinson's
VTL/Adcom/B&W "test" was flawed before it started, just as all the other
"tests" I'm aware of have been as well. If ur reading this and don't see
the flaws, sorry, I've laid them out in detail in past posts (use google
search or email me privately, if you *really* want to know).

The bottom line is that thus far there has been no *definitive* research
done, no substantive "DBT tests" and there is no basis for doing
anything but arguing back and forth like a bunch of "chickens squaking
in the hen house." Except if someone or some group actually comes up
with the venue and funding to do these sorts of tests up to the maximum
state of the technology in all respects, and possibly get back somthing
with meaning. Just my 2 cents.

_-_-bear
  #8   Report Post  
Ban
 
Posts: n/a
Default

BEAR wrote:
I am, however entirely against unwarranted conclusions, poorly set up
"test" conditions, as well as against "snake oil" and "golden ear
pronouncements."


Then what is that drivel on your web page with those "completely neutral"
silver cables?

It is my opinion that it is possible to set up a DBT that *might*
yield results that have meaning, not just statistical significance.
This has yet to be done, afaik, and I don't have the time nor funding
to do the job. If you do, contact me. We can become well known in the
field. :- )


This shows you clearly are against DBTs, despite your previous claims.


It is rather obvious to me why the erstwhile Mr. Atkinson's
VTL/Adcom/B&W "test" was flawed before it started, just as all the
other "tests" I'm aware of have been as well. If ur reading this and
don't see the flaws, sorry, I've laid them out in detail in past
posts (use google search or email me privately, if you *really* want
to know).


It seems you feel you are the *only* competent person so far on the planet,
congrats.

snip
Just my 2 cents.

_-_-bear


Hey guy, you are lacking a scientific education. You might be talented, but
with this handicap you really can come only up to a certain point.

--
ciao Ban
Bordighera, Italy
Reply
Thread Tools
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Stereophile...source of all this bitterness?...Not! lcw999 Audio Opinions 6 June 27th 05 03:17 PM
- Stereophile Show Report Steven R. Rochlin Vacuum Tubes 0 April 27th 05 01:23 PM
Stereophile Tries To Come Clean About The DiAural Fiasco Arny Krueger Audio Opinions 9 November 23rd 04 06:21 PM
Tubes V's Semiconductors in Military. Brian Bower Vacuum Tubes 137 September 7th 04 09:25 AM
Note to the Idiot George M. Middius Audio Opinions 222 January 8th 04 08:13 PM


All times are GMT +1. The time now is 04:33 PM.

Powered by: vBulletin
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 AudioBanter.com.
The comments are property of their posters.
 

About Us

"It's about Audio and hi-fi"