Home |
Search |
Today's Posts |
|
#1
|
|||
|
|||
Stereophile feels the heat.
There have been three articles on the web pages recently about the place,
or not, of blind testing. We had a double rehash of the editor's epiphany that if you make yourself subject to all the standard conditions known to produce subjective effects based on false perceptions existing only in the brain, you will have those false perceptions. This was the basis of his "debate" offering reported before in print and again in the web pieces. This week we have another's jousting at blind testing. Nothing new really, a redo of old tired arguments that boil down to saying even if the gear really really has a different enough sound, the testing gets in the way of a blind test discovering it, all in answer to why the mag doesn't do blind testing to the benefit of it's readers and with a tactical "disgruntled" thrown in here and there to make sure the point is not missed.. Sneak preview, the real motivation of the testing folk is revealed, saying more about the writer then his targets. "The Blind Leading the Blind?" http://www.stereophile.com/asweseeit/805awsi Is it a coincidence that the "debate" and this spate of articles follows the rough treatment the editor recieved here and at the hands of james randi as he tried to defend his mag's attitude toward subjective "reviewing and support of dubious technical claims"? I think the answer is yes and they will now want to tell us that all is well in the subjective enterprise now that this small blip has been met and defeated. |
#3
|
|||
|
|||
wrote in message ...
There have been three articles on the web pages recently about the place, or not, of blind testing. We had a double rehash of the editor's epiphany that if you make yourself subject to all the standard conditions known to produce subjective effects based on false perceptions existing only in the brain, you will have those false perceptions. This was the basis of his "debate" offering reported before in print and again in the web pieces. This week we have another's jousting at blind testing. Nothing new really, a redo of old tired arguments that boil down to saying even if the gear really really has a different enough sound, the testing gets in the way of a blind test discovering it, all in answer to why the mag doesn't do blind testing to the benefit of it's readers and with a tactical "disgruntled" thrown in here and there to make sure the point is not missed.. Sneak preview, the real motivation of the testing folk is revealed, saying more about the writer then his targets. "The Blind Leading the Blind?" http://www.stereophile.com/asweseeit/805awsi Let's imagine this hypothetical situation: You have a room full of audiophiles taking a blind test of amplifiers, a respected solid state amp selling for $700, and a high end tube amp selling for $4900. There are 30 people taking this test, and there is a 50% chance of being right by guesswork alone. There are 7 chances, and when the test is over the average of the entire group is 3.5--exactly what you would expect from flipping coins. But wait! Here is some guy by the name of Iverson who got it right 6 out of 7 tries, only a 6% chance of doing that well by guesswork. We must have a golden ear here--right? While we're thinking about this, we notice that there's a guy that was WRONG 6 out of 7 times. What will we say about him? Cloth ear? Hardly. How do you "earn" a score less than chance? And there's the overall score of 50% to contend with. Since we have a golden ear in the group, how do we arrange for the balance of the testees to do worse than chance, a necessity if the final score is to be average? I'm sure that any of you, faced with those results would say that the guy that was right only once out of seven tries was just plain unlucky (unless you want to insinuate that he was purposely trying to do poorly.) If that's the case, why not say that Iverson was just plain lucky? Norm Strong |
#4
|
|||
|
|||
|
#5
|
|||
|
|||
Gene Poon wrote:
wrote: Let's imagine this hypothetical situation: You have a room full of audiophiles taking a blind test of amplifiers, a respected solid state amp selling for $700, and a high end tube amp selling for $4900. There are 30 people taking this test, and there is a 50% chance of being right by guesswork alone. There are 7 chances, and when the test is over the average of the entire group is 3.5--exactly what you would expect from flipping coins. But wait! Here is some guy by the name of Iverson who got it right 6 out of 7 tries, only a 6% chance of doing that well by guesswork. We must have a golden ear here--right? While we're thinking about this, we notice that there's a guy that was WRONG 6 out of 7 times. What will we say about him?... I'm sure that any of you, faced with those results would say that the guy that was right only once out of seven tries was just plain unlucky (unless you want to insinuate that he was purposely trying to do poorly.) No, I would say he CONSISTENTLY HEARD a difference and that he just guessed wrong as to which amp was which. And you would be wrong. This was a same-different test, and didn't require subjects to guess which amp was which. They merely had to guess whether B was the same as A. Norm's statistical analysis is correct, as usual. With others, I'm mystified by S-phile's continued fixation on this issue. None their readers give a hoot about it (at least not the ones the advertisers care about). Okay, so Atkinson had his fun with Arny. But Iverson's column appears to have been unprompted, and demonstrates ignorance on a heroic scale. What's the point? bob |
#6
|
|||
|
|||
On 18 Aug 2005 23:31:08 GMT, Gene Poon wrote:
wrote: Let's imagine this hypothetical situation: You have a room full of audiophiles taking a blind test of amplifiers, a respected solid state amp selling for $700, and a high end tube amp selling for $4900. There are 30 people taking this test, and there is a 50% chance of being right by guesswork alone. There are 7 chances, and when the test is over the average of the entire group is 3.5--exactly what you would expect from flipping coins. But wait! Here is some guy by the name of Iverson who got it right 6 out of 7 tries, only a 6% chance of doing that well by guesswork. We must have a golden ear here--right? While we're thinking about this, we notice that there's a guy that was WRONG 6 out of 7 times. What will we say about him?... I'm sure that any of you, faced with those results would say that the guy that was right only once out of seven tries was just plain unlucky (unless you want to insinuate that he was purposely trying to do poorly.) No, I would say he CONSISTENTLY HEARD a difference and that he just guessed wrong as to which amp was which. His ear is just as golden as Iverson's because he was able to tell one from the other just as reliably. You miss the point, which is that if the same number of people in the same room flipped a coin seven times, you'd get the same spread of results. Basically, that's why casinos make money................ -- Stewart Pinkerton | Music is Art - Audio is Engineering |
#7
|
|||
|
|||
|
#8
|
|||
|
|||
BEAR wrote:
I am, however entirely against unwarranted conclusions, poorly set up "test" conditions, as well as against "snake oil" and "golden ear pronouncements." Then what is that drivel on your web page with those "completely neutral" silver cables? It is my opinion that it is possible to set up a DBT that *might* yield results that have meaning, not just statistical significance. This has yet to be done, afaik, and I don't have the time nor funding to do the job. If you do, contact me. We can become well known in the field. :- ) This shows you clearly are against DBTs, despite your previous claims. It is rather obvious to me why the erstwhile Mr. Atkinson's VTL/Adcom/B&W "test" was flawed before it started, just as all the other "tests" I'm aware of have been as well. If ur reading this and don't see the flaws, sorry, I've laid them out in detail in past posts (use google search or email me privately, if you *really* want to know). It seems you feel you are the *only* competent person so far on the planet, congrats. snip Just my 2 cents. _-_-bear Hey guy, you are lacking a scientific education. You might be talented, but with this handicap you really can come only up to a certain point. -- ciao Ban Bordighera, Italy |
Reply |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
Stereophile...source of all this bitterness?...Not! | Audio Opinions | |||
- Stereophile Show Report | Vacuum Tubes | |||
Stereophile Tries To Come Clean About The DiAural Fiasco | Audio Opinions | |||
Tubes V's Semiconductors in Military. | Vacuum Tubes | |||
Note to the Idiot | Audio Opinions |