Re: DBT and Penn and Teller's histerical "water test"

[Nousaine]

Joseph Oberlander wrote:

>
>Stewart Pinkerton wrote:
>
>> Indeed they are, and you can play the same trick with a 'false
>> sighted' AA test in audio, where the audience is told that it's an AB
>> test. The scales fell from my own eyes when I got all excited about
>> the wonderfully smooth treble of a new amplifier - but when I checked,
>> the old one was still connected!
>
>Heh.
>
>If you look at the insides of a piece of electornics/audio equipment,
>you'll see that ALL of the new stuff, ranging from $100 amplifiers
>to $5000+ amplifiers all use components made in the same dozen o
>so offshore companies. Rare exceptions exist, but the buttons,
>remote control IR units, the capacitors, and so on are all from
>the same dozen or so (mostly third-world)sources. All as cheaply
>as possible.
>
>The circuitboards are also usually made in simmilar factories.
>
>So are the cases.
>
>In reality, all you are doing is getting bigger or different versions
>of the same crud components with most consumer-grade electronics.
>
>So? My point?
>
>It all ties in - if the bits that make up the units are all from the
>same source(s), the sound should be very simmilar(unfortunately, too
>often - all simmilar sounding budget crud).
>
>That label is moot, as are expectations as it is all the same
>stuff remarketed and assembled a bit differently.

Several years ago I received 2, at that time, high priced stereo receivers for
evaluation. One had an American brand name. all-black cosmetics, with push
button switches and tough-guy rack handles. The other, european brand name, was
champagne with, leaf switches. Both had a bright display and required a remote
control for set-up.

I unpacked both the them, broke out the owners manuals and began set-up with
the champagne finished one. Things went along swimmingly well until I realized
that I was accidentally using the remote control and the manual from the Black
one.

Stunned I investigated more carefully by bringing in the Black one and placing
directly atop the first unit. Now it was farily obvious that the switches and
display were in the same locations. Not only that but the back panels were not
just 'similar'; they were identical right down to the "Made in Japan"
stickers.

So we had competing products from the Netherlands and the USA that were both
manufactured in the same Japanese factory by a manufacturer who had similar
(identical?) products in the market using their own brand name.

Now it is true that the first two units were not 100% identical, one of them
had a special tuning circuit along with a small extra circuit board in it. But
otherwise they looked completely identical with the case off.

It should surprise no one that they sounded exactly alike either.

[Nousaine]

Steven Sullivan wrote:

>
>Jeff Wiseman > wrote:
>> Steven Sullivan wrote:
>>>
>>> I doubt that the point was that *everyone* 'falls for the gag'. The point
>>> was that people *can* 'fall for' such gags, because perception is often
>>> influenced by suggestion. They certainly do prove that point. The humor
>>> in it is demonstrating such human fallibility, a perennial in comedy.
>>>
>>> As for different waters tasting different in blind tests...so? The crux
>>> of the Penn and Teller joke was that the *same* water was reported to have
>>> tasted very differen, depending on what the drinkers were told about it
>>> beforehand. Sound familiar?
>
>> I understand the point being made and the illustration is pretty
>> funny. However, consider the following:
>
>> 1) Was each sample of water presented in the same cup or a
>> different one? The human tongue can be very sensitive to soap
>> residue, etc.
>
>> 2) Was each sample of the water at the same temperature? Cold
>> water tends to taste significantly better since you can't taste
>> impurities in it as well as in warm water.
>
>> 3) Since all the water was coming out of the same tap, each
>> sample would not necessarily be the same. Water that first came
>> out could have a higher tin/lead/copper content than later for
>> example (that's why were are told to let the water run some
>> before getting a drink).
>
>> 4) How many samples (and how large were they) did each person
>> take. Any water tastes better to me when I'm thirsty. When I'm
>> full of water, more water doesn't taste as appetizing :-)
>
>Good points all. But then again, maybe water is water.
>
>> My point is that there may well have been very subtle differences
>> that were detectable only because the recipients were told to
>> watch for them. Normally these differences might be slight enough
>> to go unnoticed. And yet here again the issue of ridicule seems
>> to be "water is water" -- sort of like the "wire is wire"
>> concept. Someone is observed as detecting a difference when
>> "common sense says there obviously is no difference", and they
>> become the subject of a joke because someone played a mind game
>> on them.
>
>'Wire is wire' is a bad analogy -- because wire *is* wire, unless you've
>mismatched two pieces to a degree likely to produce audible
>difference. Such wires will also measure quite differently.
>
>> It may very well have been that it was all psychological. On the
>> other hand, it appears that no effort was made to allow true
>> comparisons to be accurately made (e.g., all water was taken from
>> a large, constantly stirred tank that had stabilized at the same
>> temperature, using the same cup, etc., etc.). Since the human
>> tongue is very sensitive and can vary from time to time (sorta
>> like the human ear you know...), isn't it just possible that
>> there might have been a real detectable difference there mixed in
>> with all that *assumed* psychological stuff? Could there have
>> been a small amount of "taste reality" (whatever that is) mixed
>> in with the so-called power of suggestion?
>
>Could have been. But don't you agree that the power of suggestion
>is strong enough to produce false positives?
>
>The existence of *any* illustrates why controls for suggestion
>are necessary.
>
>--
>-S.

Yes, it does. But here's another good example. In a Candid Camera piece the
'tester' filled some wine glasses from the same bottle. Then he placed a
different open bottle of wine next to each glass. Then he asked "subjects" to
evaluate and rank the wines by taste and quality.

Remembering that every glass came fromthe same bottle; it was interesting when
subjects made strong statements about the differing tastes of the wine in each
glass.

Of course, as some have suggested, it is possible that the show was edited to
show only most outrageous commentary. But the point is made: bias, inadvertant
or intended is incredibly easy to insert in an open comparison and often even
the most rudimentary controls break the 'spell' of 'obvious' differences.

I think the original post where a listener tricked himself with a forgotten
switch position is telling. Every enthusiast has had this very thing happen to
him/her and exactly this experience took me down the path of listener bias
controlled experiments and listening research.

I don't think anyone who is truly honest with himself can let such a challenge
go unpassed, although it seems plainly evident that many do and do so over a
long history.

[Nousaine]

Jeff Wiseman wrote:

>Steven Sullivan wrote:
>>
>> Jeff Wiseman > wrote:
>> > It may very well have been that it was all psychological. On the
>> > other hand, it appears that no effort was made to allow true
>> > comparisons to be accurately made (e.g., all water was taken from
>> > a large, constantly stirred tank that had stabilized at the same
>> > temperature, using the same cup, etc., etc.). Since the human
>> > tongue is very sensitive and can vary from time to time (sorta
>> > like the human ear you know...), isn't it just possible that
>> > there might have been a real detectable difference there mixed in
>> > with all that *assumed* psychological stuff? Could there have
>> > been a small amount of "taste reality" (whatever that is) mixed
>> > in with the so-called power of suggestion?
>>
>> Could have been. But don't you agree that the power of suggestion
>> is strong enough to produce false positives?
>
>Yes, I agree completely and I believe that there is a lot of
>power there as you've said. I've seen it in my own life where I
>could almost tell that I was fooling myself into believing things
>that I really wanted to believe a certain way. But being aware
>that you are suceptable to this can sometimes help you discern
>when you are being fooled.
>
>My point is that when many folks detect differences--especially
>if there seems to be any correlation between different people
>having similar "detections"--I believe that it is unwise to just
>blow it off as a bunch of dumb slobs that have been duped. By
>always allowing a small bit of room for possible unknowns,
>occasionally significant discoveries can be made. Once upon a
>time everyone knew by "common sense" that the world was flat.
>Columbus chose to explore what he thought was an exception to
>that common sense.
>
>- Jeff

Fair enough but when it comes to wire the Columbus clan has had plenty of time
to make a convincing case that wire isn't wire (after all, Columbus was
verified under far more difficult conditions) with a single, repeatable bias
controlled listening experiment that shows it isn't true.

On the other hand, even "allowing" possibility of the reports it may be unwise
to devote significant resources, time or energy to anecdotal reports that the
proponents (makers, sellers and reviewers) of 'wire sound' haven't bothered to
corroborate with credible evidence, don't you think?

[Nousaine]

Steven Sullivan wrote:

>
>S888Wheel > wrote:
>> I said
>
>>>
>>>> An entertaining show definitely. But the folks who don't fall for the
>>>> gag don't make the final edit. They don't proove the point. They
>>>> aren't as entertaining. ironically different waters do taste
>>>> different in blind tests.
>>>
>
>> Arny said
>
>>>
>>>The fact that some people fall for the gag makes the point that sighted
>>>evaluations are very prone to false positives.
>>>
>>>You tell me how to rid sighted evaluations of false positives to the degree
>>>that DBTs do, and I'll stop criticizing sighed evaluations!
>>>
>
>> In the case of food critics and wine critics blind tests would often mean
>> literal blind tests. This is a bad idea. Part of how we evaluate food and
>drink
>> is by looks.
>
>What you're saying is that *preference* for food is influenced by things
>other than taste. So? Preference for audio components is influenced by
>things other than sound, as well. The question is, what do you do if you
>wnat to determine whether two things *taste different*?
>
>> Certianly it's a good idea to take the label off when using
>> inexperienced taste testors in evaluations of drinking water.
>
>It's a good idea to take the lable off when using experienced wine tasters
>in evaluation of wine.
>
>> Believe it or not > many people actually often prefer products in the
>food industry that costs > less. They didn't figure it out with a blind
>fold in most cases. Most people > form opinions about quality with no
>intent of making scientifically valid > claims. Should I do extensive DBT
>taste tests before telling people where I > think the best Mexican food in
>town is?
>
>No, since it's purely a subjective opinion. And it's be irrational to
>think that Mexican food in different restaurants *didn't* taste
>differently.
>
>--
>-S.

But even if it did it would be easy to get people to say that it doesn't.
Actually that's one of the hallmarks of QC and process control and why
hamburgers at any McDonald's taste the same as the one you bought yesterday or
in Chicago.

But the burgers at Wendy's taste (and look) different from Mickey-Ds as well.
BUT no one knows how to make a copper conducter of nominal electrical
competency for the job at hand sound different from any other of similar
competency.

Or if they do, none of them has had the fortitude to show this under bias
controlled conditions.

[Nousaine]

Jeff Wiseman wrote:

>Steven Sullivan wrote:
>>
>> Jeff Wiseman > wrote:
>> > It may very well have been that it was all psychological. On the
>> > other hand, it appears that no effort was made to allow true
>> > comparisons to be accurately made (e.g., all water was taken from
>> > a large, constantly stirred tank that had stabilized at the same
>> > temperature, using the same cup, etc., etc.). Since the human
>> > tongue is very sensitive and can vary from time to time (sorta
>> > like the human ear you know...), isn't it just possible that
>> > there might have been a real detectable difference there mixed in
>> > with all that *assumed* psychological stuff? Could there have
>> > been a small amount of "taste reality" (whatever that is) mixed
>> > in with the so-called power of suggestion?
>>
>> Could have been. But don't you agree that the power of suggestion
>> is strong enough to produce false positives?
>
>Yes, I agree completely and I believe that there is a lot of
>power there as you've said. I've seen it in my own life where I
>could almost tell that I was fooling myself into believing things
>that I really wanted to believe a certain way. But being aware
>that you are suceptable to this can sometimes help you discern
>when you are being fooled.
>
>My point is that when many folks detect differences--especially
>if there seems to be any correlation between different people
>having similar "detections"--I believe that it is unwise to just
>blow it off as a bunch of dumb slobs that have been duped. By
>always allowing a small bit of room for possible unknowns,
>occasionally significant discoveries can be made. Once upon a
>time everyone knew by "common sense" that the world was flat.
>Columbus chose to explore what he thought was an exception to
>that common sense.
>
>- Jeff

Fair enough but when it comes to wire the Columbus clan has had plenty of time
to make a convincing case that wire isn't wire (after all, Columbus was
verified under far more difficult conditions) with a single, repeatable bias
controlled listening experiment that shows it isn't true.

On the other hand, even "allowing" possibility of the reports it may be unwise
to devote significant resources, time or energy to anecdotal reports that the
proponents (makers, sellers and reviewers) of 'wire sound' haven't bothered to
corroborate with credible evidence, don't you think?

[Wylie Williams]

Just because factories turn out the same products for different brand names
(only the faceplate is different) doesn't mean that they don't make similar
products that differ in performance. I know of one case where a US customer
asked that a standard Taiwan electronic crossover be assembled with some
upgraded parts, like capacitors and resistors beacuse the customer felt that
the basic design was OK, but was being held back by parts selection. The
Parasound HCA2200 amp was not given good reviews until John Curl was asked
to tweak it to the MKII, which I gather was done largely with upgrading
selected parts. That the same factory turns out $99 products and $5000
products means just that , and only that, not that they are the same
products.