Do blind tests need being "calibrated" ?

[[email protected]]

In an older thread about the Meyer/Moran study relating to the
audibility of "bottlenecking" the output of SACD through the A/D/A
section of a CD-recorder, the need of calibration was mentioned:

"Your results weren't calibrated, because you didn't come up with any
measurement of the differences that your experimental setup could
actually detect with your chosen source material and test subjects.
This could have been done by generating test material with known
degradations and determining which of these were audible. In other
words, by generating positive differences that could be detected, not
negative differences which could not be detected."

The same author in a current thread said:

"The real flaw is that there was no calibration of the source
material, equipment, room, etc. as to resolution and suitability to
detect the effects being evaluated".

My question is, is there a real need for calibration or is this just a
demand of audiophiles because the test came up with a negative result?

Klaus

[bob]

On Jun 18, 10:07=A0am, wrote:
>
> My question is, is there a real need for calibration or is this just a
> demand of audiophiles because the test came up with a negative result?

Pseudo-scientific rationalization often relies on the casual use of
scientific terms to make an argument sound more authoritative than it
is. This is no exception. Clearly, a lack of "calibration" did not
unduly concern the actual scientists who served as peer reviewers for
the M&M article.

Objective listening tests have been used thousands of times to
demonstrate audible differences. The results quite reliably correlate
with other methods of determining hearing thresholds. M&M's results
were no exception. (And, it should be noted, not all of their results
were negative: At higher volume levels, the lower noise floor of hi-
res digital was audible in their tests.)

Anyone who disagrees with M&M's findings has an obligation to conduct
a study that produces a different result.

bob

[Audio Empire]

On Fri, 18 Jun 2010 08:51:44 -0700, bob wrote
(in article >):

> On Jun 18, 10:07=A0am, wrote:
>>
>> My question is, is there a real need for calibration or is this just a
>> demand of audiophiles because the test came up with a negative result?
>
> Pseudo-scientific rationalization often relies on the casual use of
> scientific terms to make an argument sound more authoritative than it
> is. This is no exception. Clearly, a lack of "calibration" did not
> unduly concern the actual scientists who served as peer reviewers for
> the M&M article.
>
> Objective listening tests have been used thousands of times to
> demonstrate audible differences. The results quite reliably correlate
> with other methods of determining hearing thresholds. M&M's results
> were no exception. (And, it should be noted, not all of their results
> were negative: At higher volume levels, the lower noise floor of hi-
> res digital was audible in their tests.)
>
> Anyone who disagrees with M&M's findings has an obligation to conduct
> a study that produces a different result.
>
> bob
>

AFAICS, the only "calibration" needed is for levels between
components-under-test to be as closely matched as possible. Loudness
differences will always mask any real differences heard between components
(and in fact will generate differences where none really exist). Also, it
seems to be a characteristic of human audio perception that the louder of two
components under test will always be deemed to sound "better" than the less
loud component. If this is what is meant here by "calibration", then yes, it
is VERY necessary. I would think that anything else would be difficult to
achieve and superfluous - especially if all one was trying to accomplish is
ascertain if two components of the same type (DAC, CD Player, record deck,
Pre-amp, amplifier, tuner, etc) sound 'different".

[[email protected]]

> My question is, is there a real need for calibration or is this just a
> demand of audiophiles because the test came up with a negative result?

If one wanted to establish what threshold of any difference in gear can be
heard then it would be required. If testing to see if for any particular
bit of gear difference can be heard it doesn't. Whatever the threshold if
no difference is detected then it is below the perception threshhold and
that is the point of the test.

When time after time controled listening alone tests fail to support
subjective claims, one of the obvious responses is to question some aspect
of the test instead of the even more obvious inherent problems of
subjective listening experiences.

[Arny Krueger]

> wrote in message


> In an older thread about the Meyer/Moran study relating
> to the audibility of "bottlenecking" the output of SACD
> through the A/D/A section of a CD-recorder, the need of
> calibration was mentioned:

> "Your results weren't calibrated, because you didn't come
> up with any measurement of the differences that your
> experimental setup could actually detect with your chosen
> source material and test subjects.

This is a false claim.

It can be determined from extant psychoacoustic data exactly what kind of
roll-offs are required to cause an audible difference.

Exactly what kind of brick-wall roll-off that is required to cause an
audible difference has been known for decades, based on listening tests.

For example, my www.pcabx.com web site in 2002 had dozens of test files
composed of various wideband musical selections, to which brick wall filters
of various frequencies had been applied. The cut-off frequencies ranged
down from about 45 KHz to about 11 KHz in logical steps.

> This could have been
> done by generating test material with known degradations
> and determining which of these were audible.

A good idea, and this has been done over and over again.

> In other words, by generating positive differences that could be detected,
> not negative differences which could not be
> detected."

There's only one thing with extant data of this kind - it supports the idea
that the CD format is capable of sonically transparent reproduction of
music.

> The same author in a current thread said:

> "The real flaw is that there was no calibration of the
> source material, equipment, room, etc. as to resolution
> and suitability to detect the effects being evaluated".

There's no obvious need to re-invent the wheel.

> My question is, is there a real need for calibration or
> is this just a demand of audiophiles because the test
> came up with a negative result?

I guess the whole issue arises when audiophiles are unaware of, or unwilling
to accept the results of well-known scientific investigations that have been
done over a period of years and decades.

[C. Leeds]

nabob wrote (in message ):

> Anyone who disagrees with M&M's findings has an obligation to conduct
> a study that produces a different result.

I'm not familiar with M&M's study, but this statement is just silly.
There's no reason why anyone here has any obligation to provide
scientific studies to substantiate their opinions. Certainly there's no
mention of this in the group's charter. This isn't a strict scientific
forum - it's a Usenet discussion group.

Naturally, each reader is free to accept or reject opinions as he sees
fit. But let's not stifle discussion by accepting nabob's absurd demand.

[John Nunes]

On 6/20/2010 9:57 AM, C. Leeds wrote:
> nabob wrote (in message ):
>
>> Anyone who disagrees with M&M's findings has an obligation to conduct
>> a study that produces a different result.
>
> I'm not familiar with M&M's study, but this statement is just silly.
> There's no reason why anyone here has any obligation to provide
> scientific studies to substantiate their opinions. Certainly there's no
> mention of this in the group's charter. This isn't a strict scientific
> forum - it's a Usenet discussion group.
>
> Naturally, each reader is free to accept or reject opinions as he sees
> fit. But let's not stifle discussion by accepting nabob's absurd demand.

I don't read minds, but seems quite obvious to me nabob was referring to
those doing serious research. Granted, he shouldn't have used the word
"anybody."

BTW, a "scientific" study conducted to substantiate a personal opinion
isn't science, but rather more like propaganda. An idea that becomes a
hypothesis is not necessarily a personal opinion.

[Ed Seedhouse[_2_]]

On Jun 20, 9:57=A0am, "C. Leeds" > wrote:

> There's no reason why anyone here has any obligation to provide
> scientific studies to substantiate their opinions.

That's true, but on the other hand if one goes a step further and
claims that one's opinions are more than just opinions but facts, then
they should not be surprised or upset when asked to provide their
evidence. And if what they call evidence doesn't meet the evidentiary
standards common to reasonable discourse, then they should not get
upset when other people aren't persuaded to change their minds.

Alas, in this forum we find, time after time, that some people state
their unsubstantiated personal opinions as fact, and not opinions.
And get upset when asked for evidence to substantiate their
extraordinary claims.

[Arny Krueger]

"C. Leeds" > wrote in message


> nabob wrote (in message ):

>> Anyone who disagrees with M&M's findings has an
>> obligation to conduct a study that produces a different
>> result.

> I'm not familiar with M&M's study, but this statement is
> just silly. There's no reason why anyone here has any
> obligation to provide scientific studies to substantiate
> their opinions.

That's right. Nobody has any obligation to do even one little thing to
support their claims. Then, every reasonable person recognizes the claims
for what they are, unsubstantiated, unsupported claims, and simply moves on.

> Certainly there's no mention of this in
> the group's charter. This isn't a strict scientific forum
> - it's a Usenet discussion group.

It's a rather divided forum. On one side we have people who defend
unsubstantiated claims, and on the other side we have people who are
themselves capable of making claims and supporting them with reliable,
well-thought out evidence.

> Naturally, each reader is free to accept or reject
> opinions as he sees fit.

Exactly. Those who wish to suspend disbelief can continue to do so, and say
what they will while they are doing it.

> But let's not stifle discussion
> by accepting nabob's absurd demand.

Nabob's demands only apply to people who consider themselves rational.

[Jenn[_2_]]

In article >,
Ed Seedhouse > wrote:

> Alas, in this forum we find, time after time, that some people state
> their unsubstantiated personal opinions as fact, and not opinions.
> And get upset when asked for evidence to substantiate their
> extraordinary claims.

We also find that some people make it clear that their opinion about the
sound of ____________ (software, hardware, etc.) is just that: their
opinion.

[Audio Empire]

On Mon, 21 Jun 2010 06:55:46 -0700, Arny Krueger wrote
(in article >):

> "C. Leeds" > wrote in message
>
>=20
>> nabob wrote (in message ):
>=20
>>> Anyone who disagrees with M&M's findings has an
>>> obligation to conduct a study that produces a different
>>> result.
>=20
>> I'm not familiar with M&M's study, but this statement is
>> just silly. There's no reason why anyone here has any
>> obligation to provide scientific studies to substantiate
>> their opinions.
>=20
> That's right. Nobody has any obligation to do even one little thing to=20
> support their claims. Then, every reasonable person recognizes the clai=
ms=20
> for what they are, unsubstantiated, unsupported claims, and simply move=
s on.
>=20
>> Certainly there's no mention of this in
>> the group's charter. This isn't a strict scientific forum
>> - it's a Usenet discussion group.
>=20
> It's a rather divided forum. On one side we have people who defend=20
> unsubstantiated claims, and on the other side we have people who are=20
> themselves capable of making claims and supporting them with reliable,=20
> well-thought out evidence.
>=20
>> Naturally, each reader is free to accept or reject
>> opinions as he sees fit.
>=20
> Exactly. Those who wish to suspend disbelief can continue to do so, and=
say=20
> what they will while they are doing it.
>=20
>> But let's not stifle discussion
>> by accepting nabob's absurd demand.
>=20
> Nabob's demands only apply to people who consider themselves rational.=20
>=20
>=20

Most people on this forum consider themselves "rational" irrespective of=20
whether or not they agree with Mr. Kruger 100% of the time. Audio, as a=20
hobby, is more akin to politics than it is to science or engineering in t=
hat=20
people have strong opinions about almost everything, and not all audio=20
subjects can be distilled into their basic technologies or yield to what=20
logically "ought to be" just because some "expert" says it should.=20

Take vinyl, for instance. Good vinyl has always been one of those areas w=
here=20
the whole is certainly greater than the mere sum of it's parts. "Hi-Fi=20
Choice" writer Jimmy Hughes* put it eloquently and succinctly recently in=
an=20
opinion piece he wrote about Linn Sondek's LP-12 record deck:

"On a point-by-point basis, CD beats vinyl at every juncture. It has less=
=20
background noise, a broader dynamic range, greater separation, and lower=20
peak-level distortion. Yet, against all the odds, vinyl offers a musical=20
integrity that transcends it's limitations creating results that are spec=
ial=20
and unique. Despite its faults, LP often recreates the emotional content =
of=20
the music (and the performance). What's harder to get from CD is the same=
=20
musical honesty and cohesive integrity you achieve with LPs =AD the sense=
of=20
real people playing real instruments in a tangible space."=20

I don't know about the rest of you, but that's what I listen to recorded=20
music to experience.=20

I might also add that I make my own digital recordings that have all of t=
he=20
above - in spades. But you can't buy digital recordings on the open marke=
t=20
that sound as good. I don't pretend to understand why this should be so, =
it's=20
certainly not that difficult to make spectacular sounding digital even at=
=20
16-bit/44.1 KHz Redbook standards. I have thousands of commercial digital=
=20
recordings on CD, DVD-A, SACD, and even DAT. None sound as good as my own=
=20
"home-made" digital creations or even as good as the best available from=
=20
vinyl.=20

* "Hi Fi Choice" is a magazine that I admire more and more. They not only=
=20
listen subjectively, but they also do measurement tests of equipment and =
they=20
do DBTs as well. Like most hobbyist publications, the Brits just seem to =
do=20
it better. Better writing, better content, more in-depth evaluations...

[bob]

On Jun 20, 12:57=A0pm, "C. Leeds" > wrote:
> nabob wrote (in message ):
>
> > Anyone who disagrees with M&M's findings has an obligation to conduct
> > a study that produces a different result.
>
> I'm not familiar with M&M's study, but this statement is just silly.
> There's no reason why anyone here has any obligation to provide
> scientific studies to substantiate their opinions.

To clarify, I would draw a distinction between opinions and technical
claims:

* "I think hi-res formats make an audible difference, and I don't care
what the JAES says" is an opinion.

* "The JAES study is flawed because its methodology was not
calibrated" is a technical claim (albeit a pretty meaningless one).

I wouldn't expect scientific back-up for the first statement, though I
think it's perfectly reasonable to ask whether there is any scientific
back-up for it. (Also, I think it's perfectly reasonable to note that
if you make the first part of the statement, without said back-up, you
are also making the second part.)

However, I think anyone making the second statement *does* have an
obligation to present scientific support for it.

bob

[Arny Krueger]

"Audio Empire" > wrote in message

> On Mon, 21 Jun 2010 06:55:46 -0700, Arny Krueger wrote
> (in article >):
>
>> "C. Leeds" > wrote in message
>>
>>
>>> nabob wrote (in message
>>> ):
>>
>>>> Anyone who disagrees with M&M's findings has an
>>>> obligation to conduct a study that produces a different
>>>> result.
>>
>>> I'm not familiar with M&M's study, but this statement is
>>> just silly. There's no reason why anyone here has any
>>> obligation to provide scientific studies to substantiate
>>> their opinions.
>>
>> That's right. Nobody has any obligation to do even one
>> little thing to support their claims. Then, every
>> reasonable person recognizes the claims for what they
>> are, unsubstantiated, unsupported claims, and simply
>> moves on.
>>
>>> Certainly there's no mention of this in
>>> the group's charter. This isn't a strict scientific
>>> forum - it's a Usenet discussion group.
>>
>> It's a rather divided forum. On one side we have people
>> who defend unsubstantiated claims, and on the other side
>> we have people who are themselves capable of making
>> claims and supporting them with reliable, well-thought
>> out evidence.
>>
>>> Naturally, each reader is free to accept or reject
>>> opinions as he sees fit.
>>
>> Exactly. Those who wish to suspend disbelief can
>> continue to do so, and say what they will while they are
>> doing it.
>>
>>> But let's not stifle discussion
>>> by accepting nabob's absurd demand.
>>
>> Nabob's demands only apply to people who consider
>> themselves rational.

> Most people on this forum consider themselves "rational"
> irrespective of whether or not they agree with Mr. Kruger
> 100% of the time.

Seems like an attempt to unecessarily make this discussion personal.

> Audio, as a hobby, is more akin to
> politics than it is to science or engineering in that
> people have strong opinions about almost everything, and
> not all audio subjects can be distilled into their basic
> technologies or yield to what logically "ought to be"
> just because some "expert" says it should.

Interesting idea, that audio is more like politics than science. I guess
that would be a revelation of your true state of mind. Most audio
professionals see Audio as being both and art and a science. Being involved
in several different arts, it has been my experience that once you exclude
the dilentantes who by defintion cannot fully appreciate the art and science
aspects of audio, audio is no more political than many other arts.

> Take vinyl, for instance.

Vinyl seems to be irrelevant to the discussion since only a tiny minority of
people who are interested in audio are interested in vinyl.

[bob]

On Jun 21, 5:42=A0pm, Audio Empire > wrote:

> Most people on this forum consider themselves "rational" irrespective of
> whether or not they agree with Mr. Kruger 100% of the time. Audio, as a
> hobby, is more akin to politics than it is to science or engineering in t=
hat
> people have strong opinions about almost everything, and not all audio
> subjects can be distilled into their basic technologies or yield to what
> logically "ought to be" just because some "expert" says it should.

But some audio subjects CAN be distilled into their basic
technologies, so to speak. Some people apparently refuse to recognize
this. I suspect that's what Arny was getting at about "rational."

> Take vinyl, for instance. Good vinyl has always been one of those areas w=
here
> the whole is certainly greater than the mere sum of it's parts. "Hi-Fi
> Choice" writer Jimmy Hughes* put it eloquently and succinctly recently in=
an
> opinion piece he wrote about Linn Sondek's LP-12 record deck:
>
> "On a point-by-point basis, CD beats vinyl at every juncture. It has less
> background noise, a broader dynamic range, greater separation, and lower
> peak-level distortion. Yet, against all the odds, vinyl offers a musical
> integrity that transcends it's limitations creating results that are spec=
ial
> and unique. Despite its faults, LP often recreates the emotional content =
of
> the music (and the performance). What's harder to get from CD is the same
> musical honesty and cohesive integrity you achieve with LPs =AD the sense=
of
> real people playing real instruments in a tangible space."

This is a good example of a statement about audio that is not
rational, in that it refuses to acknowledge what engineering and
science *can* tell us about audio.

<snip>

> * "Hi Fi Choice" is a magazine that I admire more and more. They not only
> listen subjectively, but they also do measurement tests of equipment and =
they
> do DBTs as well. Like most hobbyist publications, the Brits just seem to =
do
> it better. Better writing, better content, more in-depth evaluations...

It always seemed to me that they were just better at putting an
empirical gloss on the usual subjective fluff. I wasn't aware that
they'd done much in the DBT department. Perhaps you are confusing
blind comparisons with something worthy of the term "test"?

bob

[Audio Empire]

On Mon, 21 Jun 2010 18:33:37 -0700, bob wrote
(in article >):

> On Jun 21, 5:42=A0pm, Audio Empire > wrote:
>
>> Most people on this forum consider themselves "rational" irrespective of
>> whether or not they agree with Mr. Kruger 100% of the time. Audio, as a
>> hobby, is more akin to politics than it is to science or engineering in t=
> hat
>> people have strong opinions about almost everything, and not all audio
>> subjects can be distilled into their basic technologies or yield to what
>> logically "ought to be" just because some "expert" says it should.
>
> But some audio subjects CAN be distilled into their basic
> technologies, so to speak. Some people apparently refuse to recognize
> this. I suspect that's what Arny was getting at about "rational."

I agree that many audio subjects CAN be distilled into their basic
technologies and that the analysis of same is incontrovertible. My point is
that logic doesn't always reflect human experience. This is especially true
with something as personal and emotional as listening to music.
>
>> Take vinyl, for instance. Good vinyl has always been one of those areas w=
> here
>> the whole is certainly greater than the mere sum of it's parts. "Hi-Fi
>> Choice" writer Jimmy Hughes* put it eloquently and succinctly recently in=
> an
>> opinion piece he wrote about Linn Sondek's LP-12 record deck:
>>
>> "On a point-by-point basis, CD beats vinyl at every juncture. It has less
>> background noise, a broader dynamic range, greater separation, and lower
>> peak-level distortion. Yet, against all the odds, vinyl offers a musical
>> integrity that transcends it's limitations creating results that are spec=
> ial
>> and unique. Despite its faults, LP often recreates the emotional content =
> of
>> the music (and the performance). What's harder to get from CD is the same
>> musical honesty and cohesive integrity you achieve with LPs =AD the sense=
> of
>> real people playing real instruments in a tangible space."
>
> This is a good example of a statement about audio that is not
> rational, in that it refuses to acknowledge what engineering and
> science *can* tell us about audio.

But this example DOES acknowledge it. Mr. Hughes clearly states that ""On a
point-by-point basis, CD beats vinyl at every juncture. It has less
background noise, a broader dynamic range, greater separation, and lower
peak-level distortion." Looks mighty like an acknowledgement of what
engineering and science *can* tell us about audio to me.

> <snip>
>
>> * "Hi Fi Choice" is a magazine that I admire more and more. They not only
>> listen subjectively, but they also do measurement tests of equipment and =
> they
>> do DBTs as well. Like most hobbyist publications, the Brits just seem to =
> do
>> it better. Better writing, better content, more in-depth evaluations...
>
> It always seemed to me that they were just better at putting an
> empirical gloss on the usual subjective fluff. I wasn't aware that
> they'd done much in the DBT department.

But they do.

> Perhaps you are confusing
> blind comparisons with something worthy of the term "test"?

I am confusing nothing. They say that they do blind comparisons especially
when comparing a bunch of like devices like DACs.

OTOH, they and 'Hi-Fi News' are certainly more entertaining than the US rags,
and that's mostly what magazines are for anyway; entertainment.

[Audio Empire]

On Mon, 21 Jun 2010 18:26:24 -0700, Arny Krueger wrote
(in article >):

> "Audio Empire" > wrote in message
>
>> On Mon, 21 Jun 2010 06:55:46 -0700, Arny Krueger wrote
>> (in article >):
>>
>>> "C. Leeds" > wrote in message
>>>
>>>
>>>> nabob wrote (in message
>>>> ):
>>>
>>>>> Anyone who disagrees with M&M's findings has an
>>>>> obligation to conduct a study that produces a different
>>>>> result.
>>>
>>>> I'm not familiar with M&M's study, but this statement is
>>>> just silly. There's no reason why anyone here has any
>>>> obligation to provide scientific studies to substantiate
>>>> their opinions.
>>>
>>> That's right. Nobody has any obligation to do even one
>>> little thing to support their claims. Then, every
>>> reasonable person recognizes the claims for what they
>>> are, unsubstantiated, unsupported claims, and simply
>>> moves on.
>>>
>>>> Certainly there's no mention of this in
>>>> the group's charter. This isn't a strict scientific
>>>> forum - it's a Usenet discussion group.
>>>
>>> It's a rather divided forum. On one side we have people
>>> who defend unsubstantiated claims, and on the other side
>>> we have people who are themselves capable of making
>>> claims and supporting them with reliable, well-thought
>>> out evidence.
>>>
>>>> Naturally, each reader is free to accept or reject
>>>> opinions as he sees fit.
>>>
>>> Exactly. Those who wish to suspend disbelief can
>>> continue to do so, and say what they will while they are
>>> doing it.
>>>
>>>> But let's not stifle discussion
>>>> by accepting nabob's absurd demand.
>>>
>>> Nabob's demands only apply to people who consider
>>> themselves rational.
>
>> Most people on this forum consider themselves "rational"
>> irrespective of whether or not they agree with Mr. Kruger
>> 100% of the time.
>
> Seems like an attempt to unecessarily make this discussion personal.

Not personal at all. Just noting, Mr. Kruger, that you seem to consider
yourself among the "rational" while implying that those who disagree with
some of your "informed opinions" are NOT rational. The audio hobby doesn't
have to conform to a completely rational approach (although it's certainly
nice when it does) because it's about a totally emotional pleasure -
listening to music (possibly mixed with a degree exorbitant consumerism). If
that makes the music sound better (whatever the individual listener might
consider "better"), then who are you or I to tell them otherwise. I know, for
instance, that, at audio frequencies, wire is just wire. I also know that
countless DBT tests have shown that properly constructed audio cables all
sound exactly the same, and it's certainly OK to state that as fact. But at
the end of the day, if an audiophile believes that a $4000 pair of 1 meter
interconnects, for instance, makes his CD player sound "better" to him, then
it DOES sound better to HIM. Logic has little, if anything to do with it. I
know that those cables don't do ANYTHING over a cheap set of Radio Shack
cables, you know that they don't do ANYTHING, but the owner of the cables
thinks that they do, and that belief enhances his enjoyment of the music.
Does that make him irrational? Not from where I sit.



[ Let's steer this away from the personal immediately,
please. -- dsr ]



>> Audio, as a hobby, is more akin to
>> politics than it is to science or engineering in that
>> people have strong opinions about almost everything, and
>> not all audio subjects can be distilled into their basic
>> technologies or yield to what logically "ought to be"
>> just because some "expert" says it should.
>
> Interesting idea, that audio is more like politics than science. I guess
> that would be a revelation of your true state of mind. Most audio
> professionals see Audio as being both and art and a science. Being involved
> in several different arts, it has been my experience that once you exclude
> the dilentantes who by defintion cannot fully appreciate the art and science
> aspects of audio, audio is no more political than many other arts.
>
>> Take vinyl, for instance.
>
> Vinyl seems to be irrelevant to the discussion since only a tiny minority of
> people who are interested in audio are interested in vinyl.

It's very relevant. It's a case where you (and some others who post here
regularly) have looked at the numbers and decreed the medium useless,
antiquated, obsolete, whatever. As such, it is a perfect example of what I
meant when I said that "not all audio subjects can be distilled into their
basic technologies or yield to what logically "ought to be" just because some
"expert" says it should."

[Arny Krueger]

"Audio Empire" > wrote in message


> Not personal at all. Just noting, Mr. Kruger, that you
> seem to consider yourself among the "rational" while
> implying that those who disagree with some of your
> "informed opinions" are NOT rational.

Again you're being way too personal.

Most of my opinions are not at all personal to just me, and many thoughts of
mine that some people call opinions aren't even just opinons.

This should be pretty obvious from what I write because so many of my posts
reference papers in relevant, refereed scientific journals. Furthermore,
while many of the ideas I share here seem strange to some, they are typical
of some of the best minds in the audio industry - sucessful and well-known
authors, educators, system designers, etc.

There's a problem with audio, and that is the fact that the people who write
for the popular press are frequently poorly-informed about the actual
technology that they write about. For example I think about a certain
highly-influential writer with a strong bias towards vinyl. He has no known
credentials other than his claims that he has an audio system is valued in
the six or seven figures. He has no known professional occupation other than
writing, and cannot possibly be fund his professed life style with the
writer's fees that would normally be paid for his published works. They are
a pittance compared to his investments in high end audio gear.

> The audio hobby
> doesn't have to conform to a completely rational approach
> (although it's certainly nice when it does) because it's
> about a totally emotional pleasure - listening to music
> (possibly mixed with a degree exorbitant consumerism). If
> that makes the music sound better (whatever the
> individual listener might consider "better"), then who
> are you or I to tell them otherwise.

I'll go further than that - the entire audio field (not just its hobby
aspects) does not have to be completely rational since it is generally
agreed that audio is both art and science.

One reason why audio is partially an art is that we currently lack the
science to completely support much of what we do.

> I know, for instance, that, at audio frequencies, wire is just wire.

If you gave science its proper due, you'd be willing to admit that at all
frequencies, wire is just wire.

> I also know that countless DBT tests have shown that
> properly constructed audio cables all sound exactly the
> same, and it's certainly OK to state that as fact.

At this point I would like to point out that the idea of "properly
constructed audio cables" "sound the same" is a truism. Adding no audible
colorations of their own is generally given as being one of the goals of
speaker cables.

The idea of "properly constructed audio cables" "sound the same" is a truism
is also not 100% true. I own at least one piece of professional audio gear
that was sold with speaker cables that were designed to audibly affect the
sound quality of the piece of equipment. They are properly constructed in
the sense that the entire ensemble is intended to be working most accurately
with the colorations due to the cables included in the set. But, the cables
themselves add audible coloration. BTW, the equipment is an NHTPro A10
studio monitor system.

> But at the end of the day, if an audiophile believes that a
> $4000 pair of 1 meter interconnects, for instance, makes
> his CD player sound "better" to him, then it DOES sound
> better to HIM.

That's called solipsism, and solipsism does not disprove science.
Furthermore there is a chance that he by chance has stumbled into a similar
case to the NHTPro A10 - a situation where the colorations of the cables
offset colorations elsewhere in the system. OTOH, perhaps it's the
self-gratifcation of doting on himself by spending that kind of money is
important to him. It's the audio equivalent of smoking very expensive cigars
or taking on the services of a very expensive personal entertainer or
trainer.

If this is what people want to do with their money then there are no laws
against it, and even the highest ethical standards generally given in our
culture do not criticize such things provided that a person is also
charitable.

> Logic has little, if anything to do with it.

I submit that logic still applies. It's just a non-technical kind of logic.

[Scott[_6_]]

On Jun 18, 7:07=A0am, wrote:
> In an older thread about the Meyer/Moran study relating to the
> audibility of "bottlenecking" the output of SACD through the A/D/A
> section of a CD-recorder, the need of calibration was mentioned:
>
> "Your results weren't calibrated, because you didn't come up with any
> measurement of the differences that your experimental setup could
> actually detect with your chosen source material and test subjects.
> This could have been done by generating test material with known
> degradations and determining which of these were audible. In other
> words, by generating positive differences that could be detected, not
> negative differences which could not be detected."
>
> The same author in a current thread said:
>
> "The real flaw is that there was no calibration of the source
> material, equipment, room, etc. as to resolution and suitability to
> detect the effects being evaluated".
>
> My question is, is there a real need for calibration or is this just a
> demand of audiophiles because the test came up with a negative result?
>
> Klaus

Of course it does otherwise you have no way of gauging the sensitivity
of the test. Anull with no calibration has too many variables. was the
test sensitive to audible differences? Were the subjects sensitive to
audible differences? No way to know is there?

[bob]

On Jun 21, 11:42=A0pm, Audio Empire > wrote:

> I agree that many audio subjects CAN be distilled into their basic
> technologies and that the analysis of same is incontrovertible. My point =
is
> that logic doesn't always reflect human experience.

IOW, humans aren't always logical. Sounds like you're thinking the
same way Arny was when he used the term "rational."

> This is especially true
> with something as personal and emotional as listening to music. =A0

I would submit that this has nothing to do with music, and everything
to do with some deep human need 1) to acquire things, and 2) to feel
some sense of efficacy and discernment.

<snip>

> But this example DOES acknowledge it. Mr. Hughes clearly states that ""On=
a
> point-by-point basis, CD beats vinyl at every juncture. It has less
> background noise, a broader dynamic range, greater separation, and lower
> peak-level distortion." Looks mighty like an acknowledgement of what
> engineering and science *can* tell us about audio to me.

OK, now look at the second half of the quote, where he clearly rejects
things science and engineering can tell us about audio:

>> ...Despite its faults, LP often recreates the emotional content of
>> the music (and the performance). What's harder to get from CD is the sam=
e
>> musical honesty and cohesive integrity you achieve with LPs-- the sense =
of
>> real people playing real instruments in a tangible space."

Leaving aside the pseudo-scientific mumbo-jumbo about "recreating
emotional content," it is trivially easy to achieve exactly the same
experience from a CD that one gets from vinyl--just make a CDR of the
vinyl. It takes very little science to demonstrate that.

<snip>

> > Perhaps you are confusing
> > blind comparisons with something worthy of the term "test"?
>
> I am confusing nothing. They say that they do blind comparisons especiall=
y
> when comparing a bunch of like devices like DACs.

Then you ARE confusing comparisons and tests. They do not do DBTs,
they do blind comparisons. There is all the difference in the world.

bob

[Arny Krueger]

"Scott" > wrote in message


> Of course it does otherwise you have no way of gauging
> the sensitivity of the test.

I think that it is instructive to practice a little symmetry here.

The symmetrical question is:
Do sighted tests need to be "calibrated" ?

I've never seen anybody discuss this question, probably because there is
little if any apparent problem with sighted tests lacking sensitivity.
Sighted tests almost always produce results that are favorable to a positive
hypothesis.

Why isn't the fact that sighted tests almost always produce results that are
favorable to a positive hypothesis a problem?

> A null with no calibration has too many variables.

Isn't a postive result with no calibration equally problematical? Don't
sighted evaluations always have more variables than the corresponding `blind
tests since blind tests reduce the number of variables?

[Audio Empire]

On Tue, 22 Jun 2010 07:03:39 -0700, Dick Pierce wrote
(in article >):

> Audio Empire wrote:
>>
>> OTOH, they and 'Hi-Fi News' are certainly more entertaining
> > than the US rags, and that's mostly what magazines are for
> > anyway; entertainment.
>>
>
> Magazines are mostly about generating revenue primarily through
> subscriptions and advertising. They use "entertainment" as one
> element of their business model: they promote the "entertainment"
> of their subscribers as a means of attracting advertising, and
> the subscriptions and newstand sales are merely "objective" data
> to support their pitch to the advertisers.
>
>

My god Dick, what a champion of the obvious you are today 8^).

All entertainment is based on that model. Do you think, for instance, that
commercial TV (or radio) exists solely to entertain you? It seeks to
entertain solely to get you to view the advertising.

Besides, I was talking about magazines from the reader's perspective, not
from the publishers'. They are mostly about entertaining one with articles
and features about subject matter likely to be of interest to the reader.
Occasionally they also inform, but mostly they just entertain.

[Audio Empire]

On Tue, 22 Jun 2010 07:36:00 -0700, bob wrote
(in article >):

> On Jun 21, 11:42=A0pm, Audio Empire > wrote:
>
>> I agree that many audio subjects CAN be distilled into their basic
>> technologies and that the analysis of same is incontrovertible. My point =
> is
>> that logic doesn't always reflect human experience.
>
> IOW, humans aren't always logical. Sounds like you're thinking the
> same way Arny was when he used the term "rational."
>
>> This is especially true
>> with something as personal and emotional as listening to music. =A0
>
> I would submit that this has nothing to do with music, and everything
> to do with some deep human need 1) to acquire things, and 2) to feel
> some sense of efficacy and discernment.
>
> <snip>
>
>> But this example DOES acknowledge it. Mr. Hughes clearly states that ""On=
> a
>> point-by-point basis, CD beats vinyl at every juncture. It has less
>> background noise, a broader dynamic range, greater separation, and lower
>> peak-level distortion." Looks mighty like an acknowledgement of what
>> engineering and science *can* tell us about audio to me.
>
> OK, now look at the second half of the quote, where he clearly rejects
> things science and engineering can tell us about audio:

No, he merely states an observed phenomenon (one that he is far from being
alone in observing) that seems to reject what science and engineering tells
us about vinyl records.

>>> ...Despite its faults, LP often recreates the emotional content of
>>> the music (and the performance). What's harder to get from CD is the sam=
> e
>>> musical honesty and cohesive integrity you achieve with LPs-- the sense =
> of
>>> real people playing real instruments in a tangible space."
>
> Leaving aside the pseudo-scientific mumbo-jumbo about "recreating
> emotional content," it is trivially easy to achieve exactly the same
> experience from a CD that one gets from vinyl--just make a CDR of the
> vinyl. It takes very little science to demonstrate that.

While it is possible to achieve the same emotional experience with CD that
the best vinyl can provide (and I don't mean by merely copying a vinyl record
to CD, either), it is rarely done and it is VERY easy to do. Why this
wouldn't be the goal of all record producers is beyond me, but apparently it
isn't.
>
> <snip>
>
>>> Perhaps you are confusing
>>> blind comparisons with something worthy of the term "test"?
>>
>> I am confusing nothing. They say that they do blind comparisons especiall=
> y
>> when comparing a bunch of like devices like DACs.
>
> Then you ARE confusing comparisons and tests. They do not do DBTs,
> they do blind comparisons. There is all the difference in the world.

Then the difference is lost upon me.

[Audio Empire]

On Tue, 22 Jun 2010 15:13:41 -0700, Dick Pierce wrote
(in article >):

> Audio Empire wrote:
>> On Tue, 22 Jun 2010 07:03:39 -0700, Dick Pierce wrote
>> (in article >):
>>
>>
>>> Audio Empire wrote:
>>>
>>>> OTOH, they and 'Hi-Fi News' are certainly more entertaining
>>>
>>>> than the US rags, and that's mostly what magazines are for
>>>> anyway; entertainment.
>>>
>>> Magazines are mostly about generating revenue primarily through
>>> subscriptions and advertising. They use "entertainment" as one
>>> element of their business model: they promote the "entertainment"
>>> of their subscribers as a means of attracting advertising, and
>>> the subscriptions and newstand sales are merely "objective" data
>>> to support their pitch to the advertisers.
>>>
>>>
>>
>>
>> My god Dick, what a champion of the obvious you are today 8^).
>
> I would be more than happy to conceded the title to you.
>
>> All entertainment is based on that model. Do you think, for instance, that
>> commercial TV (or radio) exists solely to entertain you?
>
> Why would you even assume such that a question like this is necessary
> and not completely rhetorical? Or maybe I missed the sarcastic irony
> emoticon.
>
>> Besides, I was talking about magazines from the reader's perspective, not
>> from the publishers'.
>
> But it is ENTIRELY the publishers choice that sets that
> perspective. When you pick up ANY magazine, you have no
> choice whatsoever what to read, only the choice of what
> NOT to read. The notion that the reader has ANY control
> over the perspective other than refusal is, well, ...

Seems to me that you are arguing a non issue. I read audio magazines to be
entertained and to see what's new in the industry. Magazines are good for
that. Occasionally, a magazine article about a new product piques my interest
such as several articles I've read about the new Magnepan MG-1.7s. The
industry "pundits" are raving about it. I went top my local Maggie dealer to
see what all the hullaballoo is about. They are easily the most speaker for
$2000 on the market today. They are very good; better than anything I've
heard from Winey and company for a long time. Without the audio rags, I
wouldn't have known about them. Sometimes I read something that actually
teaches me something about which I was unfamiliar. A recent article about
tone-arm and cartridge geometry in one of the US rags comes to mind here.

>> They are mostly about entertaining one with articles
>> and features about subject matter likely to be of
> > interest to the reader.
>
> But more of interest to the advertiser, in that it's
> all about getting the reader's attention.

That's their agenda, not the reader's.

>> Occasionally they also inform, but mostly they just
>> entertain.
>
> But earlier you said, among other things:
>
> ""Hi Fi Choice" is a magazine that I admire more
> and more. They not only listen subjectively, but they
> also do measurement tests of equipment and they do DBTs
> as well. Like most hobbyist publications, the Brits
> just seem to do it better. Better writing, better
> content, more in-depth evaluations..."
>
> Now, while you certainly do not state so explicitly
> and, indeed, you do refer to "hobbyist publications",
> but I find it a bit odd that things like "measurements"
> and "DBTs" and "in-depth evaluations" and the like are
> concepts compatible with "mostly" entertainment. I am
> not entertained by measurements, nor am I entertained
> by DBTs. I use them to learn.

So, now I'm to be crucified for finding the learning process entertaining?

> I find myself clearly able to separate entertainment from
> information, but maybe that's just something quirky about
> me.

Or perhaps that I'm more entertained by a documentary on TV that teaches me
something I didn't know than by a mindless sitcom or reality TV show means
that there is something quirky about me.

[bob]

On Jun 22, 10:03=A0am, Scott > wrote:

>
> Of course it does otherwise you have no way of gauging the sensitivity
> of the test. Anull with no calibration has too many variables. was the
> test sensitive to audible differences?

The methodology has been used thousands of times in listening tests,
often with audible results.

> Were the subjects sensitive to
> audible differences? No way to know is there?

The subjects were human beings. Human beings are known to be sensitive
to audible differences. In addition, many of the subjects in that test
specifically claimed to be able to hear differences between the very
devices being tested. And they were able to hear differences in noise
floors in this very test.

bob

[Scott[_6_]]

On Jun 22, 8:19=A0pm, bob > wrote:
> On Jun 22, 10:03=A0am, Scott > wrote:
>
>
>
> > Of course it does otherwise you have no way of gauging the sensitivity
> > of the test. Anull with no calibration has too many variables. was the
> > test sensitive to audible differences?
>
> The methodology has been used thousands of times in listening tests,
> often with audible results.

A broad vague overview does not make any individual test sensitive.
people have crossed the street billions of times around the world over
the decades. doesn't mean you don't need to look both ways to avoid
being run over.

>
> > Were the subjects sensitive to
> > audible differences? No way to know is there?
>
> The subjects were human beings. Human beings are known to be sensitive
> to audible differences.

really? All of them? Nah.

> In addition, many of the subjects in that test.....

"That" test? This was an answer to a question posed about ABX DBTs in
general. No specific test was refferenced in *my* post.

[[email protected]]

On Jun 22, 3:03=A0pm, Scott > wrote:
> On Jun 18, 7:07=3DA0am, wrote:
> > My question is, is there a real need for calibration or is this just a
> > demand of audiophiles because the test came up with a negative result?


> Of course it does otherwise you have no way of gauging the sensitivity
> of the test. Anull with no calibration has too many variables. was the
> test sensitive to audible differences? Were the subjects sensitive to
> audible differences? No way to know is there?


Assuming that there was a need for calibration. The M&M test was about
the audibility of "bottlenecking" a hi-rez signal. What calibration
signal would one use other than a bottlenecked signal (one would have
to otherwise some audiophiles would claim that the calibration signal
was not adapted for its intended purpose), so for calibration one
would use the signal that is going to be tested.

You say "was the test sensitive to audible differences". Before the
test you don't know whether or not there are audible differences! Now
assume that the differences were audible, how did you determine that
they are, in heavily biased sighted listening tests, in "non-
calibrated" blind tests?

What is perhaps worth mentioning is that on the AES forum none of the
comments referred to this particular calibration issue.

What is further worth mentioning is that in all those (AES) papers
where blind tests have been conducted calibration was never ever
mentioned.


Klaus

[Scott[_6_]]

On Jun 23, 7:44=A0pm, wrote:
> On Jun 22, 3:03=3DA0pm, Scott > wrote:
>
> > On Jun 18, 7:07=3D3DA0am, wrote:
> > > My question is, is there a real need for calibration or is this just =
a
> > > demand of audiophiles because the test came up with a negative result=
?
> > Of course it does otherwise you have no way of gauging the sensitivity
> > of the test. Anull with no calibration has too many variables. was the
> > test sensitive to audible differences? Were the subjects sensitive to
> > audible differences? No way to know is there?
>
> Assuming that there was a need for calibration. The M&M test was about
> the audibility of "bottlenecking" a hi-rez signal. What calibration
> signal would one use other than a bottlenecked signal (one would have
> to otherwise some audiophiles would claim that the calibration signal
> was not adapted for its intended purpose), so for calibration one
> would use the signal that is going to be tested.

The same signals one would use to test any set up for sensitivity to
audible differences.


>
> You say "was the test sensitive to audible differences". Before the
> test you don't know whether or not there are audible differences!

True, nor do you "know" that any given test will reveal audible
differences should there be audible differences. Given the body of
knowledge on the thresholds of human hearing that aspect of any given
ABX DBT can be gauged before conducting any further ABX DBTs. How on
earth would it ever be anything but a good idea to do so?

> Now
> assume that the differences were audible, how did you determine that
> they are, in heavily biased sighted listening tests, in "non-
> calibrated" blind tests?


Why are you asking about sighted tests? The question was do we need to
"calibrate" blind tests. The answer remains yes regardless of any
discussion of sighted tests. I'm really not interested in red herring
arguments. How would anything about sighted tests affect the need or
lack of need to calibrate blind tests to eliminate the possibility of
lack of test sensitivity to audible differences in the case of a null
result?


>
> What is perhaps worth mentioning is that on the AES forum none of the
> comments referred to this particular calibration issue.
>
> What is further worth mentioning is that in all those (AES) papers
> where blind tests have been conducted calibration was never ever
> mentioned.


In the end the AES is just a group of people with it's own baggage.
Show me one published scientific researcher who would suggest checking
a DBT for sensitivity is anything other than a good idea. Show me one
scientist who will disagree with the assertion that without checking
any given ABX DBT test for sensitivity a solid null (not one that is
dancing on the threshold of a positive result) leaves us with three
basic possible explanations. 1. the listener was not sensitive to
audible differences. 2. audible differences were masked in that
particlular test. 3 There were no audible differences present.
Calibrate the test and 1 and 2 are profoundly reduced as possible
correct conclusions.

[Arny Krueger]

"Scott" > wrote in message


>> The subjects were human beings. Human beings are known
>> to be sensitive to audible differences.
>
> really? All of them? Nah.

Good point. Experience shows that many who profess to hear, and criticize
those who have failed to hear purported differences before them, also fail
to hear when appropriate controls are added.

[bob]

On Jun 23, 10:44=A0pm, wrote:
>
> What is further worth mentioning is that in all those (AES) papers
> where blind tests have been conducted calibration was never ever
> mentioned.

Of course not. "Calibration" is just blowing smoke. It doesn't even
have a fixed meaning here. Looks at the quotes in the OP. It seems to
mean something different every time a poster reaches for it.

The most important purpose of peer review is to confirm that the
methodology of a study is sound. The peer reviewers did that. Which is
why the only recourse left is to trash peer review and make fuzzy
appeals using terminology one has only a shaky grasp of.

bob

[Arny Krueger]

"Scott" > wrote in message

> On Jun 23, 7:44=A0pm, wrote:
>> On Jun 22, 3:03=3DA0pm, Scott > wrote:
>>
>>> On Jun 18, 7:07=3D3DA0am,
>>> wrote:
>>>> My question is, is there a real need for calibration
>>>> or is this just = a demand of audiophiles because the
>>>> test came up with a negative result=
> ?
>>> Of course it does otherwise you have no way of gauging
>>> the sensitivity of the test. Anull with no calibration
>>> has too many variables. was the test sensitive to
>>> audible differences? Were the subjects sensitive to
>>> audible differences? No way to know is there?
>>
>> Assuming that there was a need for calibration. The M&M
>> test was about the audibility of "bottlenecking" a
>> hi-rez signal. What calibration signal would one use
>> other than a bottlenecked signal (one would have to
>> otherwise some audiophiles would claim that the
>> calibration signal was not adapted for its intended
>> purpose), so for calibration one would use the signal
>> that is going to be tested.
>
> The same signals one would use to test any set up for
> sensitivity to audible differences.

There is no such single thing as a "one size fits all" test signal.

As rule, different audible differences are most obvious with different test
signals.

For example, a test for level mismatch might be best tested using one kind
of test signal, and a test for nonlinear distortion might be tested using a
different kind of test signal.

This is clearly the case for tests involving test equipment, and it is also
true for listening tests.

Even high end reviewers have been telling us for decades that certain
differences are best heard with certain recordings. Admittedly we have to
treat knowlege obtained from high end reviewers a bit advisedly due to their
proven track record for claiming the existence of audible differences that
in fact cannot be heard in bias-controlled listening tests. But in this case
I'm trying to give them the benefit of the doubt in this particular matter,
because reliable listening tests show that some music is better than others
for various kinds of listening tests.

>> You say "was the test sensitive to audible differences".
>> Before the test you don't know whether or not there are
>> audible differences!

> True, nor do you "know" that any given test will reveal
> audible differences should there be audible differences.

Agreed. You actually have to do some listening tests, or know the results of
other relevant listening tests in order to make good choices of test
signals. In some cases psychoacoustics or ven just common sense can give
clues about the characteristics of recordings that are more likely to give
sensitive results.

However, this case is a little different. My previous comments relate to
situations where audible differences are known to be reliably heard.

The results of previous relevant listening tests of this kind have already
reliably shown that there is *no such thing* as a normal musical recording
where diferences were reliably heard. It has also been found that there are
no known exceptional (in a good way) musical recordings where differences
were reliably heard.

This is like saying that there is no horse that is best for finding the Lost
Grail, because the Lost Grail has never been found. Once certain very large
efforts have been put into finding something, it is not reasonable to
criticize people who fail to find it.

[Audio Empire]

On Thu, 24 Jun 2010 06:41:31 -0700, Arny Krueger wrote
(in article >):

> "Scott" > wrote in message
>
>
>>> The subjects were human beings. Human beings are known
>>> to be sensitive to audible differences.
>>
>> really? All of them? Nah.
>
> Good point. Experience shows that many who profess to hear, and criticize
> those who have failed to hear purported differences before them, also fail
> to hear when appropriate controls are added.
>

But does that mean that they "can't" hear or that there are simply no
differences TO hear?

[Scott[_6_]]

On Jun 24, 9:38=A0am, "Arny Krueger" > wrote:
> "Scott" > wrote in message
>
> > On Jun 23, 7:44=3DA0pm, wrote:
> >> On Jun 22, 3:03=3D3DA0pm, Scott > wrote:
>
> >>> On Jun 18, 7:07=3D3D3DA0am,
> >>> wrote:
> >>>> My question is, is there a real need for calibration
> >>>> or is this just a demand of audiophiles because the
> >>>> test came up with a negative result?
>
> >>> Of course it does otherwise you have no way of gauging
> >>> the sensitivity of the test. Anull with no calibration
> >>> has too many variables. was the test sensitive to
> >>> audible differences? Were the subjects sensitive to
> >>> audible differences? No way to know is there?
>
> >> Assuming that there was a need for calibration. The M&M
> >> test was about the audibility of "bottlenecking" a
> >> hi-rez signal. What calibration signal would one use
> >> other than a bottlenecked signal (one would have to
> >> otherwise some audiophiles would claim that the
> >> calibration signal was not adapted for its intended
> >> purpose), so for calibration one would use the signal
> >> that is going to be tested.
>
> > The same signals one would use to test any set up for
> > sensitivity to audible differences.
>
> There is no such single thing as a "one size fits all" test signal.
>
Of course not Arny. Hence my use of the word signal*s* which is the
plural of "signal."

[Andrew Haley]

Audio Empire > wrote:
> On Thu, 24 Jun 2010 06:41:31 -0700, Arny Krueger wrote
> (in article >):
>
>> "Scott" > wrote in message
>>
>>
>>>> The subjects were human beings. Human beings are known
>>>> to be sensitive to audible differences.
>>>
>>> really? All of them? Nah.
>>
>> Good point. Experience shows that many who profess to hear, and
>> criticize those who have failed to hear purported differences
>> before them, also fail to hear when appropriate controls are added.
>
> But does that mean that they "can't" hear or that there are simply no
> differences TO hear?

The differences caused by passing through a 16-bit/44.1-kHz
???bottleneck" can be measured with sensitive equipment, so they are
there. The question is whether anyone's hearing is good enough to
detect them.

One thing, though: the result would have been much more interesting if
the test had progressively shortened the wordlength until the test
subjects could reliably hear a difference. The experiment would have
been more difficult and time-consuming, though.

Andrew.

[Arny Krueger]

"Audio Empire" > wrote in message

> On Thu, 24 Jun 2010 06:41:31 -0700, Arny Krueger wrote
> (in article >):

[ Excess quotation snipped. -- dsr ]

>> Good point. Experience shows that many who profess to
>> hear, and criticize those who have failed to hear
>> purported differences before them, also fail to hear
>> when appropriate controls are added.

> But does that mean that they "can't" hear or that there
> are simply no differences TO hear?

If there are no differences to hear, then whether or not a particular
listener can hear it should not even be a question. It's moot.

[Arny Krueger]

"Scott" > wrote in message

> On Jun 24, 9:38 am, "Arny Krueger" >
> wrote:
>> "Scott" > wrote in message
>>


[ Excess quotation snipped. -- dsr ]


>> There is no such single thing as a "one size fits all"
>> test signal.

> Of course not Arny. Hence my use of the word signal*s*
> which is the plural of "signal."

The use of a plural was logically cancelled out by the word "same" that
preceeded it. IOW, one would not use the same signals but rather different
signals, depending on exactly which different audible difference one was
seeking to hear.

The the whole concept of the sentence was cancelled out by the fact that
there is no known audible difference in the situation that was being
discussed.

If a difference, such as excessively high sample rates, is known to be
generally not audible, then there is no such thing as a signal that "...one
would use to test any set up for sensitivity to audible differences." No
known reliably audible difference was being tested for.

IOW, we have the obvious fallacy: What test signal does one use to listen
for inaudible differences?

;-)

[Audio Empire]

On Fri, 25 Jun 2010 07:54:03 -0700, Arny Krueger wrote
(in article >):

> "Audio Empire" > wrote in message
>
>> On Thu, 24 Jun 2010 06:41:31 -0700, Arny Krueger wrote
>> (in article >):
>
> [ Excess quotation snipped. -- dsr ]
>
>>> Good point. Experience shows that many who profess to
>>> hear, and criticize those who have failed to hear
>>> purported differences before them, also fail to hear
>>> when appropriate controls are added.
>
>> But does that mean that they "can't" hear or that there
>> are simply no differences TO hear?
>
> If there are no differences to hear, then whether or not a particular
> listener can hear it should not even be a question. It's moot.

I propose that even if there were differences (like between speakers or phono
cartridges or amplifiers) and the person listening for those differences had
a "tin ear" when it comes to listening to reproduced music, that person
likely wouldn't hear them. Then it comes down to that old Frederick Nietsche
connundrum first postulated in his "Man and Superman": "If a tree falls in
the forest and there is no one there to hear it fall, did it make a sound?"

[Steven Sullivan]

Scott > wrote:
> On Jun 18, 7:07=A0am, wrote:
> > In an older thread about the Meyer/Moran study relating to the
> > audibility of "bottlenecking" the output of SACD through the A/D/A
> > section of a CD-recorder, the need of calibration was mentioned:
> >
> > "Your results weren't calibrated, because you didn't come up with any
> > measurement of the differences that your experimental setup could
> > actually detect with your chosen source material and test subjects.
> > This could have been done by generating test material with known
> > degradations and determining which of these were audible. In other
> > words, by generating positive differences that could be detected, not
> > negative differences which could not be detected."
> >
> > The same author in a current thread said:
> >
> > "The real flaw is that there was no calibration of the source
> > material, equipment, room, etc. as to resolution and suitability to
> > detect the effects being evaluated".
> >
> > My question is, is there a real need for calibration or is this just a
> > demand of audiophiles because the test came up with a negative result?
> >
> > Klaus

> Of course it does otherwise you have no way of gauging the sensitivity
> of the test. Anull with no calibration has too many variables. was the
> test sensitive to audible differences? Were the subjects sensitive to
> audible differences? No way to know is there?

The M&M test setup was sensitive enough to reveal a flaw in one of the
players (which was then replaced); it was also sensitive enough to
reveal the different noise floors during 'silence', between DSD and
DSD-->Redbook, when the volume was raised ~15dB.

--
-S
We have it in our power to begin the world over again - Thomas Paine

[Scott[_6_]]

On Jun 24, 12:35=A0pm, Audio Empire > wrote:

>
> But does that mean that they "can't" hear or that there are simply no
> differences TO hear?

Without testing for sensitivity the answer to your question is yes.
It means that they either "can't" hear or there are simply differences
to hear or they simply can't discriminate those under that particular
test. It is easy enough to screw up such a test. Just continue to test
on ABX with differences that are near the threshold of audibility way
beyond the threshold of listener fatigue and you will likely get a
false negative. That would be one of many ways one can take a simple
ABX DBT and make it insensitive to actual audible differences. There
are a lot of ways to get bad data with any given ABX DBT. Why so many
folks want to ignore or deny this simple fact is beyond me. testing
any ABX DBT for sensitivity along with following a number of other
protocols to prevent bad data only makes for better tests. Not sure
why some folks here dismiss such care as unneeded or seem to be flat
out against such due care. this is the sort of rigore that sperates
real scientific research and amatuer weekend "science" doen by
hobbyists in their garages and basements. It's fine if one wants to
have fun playing scientist but it aint real science. I suppose if one
wants to assert that the "research of the weekend warrior as the equal
to real scientific research it would make sense to dismiss the rigor
of things like test calibration.

[Scott[_6_]]

On Jun 25, 10:15=A0am, Dick Pierce > wrote:
> Scott wrote:
> >>>The same signals one would use to test any set up for
> >>>sensitivity to audible differences.
>
> >>There is no such single thing as a "one size fits all" test signal.
>
> > Of course not Arny. Hence my use of the word signal*s* which is the
> > plural of "signal."
>
> And the use of the word "same" means what?

It means "same." Why are you and Arny having such difficulty with
this simple idea? I'll spell it out. The same test signals (plural)
that would be used to test for sensitivty for any ABX DBT could have
been used for the specific test the OP asked about. That was his
question, what signal would be used. The OP used the word in the
singular form but I changed it to plural since I knew it wouldn't be
one single signal. Hope that clears things up.

[Arny Krueger]

"Scott" > wrote in message

> On Jun 24, 12:35 pm, Audio Empire
> > wrote:

>> But does that mean that they "can't" hear or that there
>> are simply no differences TO hear?

> Without testing for sensitivity the answer to your
> question is yes.

The outcome of any test you run with a negative result, can be interpreted
as follows:

> It means that they either "can't" hear
> or there are simply differences to hear or they simply
> can't discriminate those under that particular test.

That includes any so-called sensitivity tests.

Let's review the current situation. There are no extant tests of the kind
that we are discussing that have had positive outcomes. Thousands of such
tests have been attempted in a wide variety of circumstances.

> It is easy enough to screw up such a test.

This dodges addressing the current situation where thousands of listening
tests have been run to show an audible difference due to excess sample rates
with no known positive outcomes when appropriate experimental controls were
in place.

> Just continue to
> test on ABX with differences that are near the threshold
> of audibility way beyond the threshold of listener
> fatigue and you will likely get a false negative.

Since you specifically mention ABX, are you saying that there is no such
thing as listener fatique in sighted evaluations? Or are you saying that
some other methodology, such as ABC/hr is not at least equally fatiguing?
Are you asserting that all of the thousands of failed tests were all due to
listener fatique? Or, are just just dragging out an old, tired red herring?

[Scott[_6_]]

On Jun 26, 10:50=A0am, ScottW > wrote:
> On Jun 24, 6:35=A0am, Scott > wrote:
>
>
>
>
>
> > On Jun 23, 7:44=3DA0pm, wrote:
>
> > > On Jun 22, 3:03=3D3DA0pm, Scott > wrote:
>
> > > > On Jun 18, 7:07=3D3D3DA0am, wrote:
> > > > > My question is, is there a real need for calibration or is this j=
ust =3D
> > a
> > > > > demand of audiophiles because the test came up with a negative re=
sult=3D
> > ?
> > > > Of course it does otherwise you have no way of gauging the sensitiv=
ity
> > > > of the test. Anull with no calibration has too many variables. was =
the
> > > > test sensitive to audible differences? Were the subjects sensitive =
to
> > > > audible differences? No way to know is there?
>
> > > Assuming that there was a need for calibration. The M&M test was abou=
t
> > > the audibility of "bottlenecking" a hi-rez signal. What calibration
> > > signal would one use other than a bottlenecked signal (one would have
> > > to otherwise some audiophiles would claim that the calibration signal
> > > was not adapted for its intended purpose), so for calibration one
> > > would use the signal that is going to be tested.
>
> > The same signals one would use to test any set up for sensitivity to
> > audible differences.
>
> =A0Interesting proposal. =A0Could you define the categories of audible
> differences?

I could try but really I think this would be a good question for JJ
who has done extensive tests for various thresholds of human hearing.


>
> Let's just take a relatively simple one like Frequency Response and
> examine it in with just bit of speculative detail.
>
> =A0I suppose single tone amplitude would be obvious with humans having
> different sensitivity to amplitude difference at different
> frequencies.
> That sensitivity to amplitude difference also changes with amplitude.
> You can't hear difference in amplitude between two signals both of
> which you can't hear, nor do I suppose you can hear the difference
> between two signals of different amplitude when both of them are
> sufficient to make your ears bleed.
> Add a second single frequency fixed amplitude masking tone. Measure
> sensitivity to variable amplitude tones. That will vary with freqency
> of masking tone, amplitude of masking, frequency of the tone we're
> measuring sensitivity to amplitude changes of, and amplitude of the
> tone we're meausring sensitivity to amplitude changes of
> and probably of few other interactions I've failed to mention.
>
> Hmm...try to matrix that. =A0How many signal conditions have we
> constructed?
> Let's see....1hz freq resolution, 0.5 db amplitude resolution from
> threshold to pain..
> times all the interactions...
> Well, I'm going out on a limb to guess this signal matrix will be
> quite large.
> Now, How do these sensitivity tests with test signals correlate to
> real music signals used in ABX tests?

All very fair questions. Again I would defer to JJ who routinely did
such tests throughout his career and did so in many cases in
examination of human thresholds of hearing.


>
> Wait a second, we're now back to the beginning of trying to determine
> humans ability to determine audible differences with different music
> signals on a system.
>
> Seems like we'd need to know that to begin to calibrate with "test
> signals".
>

Such is the value of a large body of data.

[Scott[_6_]]

On Jun 26, 11:24=A0am, "Arny Krueger" > wrote:
> "Scott" > wrote in message
>
>
>
> > On Jun 24, 12:35 pm, Audio Empire
> > > wrote:
> >> But does that mean that they "can't" hear or that there
> >> are simply no differences TO hear?
> > Without testing for sensitivity the answer to your
> > question is yes.
>
> The outcome of any test you run with a negative result, can be interprete=
d
> as follows:
>
> > It means that they either "can't" hear
> > or there are simply differences to hear or they simply
> > can't discriminate those under that particular test.
>
> That includes any so-called sensitivity tests.

Sensitivity tests without positive results would either indicate
incomplete sensitivity tests or a complete lack of sensitivity for
audible differences.


>
> Let's review the current situation.

There is no "current situation" to review in regards to my assertions
about ABX DBTs needing to be calibrated for sensititvity. It is an
assertion about ABX DBTs in general.

>
> > It is easy enough to screw up such a test.
>
> This dodges addressing the current situation where thousands of listening
> tests have been run to show an audible difference due to excess sample ra=
tes
> with no known positive outcomes when appropriate experimental controls we=
re
> in place.


No, it doen't dodge anything. It is a basic truism about ABX DBTs and
does not make any reference to any specific tests.


>
> > Just continue to
> > test on ABX with differences that are near the threshold
> > of audibility way beyond the threshold of listener
> > fatigue and you will likely get a false negative.
>
> Since you specifically mention ABX, are you saying that there is no such
> thing as listener fatique in sighted evaluations?


No


> Or are you saying that
> some other methodology, such as ABC/hr is not at least equally fatiguing?

No.


> Are you asserting that all of the thousands of failed tests were all due =
to
> listener fatique?


No.


> Or, are just just dragging out an old, tired red herring?


What red herring Arny? My assertion is about ABX DBTs in general. The
only red herrings I see are the ones you just tried to drag out.

[KH]

On 6/26/2010 8:45 PM, Scott wrote:
> On Jun 26, 11:24=A0am, "Arny > wrote:
>> > wrote in message
>>
>>
>>
<snip>

>>> It means that they either "can't" hear
>>> or there are simply differences to hear or they simply
>>> can't discriminate those under that particular test.
>>
>> That includes any so-called sensitivity tests.
>
> Sensitivity tests without positive results would either indicate
> incomplete sensitivity tests or a complete lack of sensitivity for
> audible differences.

Sensitivity to what precisely? You cannot verify sensitivity to unknown
parameters. You can't generate test signals that effectively mimic
uncharacterized differences between presentations. Look at it another
way, if you could characterize the differences, and generate a
representative test signal with which to "calibrate" the listeners, then
the actual ABX test would be moot relative to the specific difference in
question - the answer would be known based on the sensitivity test. And
*that* would tell you precisely nothing, as none of the masking effects
present an actual musical ABX test would be present. If they are
present, then the "sensitivity" test and the ABX test are identical, and
you're back to square one.

And let's not give short shrift to the "not everything can be measured"
crowd; that position precludes even the possibility of a sensitivity
test, as you cannot ever generate a test signal, nor can you quantify
any results related to the test.

>> Let's review the current situation.
>
> There is no "current situation" to review in regards to my assertions
> about ABX DBTs needing to be calibrated for sensititvity. It is an
> assertion about ABX DBTs in general.
>
>>
>>> It is easy enough to screw up such a test.
>>
>> This dodges addressing the current situation where thousands of listening
>> tests have been run to show an audible difference due to excess sample ra=
> tes
>> with no known positive outcomes when appropriate experimental controls we=
> re
>> in place.
>
>
> No, it doen't dodge anything. It is a basic truism about ABX DBTs and
> does not make any reference to any specific tests.

No, it's not true, let alone a truism, about ABX DBT's. ABX has been
shown many times as capable of detecting audible differences when they
exist. It's been used many times to determine that excessive sample
rates don't result in audible differences, using the same methodology
and bias controls. That test base testifies to the precision of the
method which is the only type of "calibration" that is relevant to this
type of *difference discernment* test.

And once again, one would need to show how your supposed "calibration
deficit" would in fact have relevance, were it to exist, in the context
of actually applying ABX methods. For example, in situations like Arny
referred to above, where the only countervailing "evidence" is gathered
via listening tests that have the same "problems" (be they calibration,
fatigue, whatever) as is claimed for ABX, *plus* additional uncontrolled
error sources as well, an ABX test confirming the results expected,
based upon engineering and psychoacoustic knowledge, have no requirement
for accuracy (which would require some calibration activity), as they
are not *quantifying* anything. They require only establishment of
precision, and that's provided by a database of many test/subject
replicates.

>>> Just continue to
>>> test on ABX with differences that are near the threshold
>>> of audibility way beyond the threshold of listener
>>> fatigue and you will likely get a false negative.

And so? Put earmuffs on the subjects and you'll get the same false
negative. There are endless ways to screw up a test, how is that
relevant to the discussion here? Failure to "calibrate" a method that
does not seek to quantify anything is *not* one of those failure modes.

Keith Hughes

[Scott[_6_]]

On Jun 27, 10:41=A0am, KH > wrote:
> On 6/26/2010 8:45 PM, Scott wrote:> On Jun 26, 11:24=3DA0am, "Arny Kruege=
> =A0wrote:
> >> > =A0wrote in message
>
>
>
> <snip>
>
> >>> It means that they either "can't" hear
> >>> or there are simply differences to hear or they simply
> >>> can't discriminate those under that particular test.
>
> >> That includes any so-called sensitivity tests.
>
> > Sensitivity tests without positive results would either indicate
> > incomplete sensitivity tests or a complete lack of sensitivity for
> > audible differences.
>
> Sensitivity to what precisely?


Actual audible differences.

>=A0You cannot verify sensitivity to unknown
> parameters.


What unknown parameters are you talking about? We got a pretty good
idea what the parameters are of the thresholds of human hearing.

>=A0You can't generate test signals that effectively mimic
> uncharacterized differences between presentations.


How are known audible differences uncharacterized?

> =A0Look at it another
> way, if you could characterize the differences, and generate a
> representative test signal with which to "calibrate" the listeners, then
> the actual ABX test would be moot relative to the specific difference in
> question - the answer would be known based on the sensitivity test.

That would be true if the claim under test were already known to be
audibly different. think about it.


> =A0And
> *that* would tell you precisely nothing, as none of the masking effects
> present an actual musical ABX test would be present. If they are
> present, then the "sensitivity" test and the ABX test are identical, and
> you're back to square one.

No. The question is can a given ABX test setup reveal actual audible
differences. The answer is yes once you show it can do so. Until then
the answer is maybe.



>
> And let's not give short shrift to the "not everything can be measured"
> crowd; that position precludes even the possibility of a sensitivity
> test, as you cannot ever generate a test signal, nor can you quantify
> any results related to the test.


Why on earth would we want to address that crowd? How would that help
make ABX DBTs better?
>
>
>
>
>
> >> Let's review the current situation.
>
> > There is no "current situation" to review in regards to my assertions
> > about ABX DBTs needing to be calibrated for sensititvity. It is an
> > assertion about ABX DBTs in general.
>
> >>> It is easy enough to screw up such a test.
>
> >> This dodges addressing the current situation where thousands of listen=
ing
> >> tests have been run to show an audible difference due to excess sample=
ra=3D
> > tes
> >> with no known positive outcomes when appropriate experimental controls=
we=3D
> > re
> >> in place.
>
> > No, it doen't dodge anything. It is a basic truism about ABX DBTs and
> > does not make any reference to any specific tests.
>
> No, it's not true, let alone a truism, about ABX DBT's. =A0ABX has been
> shown many times as capable of detecting audible differences when they
> exist.

Really? How have ABX tests that never wrought any positives been shown
to be capable of detecting audible differences without some sort of
check for test sensitivity?


> =A0It's been used many times to determine that excessive sample
> rates don't result in audible differences, using the same methodology
> and bias controls. =A0That test base testifies to the precision of the
> method which is the only type of "calibration" that is relevant to this
> type of *difference discernment* test.


In the middle ages putting a suspected heretic's hand in boiling water
was also used many times and the results were every bit as consistant.
Doesn't mean it actually worked.


>
> And once again, one would need to show how your supposed "calibration
> deficit" would in fact have relevance, were it to exist, in the context
> of actually applying ABX methods.


Can't say I am buying that. The burden of rigor is on those who do
such tests. No one needs to show these folks the need for due rigor.

> =A0For example, in situations like Arny
> referred to above, where the only countervailing "evidence" is gathered
> via listening tests that have the same "problems" (be they calibration,
> fatigue, whatever) as is claimed for ABX, *plus* additional uncontrolled
> error sources as well, an ABX test confirming the results expected,
> based upon engineering and psychoacoustic knowledge, have no requirement
> for accuracy (which would require some calibration activity), as they
> are not *quantifying* anything. =A0They require only establishment of
> precision, and that's provided by a database of many test/subject
> replicates.


Not buying that. Consistant results is not proof per se that results
are accurate.



>
> >>> Just continue to
> >>> test on ABX with differences that are near the threshold
> >>> of audibility way beyond the threshold of listener
> >>> fatigue and you will likely get a false negative.
>
> And so? Put earmuffs on the subjects and you'll get the same false
> negative. =A0There are endless ways to screw up a test, how is that
> relevant to the discussion here? =A0Failure to "calibrate" a method that
> does not seek to quantify anything is *not* one of those failure modes.
>

Unfortunately you snipped the context of my assertion which simply was
there were many ways to get a false negative and that the causes of
false negatives and false positives tended to be different. It was not
claimed that listener fatigue had any direct corolation to testing for
test sensitivity. It's just another one particular way to desentize a
test. If a test lacks due sensitivity it may wrought false negative
results. There are indeed "endless ways to screw up a test" and
checking for sensitivity can actually prevent any number of those
endless ways from creeping in and corrupting a given test. Perhaps you
don't think such a precaution is a good idea. i think it is.

[[email protected]]

My question was: What calibration signal would one use other than a
bottlenecked signal?


On Jun 24, 2:35 pm, Scott > answered:

> The same signals one would use to test any set up for sensitivity to
> audible differences.

> True, nor do you "know" that any given test will reveal audible
> differences should there be audible differences. Given the body of
> knowledge on the thresholds of human hearing that aspect of any given
> ABX DBT can be gauged before conducting any further ABX DBTs. How on
> earth would it ever be anything but a good idea to do so?


Are you saying that one could use any known audible difference to see
whether the test is capable of judging the audibility of
bottlenecking? So I could use, say, audible phase shift, IM
distortion, or group delay? If ABX yields positive results you have
shown that ABX is sensitive to phase shift, IM distortion, or group
delay, but nothing more. It won=92t tell you that it will be sensitive
to bottlenecking.

Further, thresholds of human hearing are one thing, differences
between CD-players, amplifiers or, in this case, different formats,
another. Either you hear a difference or you don=92t, there is no such
as thing as threshold of perception of that difference.

Therefore, you cannot use any arbitrary signal to calibrate the
bottleneck test. You have to use a bottlenecked signal, so the
calibration step would constitute the very test you want to calibrate.

> In the end the AES is just a group of people with it's own baggage.
> Show me one published scientific researcher who would suggest checking
> a DBT for sensitivity is anything other than a good idea.

You want names, here are some:

Ted Grusec (Canadian department of communication)
Soren Bech (Bang & Olufsen)
W. H. Schmidt (University for technology and economy, Berlin)
Kaoru Watanabe (NHK)
Stan Lip****z (Waterloo University)
Kaoru Ashihara (National Institute of Advanced Industrial Science and
Technology)
W.A. Munson ((Bell Labs)
W.A. Rosenblith (MIT)
J. Hillenbrand (Dep. of speech pathology and audiology, Western
Michigan University)
D. Kewley-Port (Speech research lab., Indiana University)
Ian B. Thomas (Speech communications lab., U. of Massachusetts)

Not only AES, also JASA peer reviewers seem to have no objections
against non-calibrated blind tests.

Now it=92s your turn: name published scientists who do think that blind
tests need being calibrated, after all, you made the claim that they
do!

Klaus

[Arny Krueger]

"Scott" > wrote in message
...

> No. The question is can a given ABX test setup reveal actual audible
> differences. The answer is yes once you show it can do so. Until then
> the answer is maybe.

Yet another example of anti-ABX bias by raising a general question about
listening tests in such a way that it appears that only ABX tests are
affected.

Here's a corrected version:

The question is can a given listening test setup reveal actual audible
differences. The answer is yes once you show it can do so. Until then
the answer is maybe.

Given their built-in bias towards false positive results, most audiophile
listening evaluations cannot be properly called tests. They are useless for
determining actual audible differences unless the differences are relatively
gross.

[[email protected]]

Klaus mentioned:

> Further, thresholds of human hearing are one thing, differences between
>CD-players, amplifiers or, in this case, different formats, another.
> E ither you hear a difference or you don=92t, there is no such as thing
> as
> threshold of perception of that difference.
>
> Therefore, you cannot use any arbitrary signal to calibrate the
> bottleneck test. You have to use a bottlenecked signal, so the
> calibration step would constitute the very test you want to calibrate.

This is all rhetorical tap dancing. In a listening alone context
thresholds of hearing perceptions of difference is all that counts. All
difference in a hi fi bit of gear is electrical as to signal differences.
All signal differences have by experiment a threshold by which difference
can be percieved.

Distortion will serve. Any two bits of gear can be made to sound
different if one causes one to produce enough difference in distortion
then the other. As one lowers that difference a threshold is reached
beyond which no difference can be percieved but by measurement it still
exists.

If one claims that cd player x but not y sounds "sweeter" then one is
making reference to electrical properties one thinks result in that
difference. Controlled listening alone testing will soon resolve this
perception claim.

If no difference beyond guessing can be identified then the difference
lies in ones brain's subjective perception producing process not the
electrical threshold potential of the gear. The perception
toggle"sweeter"
on and off as one knows or not which bit of gear is active confirms all.

As for source of signal, it is irrelevant. If electrical differences are
thought to produce a "sweeter" perception then choose a signal source one
claims to have experienced it while listening. The initial claim is the
"calibration" by listening alone, as will be the follow up listening alone
test of perception difference.

[[email protected]]

On Jun 29, 3:44 pm, wrote:
> Klaus mentioned:
>
> > Further, thresholds of human hearing are one thing, differences between
> >CD-players, amplifiers or, in this case, different formats, another.
> > E ither you hear a difference or you don=92t, there is no such as thing
> > as
> > threshold of perception of that difference.
>
> > Therefore, you cannot use any arbitrary signal to calibrate the
> > bottleneck test. You have to use a bottlenecked signal, so the
> > calibration step would constitute the very test you want to calibrate.
>
> This is all rhetorical tap dancing. In a listening alone context
> thresholds of hearing perceptions of difference is all that counts.


If you take stuff like distortion, you start with zero % and increase
the amount until you perceive it: threshold found. In the Meyer/Moran
study I referred to, they converted the SACD signal to 16 bit/44.1 kHz
and looked whether or not one could hear the difference. Please care
to explain how in this particular case a threshold of perception can
exist? The amount of which parameter do you increase until you find
this threshold?

The same is valid when you compare two pieces of gear: the amount of
which parameter do you increase to find a threshold?


Klaus

[[email protected]]

Observed:

>> This is all rhetorical tap dancing. In a listening alone context
>> thresholds of hearing perceptions of difference is all that counts.
>

Klaus responded:

>If you take stuff like distortion, you start with zero % and increase
>the amount until you perceive it: threshold found. In the Meyer/Moran
>study I referred to, they converted the SACD signal to 16 bit/44.1 kHz
>and looked whether or not one could hear the difference. Please care
>to explain how in this particular case a threshold of perception can
>exist? The amount of which parameter do you increase until you find
>this threshold?
>
>The same is valid when you compare two pieces of gear: the amount of
>which parameter do you increase to find a threshold?

You are focused on the wrong question. In psychoacoustics we might want
to know the source of what difference produces what threshold. Or in gear
design one might be interested to know what signal source produces what
threshold so as to make the gear produce as desired.

In listening alone testing frankly we don't care and dwelling on your
question is only a diversion.

If cd player x is said to be "sweeter" then y, we are to test the claimed
difference not to discover what electrical source might have produced it.
In fact if when listening alone testing establishes that no difference can
be reliably spotted, any effort to answer your question is moot.

Such a test result only adds to the body of research showing such
percieved differences do not exist in electtrical difference but are
generated in the brain post reception of the signal at the ears. That
percieved difference responding to non acoustical/electrical perception
stimuli when the gear being tested is known.

[Arny Krueger]

> wrote in message
...
> On Jun 29, 3:44 pm, wrote:
>> Klaus mentioned:
>>
>> > Further, thresholds of human hearing are one thing, differences between
>> >CD-players, amplifiers or, in this case, different formats, another.
>> > E ither you hear a difference or you don=92t, there is no such as thing
>> > as
>> > threshold of perception of that difference.
>>
>> > Therefore, you cannot use any arbitrary signal to calibrate the
>> > bottleneck test. You have to use a bottlenecked signal, so the
>> > calibration step would constitute the very test you want to calibrate.
>>
>> This is all rhetorical tap dancing. In a listening alone context
>> thresholds of hearing perceptions of difference is all that counts.
>
>
> If you take stuff like distortion, you start with zero % and increase
> the amount until you perceive it: threshold found. In the Meyer/Moran
> study I referred to, they converted the SACD signal to 16 bit/44.1 kHz
> and looked whether or not one could hear the difference. Please care
> to explain how in this particular case a threshold of perception can
> exist? The amount of which parameter do you increase until you find
> this threshold?

I've done extensive tests with other musical program material that was
recorded at 24/96 under near-lab conditions in the interest of high actual
recorded dynamic range and extended bandwidth. For example, some of my
recordings had about 90 dB dynamic range, as compared to an exceptionally
good commercial recording's dynamic range which is in the area of 75 dB.

I've then reduced this signal's dynamic range and bandwidth progressively
until I could find even one listener (of many tried) who could hear a
difference. I removed one bit of resolution at a time by rudely stripping
off bits without dither. I removed bandwidth by means of downsampling
using brick wall filtering.

I found that resolution reduction became reliably audible with 14 bits
resolution, and that bandwidth reduction became reliably audible with 16 KHz
bandwidth. Resolution reduction to 15 bits and 19 KHz bandwidth reduction
was undetectible.

> The same is valid when you compare two pieces of gear: the amount of
> which parameter do you increase to find a threshold?

One can increase the noise, response errors and distortion of audio gear
progressivly and in ways that simply extend the inaccuracies of the
particular piece of gear, by passing audio signals through that gear again
and a again. I found that for example passing demanding musical signals
(see above) through a power amplifier can become audible (due to small HF
frequency response losses) after about 5 passes. Other types of equipment,
such as the converters that I used in all of these experiements could pass a
signal in excess of 20 times without an reliably detectable audible change.

[Scott[_6_]]

On Jun 29, 4:46=A0am, wrote:


> You want names, here are some:
>
> Ted Grusec (Canadian department of communication)
> Soren Bech (Bang & Olufsen)
> W. H. Schmidt (University for technology and economy, Berlin)
> Kaoru Watanabe (NHK)
> Stan Lip****z (Waterloo University)
> Kaoru Ashihara (National Institute of Advanced Industrial Science and
> Technology)
> W.A. Munson ((Bell Labs)
> W.A. Rosenblith (MIT)
> J. Hillenbrand (Dep. of speech pathology and audiology, Western
> Michigan University)
> D. Kewley-Port (Speech research lab., Indiana University)
> Ian B. Thomas (Speech communications lab., U. of Massachusetts)
>

That certainly *is* a list of names.....

> Not only AES, also JASA peer reviewers seem to have no objections
> against non-calibrated blind tests.

"seem?"


>
> Now it=3D92s your turn: name published scientists who do think that blind
> tests need being calibrated, after all, you made the claim that they
> do!


Since you are not limiting it to audio this could turn into one of
those lists like the long list of scientists who believe in evolution
named Steve. You will find a lot of discussion on calibration in
medical DBTs.

[Scott[_6_]]

On Jun 29, 7:44=A0am, "Arny Krueger" > wrote:
> "Scott" > wrote in message
>
> ...
>
> > No. The question is can a given ABX test setup reveal actual audible
> > differences. The answer is yes once you show it can do so. Until then
> > the answer is maybe.
>
> Yet another example of anti-ABX bias by raising a general question about
> listening tests in such a way that it appears that only ABX tests are
> affected.

How is that an example of anti ABX? I could just as easily say its an
example of pro ABX. Where is the bias?



> Here's a corrected version:
>
> The question is can a given listening test setup reveal actual audible
> differences. The answer is yes once you show it can do so. Until then
> the answer is maybe.

That is just a different version not a corrected version. The question
works just as well for listening tests as a general catagory or ABX as
a specific subset.

[Arny Krueger]

"Scott" > wrote in message
...
> On Jun 29, 7:44=A0am, "Arny Krueger" > wrote:
>> "Scott" > wrote in message
>>
>> ...
>>
>> > No. The question is can a given ABX test setup reveal actual audible
>> > differences. The answer is yes once you show it can do so. Until then
>> > the answer is maybe.

>> Yet another example of anti-ABX bias by raising a general question about
>> listening tests in such a way that it appears that only ABX tests are
>> affected.

> How is that an example of anti ABX?

It makes it look like only ABX tests have any problems.

> I could just as easily say its an example of pro ABX.

Thats a problem - you say things like this so easily and glibly.

> Where is the bias?

The fact that it makes a general problem look like it only applies to ABX.

>> Here's a corrected version:

>> The question is can a given listening test setup reveal actual audible
>> differences. The answer is yes once you show it can do so. Until then
>> the answer is maybe.

> That is just a different version not a corrected version.

No, it corrects the obvious writer's misapprehension that only ABX tests
have the problem that is being pointed out.

> The question works just as well for listening tests as a general catagory
> or ABX as
> a specific subset.

That's exactly right, so explain why you made a point of picking on ABX
tests, and didn't phrase it so it applied to listening tests in general?

[[email protected]]

Scott observed:

>named Steve. You will find a lot of discussion on calibration in
>medical DBTs.

In a typical medical test two or more groups are given different medical
treatments and the results measured against a calibrated measure of some
body function or presence of some substance.

In listening alone tests this is meaningless except for calibration of
both arms of the test gear lineup. Also of statistical methods used
having validity for the test used.


We want to know if the difference statement "cd player x sounds sweeter
then y" has its source in the signal reaching the ears or after it in the
brain.

In which case if the reproduction gear has been calibrated as to making
all things but the bit of gear being tested as equal as possible, nothing
more is required in terms of calibration there.

Results are measured against odds of spotting differences being at the
guessing level alone. Calibration then is for proper statistical testing
methods to have been employed.

[Scott[_6_]]

On Jul 1, 9:59=A0am, "Arny Krueger" > wrote:
> "Scott" > wrote in message
>
> ...
>
> > On Jun 29, 7:44=3DA0am, "Arny Krueger" > wrote:
> >> "Scott" > wrote in message
>
> ...
>
> >> > No. The question is can a given ABX test setup reveal actual audible
> >> > differences. The answer is yes once you show it can do so. Until the=
n
> >> > the answer is maybe.
> >> Yet another example of anti-ABX bias by raising a general question abo=
ut
> >> listening tests in such a way that it appears that only ABX tests are
> >> affected.
> > How is that an example of anti ABX?
>
> It makes it look like only ABX tests have any problems.

How?


>
> > I could just as easily say its an example of pro ABX.
>
> Thats a problem - you say things like this so easily and glibly.


How is that a problem? These percpetions of anti ABX are really in
your head not in my posts.

>
> > Where is the bias?
>
> The fact that it makes a general problem look like it only applies to ABX=

[Scott[_6_]]

On Jul 1, 11:34=A0am, wrote:
> Scott observed:
>
> >named Steve. You will find a lot of discussion on calibration in
> >medical DBTs.
>
> In a typical medical test two or more groups are given different medical
> treatments and the results measured against a calibrated measure of some
> body function or presence of some substance.
>

There is a lot more calibration going on than that.


> In listening alone tests this is meaningless except for calibration of
> both arms of the test gear lineup. =A0Also of statistical methods used
> having validity for the test used.
>
> We want to know if the difference statement "cd player x sounds sweeter
> then y" has its source in the signal reaching the ears or after it in the
> brain.
>
> In which case if the reproduction gear has been calibrated as to making
> all things but the bit of gear being tested as equal as possible, nothing
> more is required in terms of calibration there.
>
> Results are measured against odds of spotting differences being at the
> guessing level alone. =A0Calibration then is for proper statistical testi=
ng
> methods to have been employed.

Let me demonstrate how this is an issue by using an extreme. let's say
the test had the wires screwed up and the ABX test was in reality
wired as an AAX test and B was accidentally cut out of the loop. How
sensitive would this test be to audible differences between A and B?
How would you know if you did a test under these circumstances and a
got a null and never checked up on the setup?

It's about testing the test. Or at least checking to make sure
everything is working as it should be. How this is a bad idea still is
beyond me.

[Arny Krueger]

"Scott" > wrote in message


> Let me demonstrate how this is an issue by using an
> extreme. let's say the test had the wires screwed up and
> the ABX test was in reality wired as an AAX test and B
> was accidentally cut out of the loop. How sensitive would
> this test be to audible differences between A and B? How
> would you know if you did a test under these
> circumstances and a got a null and never checked up on
> the setup?

You need to come up with an example that ABX testing doesn't make
impossible. Level matching prevents miswiring an ABX test because you need
to have proper identification and wiring of A and B to get through the level
matching step.

> It's about testing the test.

The test tests itself during the setup phase.

[bob]

On Jul 1, 6:05=A0pm, Scott > wrote:

> Let me demonstrate how this is an issue by using an extreme. let's say
> the test had the wires screwed up and the ABX test was in reality
> wired as an AAX test and B was accidentally cut out of the loop. How
> sensitive would this test be to audible differences between A and B?
> How would you know if you did a test under these circumstances and a
> got a null and never checked up on the setup?

So it's come to this. The thread began with a couple of quotes
challenging the validity of a DBT published in a peer-reviewed
journal, on the basis that they had not "calibrated" their test.

And here we are, 60-odd posts later, learning that "calibration"
means, make sure the equipment is wired correctly.

I think I said before that the demand for "calibration" was just
handwaving by people with no better case to make. This post seems to
vindicate that assessment.

bob

[John Corbett]

Arny Krueger wrote:

> I've done extensive tests with other musical program material that was
> recorded at 24/96 under near-lab conditions in the interest of high actual
> recorded dynamic range and extended bandwidth. For example, some of my
> recordings had about 90 dB dynamic range, as compared to an exceptionally
> good commercial recording's dynamic range which is in the area of 75 dB.
>
> I've then reduced this signal's dynamic range and bandwidth progressively
> until I could find even one listener (of many tried) who could hear a
> difference. I removed one bit of resolution at a time by rudely stripping
> off bits without dither. I removed bandwidth by means of downsampling
> using brick wall filtering.
>
> I found that resolution reduction became reliably audible with 14 bits
> resolution, and that bandwidth reduction became reliably audible with 16 KHz
> bandwidth. Resolution reduction to 15 bits and 19 KHz bandwidth reduction
> was undetectible.

What were your criteria for establishing that an effect was reliably
audible or (reliably) undetectable?

[[email protected]]

On Jul 1, 5:59=A0pm, Scott > wrote:

> Since you are not limiting it to audio this could turn into one of
> those lists like the long list of scientists who believe in evolution
> named Steve. You will find a lot of discussion on calibration in
> medical DBTs.

In other disciplines calibration might be needed, but this is an audio
forum, and the thread is about blind tests in audio, not drugs or
food. Just like audio, psychoacoustic research uses hearing as
detection tool, and 60 years of non-calibrated blind tests should have
some weight, or so I presume. I only searched JASA and only for ABX
and I did not look at all the research referenced to in the papers I
found, so that list of names could be much longer.

So far there's only the claim that blind tests need being calibrated,
no evidence, no indication of hearing related research where
calibration was indeed used. I hence do consider this claim as lame
excuse for not accepting the particular study, its results and
conclusions.

Klaus

[Arny Krueger]

"John Corbett" > wrote in message

> Arny Krueger wrote:
>
>> I've done extensive tests with other musical program
>> material that was recorded at 24/96 under near-lab
>> conditions in the interest of high actual recorded
>> dynamic range and extended bandwidth. For example, some
>> of my recordings had about 90 dB dynamic range, as
>> compared to an exceptionally good commercial recording's
>> dynamic range which is in the area of 75 dB.
>>
>> I've then reduced this signal's dynamic range and
>> bandwidth progressively until I could find even one
>> listener (of many tried) who could hear a difference. I
>> removed one bit of resolution at a time by rudely
>> stripping off bits without dither. I removed bandwidth
>> by means of downsampling using brick wall filtering.
>>
>> I found that resolution reduction became reliably
>> audible with 14 bits resolution, and that bandwidth
>> reduction became reliably audible with 16 KHz bandwidth.
>> Resolution reduction to 15 bits and 19 KHz bandwidth
>> reduction was undetectible.

> What were your criteria for establishing that an effect
> was reliably audible or (reliably) undetectable?

You're asking a question you know the answer to, John.

What's your point?

Since statistics is your bag, what criteria would you use?

[John Corbett]

Arny Krueger wrote:
> "Scott" > wrote in message
>
>
>> Let me demonstrate how this is an issue by using an
>> extreme. let's say the test had the wires screwed up and
>> the ABX test was in reality wired as an AAX test and B
>> was accidentally cut out of the loop. How sensitive would
>> this test be to audible differences between A and B? How
>> would you know if you did a test under these
>> circumstances and a got a null and never checked up on
>> the setup?
>
> You need to come up with an example that ABX testing doesn't make
> impossible. Level matching prevents miswiring an ABX test because you need
> to have proper identification and wiring of A and B to get through the level
> matching step.
>
>> It's about testing the test.
>
> The test tests itself during the setup phase.

Of course the sort of error that Scott described is not impossible.
See
http://www.bostonaudiosociety.org/bas_speaker/abx_testing2.htm
for a real-world example.

[John Corbett]

Arny Krueger wrote:
> "John Corbett" > wrote in message
>
>> Arny Krueger wrote:
>>
>>> I've done extensive tests with other musical program
>>> material that was recorded at 24/96 under near-lab
>>> conditions in the interest of high actual recorded
>>> dynamic range and extended bandwidth. For example, some
>>> of my recordings had about 90 dB dynamic range, as
>>> compared to an exceptionally good commercial recording's
>>> dynamic range which is in the area of 75 dB.
>>>
>>> I've then reduced this signal's dynamic range and
>>> bandwidth progressively until I could find even one
>>> listener (of many tried) who could hear a difference. I
>>> removed one bit of resolution at a time by rudely
>>> stripping off bits without dither. I removed bandwidth
>>> by means of downsampling using brick wall filtering.
>>>
>>> I found that resolution reduction became reliably
>>> audible with 14 bits resolution, and that bandwidth
>>> reduction became reliably audible with 16 KHz bandwidth.
>>> Resolution reduction to 15 bits and 19 KHz bandwidth
>>> reduction was undetectible.
>
>> What were your criteria for establishing that an effect
>> was reliably audible or (reliably) undetectable?
>
> You're asking a question you know the answer to, John.
>
> What's your point?
>
> Since statistics is your bag, what criteria would you use?


A well-designed and carefully executed ABX test can provide strong
evidence about audibility of a given stimulus.
However, an ABX test with a small number of trials (e.g., 16) and small
alpha level (such as .01) is inherently incapable of reliably detecting
small effects.

If you claim to have shown that something was inaudible, but are
unwilling to provide evidence to support your claim, then maybe I should
follow the advice you gave earlier in this very thread, and consider
your claims to be unsubstantiated and unsupported. In your own words:


> Nobody has any obligation to do even one little thing to support
their > claims. Then, every reasonable person recognizes the claims for
what
> they are, unsubstantiated, unsupported claims, and simply moves on.

And,

> On one side we have people who defend
> unsubstantiated claims, and on the other side we have people who are
> themselves capable of making claims and supporting them with reliable,
> well-thought out evidence.

[Arny Krueger]

"John Corbett" > wrote in message

> Arny Krueger wrote:
>> "John Corbett" > wrote in message
>>
>>> Arny Krueger wrote:
>>>
>>>> I've done extensive tests with other musical program
>>>> material that was recorded at 24/96 under near-lab
>>>> conditions in the interest of high actual recorded
>>>> dynamic range and extended bandwidth. For example,
>>>> some of my recordings had about 90 dB dynamic range, as
>>>> compared to an exceptionally good commercial
>>>> recording's dynamic range which is in the area of 75
>>>> dB.
>>>>
>>>> I've then reduced this signal's dynamic range and
>>>> bandwidth progressively until I could find even one
>>>> listener (of many tried) who could hear a difference.
>>>> I removed one bit of resolution at a time by rudely
>>>> stripping off bits without dither. I removed
>>>> bandwidth by means of downsampling using brick wall
>>>> filtering.
>>>>
>>>> I found that resolution reduction became reliably
>>>> audible with 14 bits resolution, and that bandwidth
>>>> reduction became reliably audible with 16 KHz
>>>> bandwidth. Resolution reduction to 15 bits and 19 KHz
>>>> bandwidth reduction was undetectible.
>>
>>> What were your criteria for establishing that an effect
>>> was reliably audible or (reliably) undetectable?
>>
>> You're asking a question you know the answer to, John.
>>
>> What's your point?
>>
>> Since statistics is your bag, what criteria would you
>> use?
>
>
> A well-designed and carefully executed ABX test can
> provide strong evidence about audibility of a given
> stimulus.
> However, an ABX test with a small number of trials (e.g.,
> 16) and small alpha level (such as .01) is inherently
> incapable of reliably detecting small effects.

The assertion without relevant substantiation is noted.

This is a very old discussion point, with divergent opinions on both sides.

I'm hardly married to 16 trials and 0.01 probability. It does turn out that
all observed circumstances where many more trials were attempted, showed
convergence to a random mean.

> If you claim to have shown that something was inaudible,
> but are unwilling to provide evidence to support your
> claim, then maybe I should follow the advice you gave
> earlier in this very thread, and consider your claims to
> be unsubstantiated and unsupported. In your own words:

>> Nobody has any obligation to do even one little thing to
>> support their > claims. Then, every reasonable person recognizes
> > the claims for what
>> they are, unsubstantiated, unsupported claims, and
>> simply moves on.

Interesting that we have yet another example of the same, above.

> And,

>> On one side we have people who defend
>> unsubstantiated claims, and on the other side we have
>> people who are themselves capable of making claims and
>> supporting them with reliable, well-thought out evidence.

Certainly, any well-thought-out evidence would be appreciated, but seeing
nothing new...

[Arny Krueger]

"John Corbett" > wrote in message

> Arny Krueger wrote:
>> "Scott" > wrote in message
>>
>>
>>> Let me demonstrate how this is an issue by using an
>>> extreme. let's say the test had the wires screwed up and
>>> the ABX test was in reality wired as an AAX test and B
>>> was accidentally cut out of the loop. How sensitive
>>> would this test be to audible differences between A and
>>> B? How would you know if you did a test under these
>>> circumstances and a got a null and never checked up on
>>> the setup?
>>
>> You need to come up with an example that ABX testing
>> doesn't make impossible. Level matching prevents
>> miswiring an ABX test because you need to have proper
>> identification and wiring of A and B to get through the
>> level matching step.
>>
>>> It's about testing the test.
>>
>> The test tests itself during the setup phase.
>
> Of course the sort of error that Scott described is not
> impossible. See
> http://www.bostonaudiosociety.org/bas_speaker/abx_testing2.htm
> for a real-world example.

Waiting for a well-thought-out analysis, as opposed to yet another
unsubstantiated assertion.


Also, the reference cited above is dated 1984 which was merely 26 years ago.

For something a little more current from the same publication, please see:

http://www.bostonaudiosociety.org/explanation.htm