Reply
 
Thread Tools Display Modes
  #1   Report Post  
Posted to rec.audio.high-end
Steven Sullivan Steven Sullivan is offline
external usenet poster
 
Posts: 1,268
Default AES article: hi-rez more like analog?

Actually it's from a 2007 AES conference, not the JAES, so I'm not sure it's peer
reviewed...but anyway:

"Which of the Two Digital Audio Systems Best Matches the Quality of the Analog System?"

http://www.hitech-projects.com/hera/...s/aar07pu4.pdf

It appears that the formats compared to a live feed (analog) were DXD (353.8 kHz/24) and
44.1/24, both in surround, using a blind comparison protocol. Two additional listening
conditions were tested : one where the A/D signal bandwith was 100 kHz (thanks to special
microphone 'super-tweeters') and the other where bandwidth was limited to 20kHz. The authors
say their results show that listeners 'more often than not' identify the hi-rez audio (and
not
the 44.1 kHz audio) as being similar in quality to the analog feed...but only when the
bandwidth is limited! In other words, only the initial sampling need be done in hi-rez, the
listening can (and SHOULD) be done in 'standard rez', to achieve the analog-like effect.

Their test setup and signals must be seen/read to be believed (the pdf includes photos).
They're not your typical listener setup, to say the least (compare to, say, Meyer and Moran's
setup for their SACD vs CD test). Also, I'm having trouble making heads or tails of their
statistics.

Also being discussed on HA.org

http://www.hydrogenaudio.org/forums/...howtopic=55966

___
-S
"As human beings, we understand the world through simile, analogy,
metaphor, narrative and, sometimes, claymation." - B. Mason
  #2   Report Post  
Posted to rec.audio.high-end
bob bob is offline
external usenet poster
 
Posts: 670
Default AES article: hi-rez more like analog?

Here's the first paragraph of their Conclusion section:

"The results of this test indicate that listeners more often than not
identify high-resolution audio as being similar in quality to the
unprocessed analog audio. This conclusion, based on listening to the
audio scene captured and reproduced with microphones and loudspeakers
limited to 20 kHz bandwidth, indicates that high-sampling conversion
system seems to be more transparent and provides a higher degree of
fidelity to the analog reference."

Here's what their data says: Out of 54 trials with that limited
bandwidth, subjects chose the high-sampling-rate system as closer to
the original analog feed 31 times. That's not even statistically
significant at the 90% confidence level. So by their own data, they
can't reject the possibility that their subjects could not tell the
difference between 352.8 kHz sampling and 44.1, even under the best
scenario.

What's odd is that, when they used mikes and speakers with a bandwidth
extended to 100 kHz, the high sampling rate did substantially worse
(23/60). That result is highly counterintuitive, and the authors twist
themselves into pretzels trying to explain it, with no success.

So they're making claims their own data don't fully support and they
can't explain. I think there's no there there.

bob
  #3   Report Post  
Posted to rec.audio.high-end
willbill willbill is offline
external usenet poster
 
Posts: 58
Default AES article: hi-rez more like analog?

Steven Sullivan wrote:

Actually it's from a 2007 AES conference, not the JAES,
so I'm not sure it's peer reviewed...but anyway:

"Which of the Two Digital Audio Systems Best Matches the Quality
of the Analog System?"

http://www.hitech-projects.com/hera/...s/aar07pu4.pdf


nice ref! thank you very much


It appears that the formats compared to a live feed (analog) were
DXD (353.8 kHz/24) and 44.1/24, both in surround, using a blind
comparison protocol. Two additional listening conditions were tested:
one where the A/D signal bandwith was 100 kHz (thanks to special
microphone 'super-tweeters') and the other where bandwidth was
limited to 20kHz. The authors say their results show that listeners
'more often than not' identify the hi-rez audio (and not the 44.1 kHz
audio) as being similar in quality to the analog feed...but only when
the bandwidth is limited! In other words, only the initial sampling
need be done in hi-rez, the listening can (and SHOULD) be done in
'standard rez', to achieve the analog-like effect.

Their test setup and signals must be seen/read to be believed (the pdf
includes photos). They're not your typical listener setup, to say the
least (compare to, say, Meyer and Moran's setup for their SACD vs
CD test).


if there is an open on-line ref to this, i'd very
much appreciate someone providing it; thanks in advance

Also, I'm having trouble making heads or tails of their statistics.

Also being discussed on HA.org

http://www.hydrogenaudio.org/forums/...howtopic=55966


very nice 2nd ref!

only 5 people, but still very interesting

i'm going to have to start reading www.hydrogenaudio.org
on a regular basis

imho, multichannel sound is not driven by audio,
it is driven by movies. of the recent 2 formats
(HD-DVD and Blu-ray), which really has the better
multichannel sound?

imho#2, better sound isn't going to make any diff
on which wins. or are they both too similar
(on sound) to be worth worrying about it?

where is Kalman when we need him?

fwiw this past few days i've gotten
a Toshiba HD-DVD A35 player, with an
open order for a copy of Magic Flute
(arriving after Xmas)

http://www.amazon.com/Mozart-Zauberf...7563048&sr=8-1

the 4 current customer reviews give it high marks
on sound. read them for yourself

of course SACD doesn't have video, but is the
multichannel sound on Magic Flute comparable
to SACD?

all ears, bill

"As human beings, we understand the world through simile, analogy,
metaphor, narrative and, sometimes, claymation." - B. Mason


Never confuse movement with action.
Ernest Hemingway (1899-1961)

  #4   Report Post  
Posted to rec.audio.high-end
Steven Sullivan Steven Sullivan is offline
external usenet poster
 
Posts: 1,268
Default AES article: hi-rez more like analog?

willbill wrote:
Steven Sullivan wrote:


Actually it's from a 2007 AES conference, not the JAES,
so I'm not sure it's peer reviewed...but anyway:

"Which of the Two Digital Audio Systems Best Matches the Quality
of the Analog System?"

http://www.hitech-projects.com/hera/...s/aar07pu4.pdf


nice ref! thank you very much



It appears that the formats compared to a live feed (analog) were
DXD (353.8 kHz/24) and 44.1/24, both in surround, using a blind
comparison protocol. Two additional listening conditions were tested:
one where the A/D signal bandwith was 100 kHz (thanks to special
microphone 'super-tweeters') and the other where bandwidth was
limited to 20kHz. The authors say their results show that listeners
'more often than not' identify the hi-rez audio (and not the 44.1 kHz
audio) as being similar in quality to the analog feed...but only when
the bandwidth is limited! In other words, only the initial sampling
need be done in hi-rez, the listening can (and SHOULD) be done in
'standard rez', to achieve the analog-like effect.

Their test setup and signals must be seen/read to be believed (the pdf
includes photos). They're not your typical listener setup, to say the
least (compare to, say, Meyer and Moran's setup for their SACD vs
CD test).


if there is an open on-line ref to this, i'd very
much appreciate someone providing it; thanks in advance


Meyer and Moran's paper isn't available online AFAIK, but it's in
circulation...

Also, I'm having trouble making heads or tails of their statistics.

Also being discussed on HA.org

http://www.hydrogenaudio.org/forums/...howtopic=55966


very nice 2nd ref!


only 5 people, but still very interesting


i'm going to have to start reading www.hydrogenaudio.org
on a regular basis


Also discussed on AVSforum

http://www.avsforum.com/avs-vb/showthread.php?t=953640

I'm kinda guessing it WON'T be hashed over in Stereophile, though
even thougb John Atkinson is citing this paper as proof that
Redbook sample rate is inadequate for home audio.

'"In October I will write more on Peter's ideas about why this should be. I will end this month's essay by quoting, from
a paper given at the conference, the results of experiments on the audibility of high sampling rates: "To achieve a
higher degree of fidelity to the live analog reference, we need to convert audio using a high sampling rate even when we
do not use microphones and loudspeakers having bandwidth extended far beyond 20kHz. Listeners judge high sampling
conversion as sounding more like the analog reference when listening to standard audio bandwidth." (footnote 2)

So that's that, then. "

http://stereophile.com/asweseeit/907awsi/

Atkinson used to post here, maybe he can be induced to comment?

of course SACD doesn't have video, but is the
multichannel sound on Magic Flute comparable
to SACD?


all ears, bill


Surely DVD-A is comparable to SACD (I'd wager that in a blind test, DTS 24/96 is
indistinguisbale form them too). So why not the new 'hi rez' formats?

___
-S
"As human beings, we understand the world through simile, analogy,
metaphor, narrative and, sometimes, claymation." - B. Mason
  #5   Report Post  
Posted to rec.audio.high-end
Doug McDonald[_2_] Doug McDonald[_2_] is offline
external usenet poster
 
Posts: 6
Default AES article: hi-rez more like analog?

bob wrote:

What's odd is that, when they used mikes and speakers with a bandwidth
extended to 100 kHz, the high sampling rate did substantially worse
(23/60). That result is highly counterintuitive, and the authors twist
themselves into pretzels trying to explain it, with no success.


There is a stock answer to this, which may or not be right, but always
needs to be tested. That is, with nonlinear speakers, inaudible
high frequencies can cause audible intermodulation
distortion. This can of course be tested if you have a really really
good high frequency test system (super linear
high frequency microphone and digitizer)
to aim at the speakers.

Doug McDonald


  #6   Report Post  
Posted to rec.audio.high-end
willbill willbill is offline
external usenet poster
 
Posts: 58
Default AES article: hi-rez more like analog?

Steven Sullivan wrote:

willbill wrote:


Also discussed on AVSforum

http://www.avsforum.com/avs-vb/showthread.php?t=953640

I'm kinda guessing it WON'T be hashed over in Stereophile, though
even thougb John Atkinson is citing this paper as proof that
Redbook sample rate is inadequate for home audio.

'"In October I will write more on Peter's ideas about why this should be. I will end this month's essay by quoting, from
a paper given at the conference, the results of experiments on the audibility of high sampling rates: "To achieve a
higher degree of fidelity to the live analog reference, we need to convert audio using a high sampling rate even when we
do not use microphones and loudspeakers having bandwidth extended far beyond 20kHz. Listeners judge high sampling
conversion as sounding more like the analog reference when listening to standard audio bandwidth." (footnote 2)

So that's that, then. "

http://stereophile.com/asweseeit/907awsi/


i've got the hard copy but hadn't thought
to look at the on-line, which makes it easy
to look at his refs in earlier editions


Atkinson used to post here, maybe he can be induced to comment?


fwiw, he did some minor posting in one of
the other rec.audio... newsgroups a week
or two ago


of course SACD doesn't have video, but is the
multichannel sound on Magic Flute comparable
to SACD?


all ears, bill


Surely DVD-A is comparable to SACD (I'd wager that in a blind test, DTS 24/96 is
indistinguisbale form them too). So why not the new 'hi rez' formats?


i've got the player (OPPO), but so far have
only bought SACD disks

bill
  #7   Report Post  
Posted to rec.audio.high-end
Arny Krueger Arny Krueger is offline
external usenet poster
 
Posts: 17,262
Default AES article: hi-rez more like analog?

"bob" wrote in message


Here's the first paragraph of their Conclusion section:


"The results of this test indicate that listeners more
often than not identify high-resolution audio as being
similar in quality to the unprocessed analog audio.


IOW, they mostly fail to hear a difference.

This conclusion, based on listening to the audio scene
captured and reproduced with microphones and loudspeakers
limited to 20 kHz bandwidth, indicates that high-sampling
conversion system seems to be more transparent and
provides a higher degree of fidelity to the analog
reference."


????????????????

Here's what their data says: Out of 54 trials with that
limited bandwidth, subjects chose the high-sampling-rate
system as closer to the original analog feed 31 times.


I sense a conclusion that they were randomly guessing coming on.

That's not even statistically significant at the 90%
confidence level. So by their own data, they can't reject
the possibility that their subjects could not tell the
difference between 352.8 kHz sampling and 44.1, even
under the best scenario.


So much for the hypothesis that the audio CD format is somehow deficient.

What's odd is that, when they used mikes and speakers
with a bandwidth extended to 100 kHz, the high sampling
rate did substantially worse (23/60).


Random guessing sometimes works out that way. Reams have written by ignorant
audiophiles about the meaning of worse results than random guessing. In the
end, worse than random guessing is probably best interpreted as being an
indictment of the experiment.

If people are truely producing independent results, then their results will
converge to random guessing when they can't hear a difference. When they
are guessing wrong more often than 50%, the hypothesis that they are
producing independent results can be questioned.

That result is
highly counterintuitive, and the authors twist themselves
into pretzels trying to explain it, with no success.


They don't want to admit that their results are critical of the over-all
quality of the experiment.

So they're making claims their own data don't fully
support and they can't explain. I think there's no there
there.


Agreed.

  #8   Report Post  
Posted to rec.audio.high-end
Arny Krueger Arny Krueger is offline
external usenet poster
 
Posts: 17,262
Default AES article: hi-rez more like analog?

"Steven Sullivan" wrote in message


Actually it's from a 2007 AES conference, not the JAES,
so I'm not sure it's peer reviewed


Conference papers aren't reviewed at all - only the title and abstract have
been seen by the session chairman.

...but anyway:


"Which of the Two Digital Audio Systems Best Matches the
Quality of the Analog System?"

http://www.hitech-projects.com/hera/...s/aar07pu4.pdf


It appears that the formats compared to a live feed
(analog) were DXD (353.8 kHz/24) and
44.1/24, both in surround, using a blind comparison
protocol.


It appears that the program material is best described as being "sound
effects".

Also, every time some high resoution advocate cites this paper, ask them to
reflect on how closely it resembles listening to music for pleasure in a
residential listening room. ;-)

Two additional listening conditions were tested
: one where the A/D signal bandwith was 100 kHz (thanks
to special microphone 'super-tweeters') and the other
where bandwidth was limited to 20kHz. The authors say
their results show that listeners 'more often than not'
identify the hi-rez audio (and not
the 44.1 kHz audio) as being similar in quality to the
analog feed...but only when the bandwidth is limited!


Let's cut to the chase and look at the raw data on page 19.

As I read it, there were 10 subjects and 6 listening sessions for which
individual responses were required. IOW, 60 trials.

I pasted their matrix into Excel and tried to do some quick sums. I came up
with Test Condition 1 = 23/60 and Test Condition 2 = 31/60. IOW, one test
produced an outcome that was worse than random guessing, and the other was
random guessing.

Results that are worse than random guessing may cause some head scratching,
but they are not all that unusual in experiments like this where
communication between the listeners can affect the outcome.

The most probable explanation for worse-than-random results is that that
some of the listeners were basing their results on their perceptions of what
other listeners were perceiving, and the total number of independent
responses was far less than what you get from a naive count of the actual
responses.

IOW, the actual situation was not 23 independent responses of 60 trials, but
maybe more like 4 independent responses of 10 independent trials, so the
true numbers were so small that statistics doesn't really apply.

I think these results would pretty well explain themselves to just about
anybody, were they reproduced any place within the actual paper.

Short answer - the outcome was random guessing, and both the test itself and
the statistical analysis were greviously flawed.

I guess we can chalk this paper up as yet another "Proof by complex
statistical analysis" which defies common sense.

  #9   Report Post  
Posted to rec.audio.high-end
bob bob is offline
external usenet poster
 
Posts: 670
Default AES article: hi-rez more like analog?

On Dec 13, 10:15 pm, Doug McDonald
wrote:
bob wrote:
What's odd is that, when they used mikes and speakers with a bandwidth
extended to 100 kHz, the high sampling rate did substantially worse
(23/60). That result is highly counterintuitive, and the authors twist
themselves into pretzels trying to explain it, with no success.


There is a stock answer to this, which may or not be right, but always
needs to be tested. That is, with nonlinear speakers, inaudible
high frequencies can cause audible intermodulation
distortion.


But wouldn't these inaudible frequencies be present in the original
analog feed, as well as the high-sampling rate digital conversion?
That should make the two sound *more* similar, and less like the 44.1
conversion, which would filter those frequencies out before they
reached the speakers.

bob
  #10   Report Post  
Posted to rec.audio.high-end
Steven Sullivan Steven Sullivan is offline
external usenet poster
 
Posts: 1,268
Default AES article: hi-rez more like analog?

Arny Krueger wrote:
"Steven Sullivan" wrote in message


Actually it's from a 2007 AES conference, not the JAES,
so I'm not sure it's peer reviewed


Conference papers aren't reviewed at all - only the title and abstract have
been seen by the session chairman.


...but anyway:


"Which of the Two Digital Audio Systems Best Matches the
Quality of the Analog System?"

http://www.hitech-projects.com/hera/...s/aar07pu4.pdf


It appears that the formats compared to a live feed
(analog) were DXD (353.8 kHz/24) and
44.1/24, both in surround, using a blind comparison
protocol.


It appears that the program material is best described as being "sound
effects".


Also, every time some high resoution advocate cites this paper, ask them to
reflect on how closely it resembles listening to music for pleasure in a
residential listening room. ;-)


Two additional listening conditions were tested
: one where the A/D signal bandwith was 100 kHz (thanks
to special microphone 'super-tweeters') and the other
where bandwidth was limited to 20kHz. The authors say
their results show that listeners 'more often than not'
identify the hi-rez audio (and not
the 44.1 kHz audio) as being similar in quality to the
analog feed...but only when the bandwidth is limited!


Let's cut to the chase and look at the raw data on page 19.


As I read it, there were 10 subjects and 6 listening sessions for which
individual responses were required. IOW, 60 trials.


I pasted their matrix into Excel and tried to do some quick sums. I came up
with Test Condition 1 = 23/60 and Test Condition 2 = 31/60. IOW, one test
produced an outcome that was worse than random guessing, and the other was
random guessing.


Results that are worse than random guessing may cause some head scratching,
but they are not all that unusual in experiments like this where
communication between the listeners can affect the outcome.


The most probable explanation for worse-than-random results is that that
some of the listeners were basing their results on their perceptions of what
other listeners were perceiving, and the total number of independent
responses was far less than what you get from a naive count of the actual
responses.


There was science news article recently -- can't find it right now, but it was in the last
week or so -- reporting a study demonstrating the influence of having someone else in the
room, on perception of *comedy* (in this case, the probe signal was sketches from Saturday
Night Live). Just seeing the back of someone else's head, as they watched, was enough to
influence subjects to find the presentation funny or not.

___
-S
"As human beings, we understand the world through simile, analogy,
metaphor, narrative and, sometimes, claymation." - B. Mason


  #11   Report Post  
Posted to rec.audio.high-end
bob bob is offline
external usenet poster
 
Posts: 670
Default AES article: hi-rez more like analog?

On Dec 14, 1:01 pm, "Arny Krueger" wrote:

I pasted their matrix into Excel and tried to do some quick sums. I came up
with Test Condition 1 = 23/60 and Test Condition 2 = 31/60.


To be precise, Condition 2 is 31/54, since one subject's results were
not available. (It happens.)

IOW, one test
produced an outcome that was worse than random guessing, and the other was
random guessing.


Actually, both could be random guessing, since neither hits even the
90% confidence level.

Results that are worse than random guessing may cause some head scratching,
but they are not all that unusual in experiments like this where
communication between the listeners can affect the outcome.


Not the case here, where each subject was tested individually. Also,
unlike an ABX test, there is no "wrong" answer here. Either conversion
could be judged better. (Unless, of course, you're Philips and you
sell hi-rez converters.)

The most probable explanation for worse-than-random results is that that
some of the listeners were basing their results on their perceptions of what
other listeners were perceiving, and the total number of independent
responses was far less than what you get from a naive count of the actual
responses.

IOW, the actual situation was not 23 independent responses of 60 trials, but
maybe more like 4 independent responses of 10 independent trials, so the
true numbers were so small that statistics doesn't really apply.


Again, this is an inaccurate description of the actual test. The
responses were independent.

I think these results would pretty well explain themselves to just about
anybody, were they reproduced any place within the actual paper.

Short answer - the outcome was random guessing, and both the test itself and
the statistical analysis were greviously flawed.


The test itself was not grievously flawed. The only obvious problem
was the one Scott mentioned--using different mikes for the two
conditions. But that only matters if you're comparing the results of
under the two conditions. Looked at individually, the two test
conditions tell us nothing.

The statistical analysis is another story.

I guess we can chalk this paper up as yet another "Proof by complex
statistical analysis" which defies common sense.


Agreed.

bob
  #12   Report Post  
Posted to rec.audio.high-end
Arny Krueger Arny Krueger is offline
external usenet poster
 
Posts: 17,262
Default AES article: hi-rez more like analog?

"bob" wrote in message

On Dec 14, 1:01 pm, "Arny Krueger"
wrote:

I pasted their matrix into Excel and tried to do some
quick sums. I came up with Test Condition 1 = 23/60 and
Test Condition 2 = 31/60.


To be precise, Condition 2 is 31/54, since one subject's
results were not available. (It happens.)

IOW, one test
produced an outcome that was worse than random guessing,
and the other was random guessing.


Actually, both could be random guessing, since neither
hits even the 90% confidence level.

Results that are worse than random guessing may cause
some head scratching, but they are not all that unusual
in experiments like this where communication between the
listeners can affect the outcome.


Not the case here, where each subject was tested
individually.


Perhaps not that individually.

Also, unlike an ABX test, there is no "wrong" answer here.


Sure there is.

Inconsistency is wrong.

Either conversion could be judged
better. (Unless, of course, you're Philips and you sell
hi-rez converters.)


The most probable explanation for worse-than-random
results is that that some of the listeners were basing
their results on their perceptions of what other
listeners were perceiving, and the total number of
independent responses was far less than what you get
from a naive count of the actual responses.


IOW, the actual situation was not 23 independent
responses of 60 trials, but maybe more like 4
independent responses of 10 independent trials, so the
true numbers were so small that statistics doesn't
really apply.


Again, this is an inaccurate description of the actual
test. The responses were independent.


But how independent?

I think these results would pretty well explain
themselves to just about anybody, were they reproduced
any place within the actual paper.


Short answer - the outcome was random guessing, and both
the test itself and the statistical analysis were
greviously flawed.


The test itself was not grievously flawed.


It acted that way. A good test of perception is either random or correlated
with the stimulus. When the correlation is negative, something went badly
wrong.

The only
obvious problem was the one Scott mentioned--using
different mikes for the two conditions. But that only
matters if you're comparing the results of under the two
conditions. Looked at individually, the two test
conditions tell us nothing.


That's a sign of a flawed test - it tells us less than we expected to find
out.

The statistical analysis is another story.


I guess we can chalk this paper up as yet another "Proof
by complex statistical analysis" which defies common
sense.


Agreed.


Reply
Thread Tools
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
IEEE Article Bob Cain Pro Audio 0 May 3rd 06 12:31 AM
Did anyone see the article . . . . Jon Yaeger Vacuum Tubes 3 January 25th 06 11:22 PM
For analog fans: BOSS DM-3 DM3 ANALOG DELAY PEDAL VINTAGE RARE [email protected] Tech 0 January 28th 05 06:21 PM
Whole article John Payne Pro Audio 2 September 5th 03 03:02 PM


All times are GMT +1. The time now is 02:31 PM.

Powered by: vBulletin
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 AudioBanter.com.
The comments are property of their posters.
 

About Us

"It's about Audio and hi-fi"