Home |
Search |
Today's Posts |
|
#1
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
Darryl Miyaguchi wrote in message ...
For what it's worth, I have performed enough ABX testing to convince myself that it's possible for me to detect volume differences 0.5 dB using music, so I doubt very highly that a group test would fail to show that 1.75 dB differences on a variety of different music are not audible using a DBT. I think it's generally acknowledged that such differences are audible. Mirabel seems to be arguing that, given what he claims is a 1.75 dB difference, every member of Greenhill's panel should have scored at or near perfection, and the fact that they didn't bespeaks some flaw in Greenhill's methodology. I'm not yet convinced that there really was a 1.75 dB difference here, however. What Greenhill says about the 24-gauge cable is: "Its 1.8-ohm resistance resulted in a 1.76-dB insertion loss with an 8-ohm resistive load." How does this translate to the specific test in question, which used a recording of a male a cappella chorus (where the fundamental tones, at least, range from 60 to less than 1000 Hz)? Greenhill only level-matched a single pink noise test, and the only times he discusses levels in the article appear to be in reference to pink noise tests. E.g.: "A 1- to 2-dB decrease in sound level was measured for the 24-gauge wire during the pink noise listening tests." I freely admit that I'm out of my element here, but I don't think we can automatically assume that there was a similar difference in SPL when listening to the choral music. Hopefully, someone with some technical expertise can shed some further light on this. bob |
#2
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
"ludovic mirabel" wrote in message
news:uCJLa.56660$Ab2.130013@sccrnsc01 (KikeG) wrote in message et... (ludovic mirabel) wrote in message news:bn2La.36126$Ab2.61637@sccrnsc01... I haven't read Greenhill's tests report, but it seems there's some controversy over what you are saying. Even if that was true, that would suggest there were some problems at the test, since anyone of my family could ABX that wideband level difference quite easily. Enrique (Kike For friends) Apologies for dealing just with this for the time being . It concerns intellectual honesty something I happen to be touchy about. The " Cable test's (" Stereo Review",Aug. Â'83) proctor and reporter was immaculately "objectivist" L. Greenhill- still alive and writing for "The Stereophile" Looking at the calendar, I see that in two months it will be 20 years since this test was published. Considering editing and publishing delays, it's already been 20 years since the test was done. If this were the only test that was done in the history of man, or if every other or the vast majority of DBTs that were done since then agreed with its results, then citing it would make some sense. Regrettably, DBTs and even ABX tests involving level differences have been done many times since then, and very many of those listening tests have provided far more sensitive results. Therefore, discussion of Greenhill's 1983 test as if it were indicative, representative or binding on what's happening today is futile and misleading. Anybody who wishes to do DBTs to investigate the audibility of level differences can do so easily using files they can freely download from the PCABX web site. I think it would be interesting for people to report the results they obtain with those files. IME the audibility of level differences for people with normal hearing and typical listening environments is closer to 0.5 dB than the 1.75 dB reported by Greenhill. Since individual listeners have different test environments and different ears, their results can be reasonably be expected to vary. In fact if the results didn't vary, it would suggest that there is something wrong with the test procedure since it would not be demonstrating the existence of well-known differences in individual listening acuity. However, it is equally well known that some place around level differences of 0.2 dB, nobody hears nuttin'. |
#3
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
|
#4
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
|
#7
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
Darryl Miyaguchi
wrote: In some parts I will reply to the post to which Mr Miyaguch is replying: On 1 Jul 2003 15:10:59 GMT, (ludovic mirabel) wrote: We're talking about Greenhill old test not because it is perfect but because no better COMPONENT COMPARISON tests are available. In fact none have been published according to MTry's and Ramplemann's bibliographies since 1990. This is simply not true. I have published 2 double blind tests personally, one of which covered 3 different wires subsequent to 1990. I frequently see the distinction being made between audio components and other audio related things (such as codecs) when it comes to talking about DBT's. What is the reason for this? In my opinion, there are two topics which should not be mixed up: 1) The effectiveness of DBT's for determining whether an audible difference exists 2) The practical usefulness of using DBT's for choosing one audio producer (component or codec) over another. Actually if nominally competent components such as wires, parts, bits and amplifiers have never been shown to materially affect the sound of reproduced music in normally reverberant conditions why would ANYONE need to conduct more experimentation, or any listening test, to choose between components? Simply choose the one with the other non-sonic characteristics (features, price, terms, availability, cosmetics, style...) that suit your fancy. Indeed 20 years ago, when I still had a day job, Radio Shack often had the "perfect" characteristic to guide purchase, which was "open on Sunday." I am not knowledgeable enough to decide on differences between your and Greenhill's interpretation of the methods and results. In my simplistic way I'd ask you to consider the following: PINK NOISE signal: 10 out of 11 participants got the maximum possible correct answers: 15 out of 15 ie. 100%. ONE was 1 guess short. He got only 14 out of 15. When MUSIC was used as a signal 1 (ONE) listener got 15 corrects, 1 got 14 and one 12. The others had results ranging from 7 and 8 through 10 to (1) 11. My question is: was there are ANY significant difference between those two sets of results? Is there a *possibility* that music disagrees with ABX or ABX with music? No: it just means with the right set of music 2 dB is at the threshold. Don't forget that listening position affects this stuff too. Also Mr Atkinson would say that perhaps the lower scoring subjects didn't have personal control of the switching. Even between two samples of music (no pink noise involved), I can certainly believe that a listening panel might have more or less difficulty in determining if they hear an audible difference. It doesn't follow that music in general is interfering with the ability to discriminate differences when using a DBT. Actually it simp,y shows that pink noise and other test signals are the most sensitive of programs. It may be possible to divulge a 'difference' with noise that would never be encountered with any known program material. It's also possible that certain programs, such as Arny Kreuger's special signals, might disclose differences that may never be encountered with commercially available music (or other) programs. So? I would aoppreciate it if would try and make it simple leaving "confidence levels" and such out of it. You're talking to ordinary audiophiles wanting to hear if your test will help them decide what COMPONENTS to buy. As before; you haven't ever been precluded from making any purchase decisions from scientific evidence before; why should any disclosure affect that now or in the future. Examination of the extant body of controlled listening tests available contain enough information to aid any enthusiast in making good decisions. Even IF the existing evidence shows that wire is wire (and it does) how does that preclude any person from making any purchase decision? In my way of thinking it just might be useful for a given individual to know what has gone before (and what hasn't.) I still don't know how this cannot do anything but IMPROVE decision making? See my first comments. It's too easy to mix up the topic of the sensitivity of DBT's as instruments for detecting audible differences with the topic of the practicality of using DBT's to choose hifi hardware. The latter is impractical for the average audiophile. No it's not. Just like 0-60 times, skid-pad and EPA mileage tests simply cannot be made by the typical individual that doesn't mean that they cannot be used to improve decision-making. Likewise the body of controlled listening test results can be very useful to any individual that wishes to make use of them to guide decisions. Otherwise the only information one has is "guidance" from sellers, anecdotal reports and "open" listening tests. The latter , of course, is quite subject to non-sonic influence. So IMO, a person truly interested in maximizing the sonic-quality throughput of his system simply MUST examine the results of bias controlled listening tests OR fall prey to non-sonic biasing factors, even if they are inadvertent. Who can argue with motherhood? The problem is that there are NO ABX COMPONENT tests being published- neither better nor worse, NONE. I heard of several audio societies considering them. No results. Not from the objectivist citadels: Detroit and Boston. Why?. Did they pan out? Given the two dozen controlled listening tests of power amplifiers published through 1991 doesn't it seem that no one needs to conduct more? Wires? The last test I published was in 1995. Not late enough? Why not? No manufacturer has EVER produced a single bias controlled experiment that showed their wires had a sound of their own in over 30 years. Why should one expect one now? I certainly can't do it; although I've given it my level (no pun intended) best. IOW, I can't produce an experiment that shows nominally competent wires ain't wires .... 'cuz they ain't. I can think of a couple of reasons: 1. It's expensive and time consuming to perfom this type of testing 2. The audible differences are, in actuality, too subtle to hear, ABX or not. Why bother with such a test? Why bother in performing a sound quality "test" that the manufacturers of the equipment can't produce? IF amps ain't amps; wires ain't wires and parts ain't parts then why haven't the makers and sellers of this stuff produced repeatable bias controlled listening tests that show this to be untrue? Then there is the possibility that you seem to be focussing on, ignoring the above two: 3. DBT's in general may be decreasing the ability to hear subtle differences. Actually they preclude the ability to "hear" non-sonic differences. Which of the the above reaons do you think are most likely? Moving away from the question Greenhill was investigating (audible differences between cables) and focusing only on DBT testing and volume differences: it is trivial to perform a test of volume difference, if the contention is being made that a DBT hinders the listener from detecting 1.75 dB of volume difference. Especially if the listeners have been trained specifically for detecting volume differences prior to the test. However, such an experiment would be exceedingly uninteresting, and I have doubts it would sway the opinion of anybody participating in this debate. The volume difference was just a by-effect of a comparison between cables. And yes, TRAINED people would do better than Greenhill's "Expert audiophiles" ie rank amateurs just like us. Would some though do better than the others and some remain untrainable? Just like us. I think Ludovic is "untrainable" because he will accept only answers he already believes are true. I have no doubt that there are some people who are unreliable when it comes to performing a DBT test. In a codec test using ABC/HR, if somebody rates the hidden reference worse than the revealed reference (both references are identical), his listening opinion is either weighted less or thrown out altogether. What you are describing is 'reverse significance' which is typically a inadvertant form of internal bias. For what it's worth, I have performed enough ABX testing to convince myself that it's possible for me to detect volume differences 0.5 dB using music, so I doubt very highly that a group test would fail to show that 1.75 dB differences on a variety of different music are not audible using a DBT. I can easily hear 1db difference between channels, and a change of 1 db. What I can't do is to have 80 db changed to 81 db, then be asked if the third unknown is 80 or 81 dbs. and be consistently correct. Perhaps I could if I trained as much as you have done. Perhaps not Some others could, some couldn't. We're all different. Produce a test which will be valid for all ages, genders, extent of training, innate musical and ABxing abilities, all kinds of musical experience and preference. Then prove BY EXPERIMENT that it works for COMPARING COMPONENTS. So that anyone can do it and if he gets a null result BE CERTAIN that with more training or different musical experience he would not hear what he did not hear before. And perhaps just get on widening his musical experience and then rcompare (with his eyes covered if he is marketing susceptible) Let's keep it simple. We're audiophiles here. We're talking about MUSICAL REPRODUCTION DIFFERENCES between AUDIO COMPONENTS. I looked at your internet graphs. They mean zero to me. I know M. Levinsohn, Quad, Apogee, Acoustat not the names of your codecs. You assure me that they are relevant. Perhaps. Let's see BY EXPERIMENT if they do. In the meantime enjoy your lab work. Ludovic Mirabel Are you really telling me that you didn't understand the gist of the group listening test I pointed you to? For one thing, it says that although people have different individual preferences about how they evaluate codec quality, as a group, they can identify trends. This, despite the variety of training, hearing acuity, audio equipment, and listening environment. Another point is that it would be more difficult to identify trends if such a study included the opinions of people who judge the hidden reference to be worse than the revealed reference (simultaneously judging the encoded signal to be the same as the revealed reference). In other words, there are people whose listening opinions can't be trusted, and the DBT is designed to identify them. That result identifies a form of experimental bias, does it not? The last point is that I can see no reason why such procedures could not (in theory, if perhaps not in practical terms) be applied to audio components. Why don't you explain to me what the difference is (in terms of sensitivity) between using DBT's for audio codecs and using DBT's for audio components? Darryl Miyaguchi There is no difference. It seems to me that this poster may have never taken a bias controlled listening test or, if he has, the results didn't fit with prior held expectations. It's much easier to argue with the existing evidence than prove that you can hear things that no human has been able to demonstrate, when not peeking. As I've said before; there are many proponents of high-end sound of wires, amps and parts ... but, so far, no one (in over 30 years) has ever produced a single repeatable bias controlled experiment that shows that nominally competent products in a normally reverberant environment (listening room) have any sonic contribution of their own. Nobody! Never! How about some evidence? I'll believe in BigFoot ....just show me the body! |
#8
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
(ludovic mirabel) wrote in message ...
We're talking about Greenhill old test not because it is perfect but because no better COMPONENT COMPARISON tests are available. In fact none have been published according to MTry's and Ramplemann's bibliographies since 1990. I gave a link to one at my previous message, related to soundcards and a DAT ( http://www.hydrogenaudio.org/index.p...hl=pitch&st=0& ). It revealed some audible differences. Soundcards and DATs are audio components, aren't they? There's another one concerning just a soundcard he http://www.hydrogenaudio.org/index.p...d6a76f1f8d0738 It finally revealed no audible differences. About Greenhill's test and level differences: PINK NOISE signal: 10 out of 11 participants got the maximum possible correct answers: 15 out of 15 ie. 100%. ONE was 1 guess short. He got only 14 out of 15. When MUSIC was used as a signal 1 (ONE) listener got 15 corrects, 1 got 14 and one 12. The others had results ranging from 7 and 8 through 10 to (1) 11. My question is: was there are ANY significant difference between those two sets of results? Is there a *possibility* that music disagrees with ABX or ABX with music? I would aoppreciate it if would try and make it simple leaving "confidence levels" and such out of it. You're talking to ordinary audiophiles wanting to hear if your test will help them decide what COMPONENTS to buy. Citing Greenhill's article, refering to 24-gauge cable: "Its 1.8-ohm resistance resulted in a 1.76-dB insertion loss with an 8-ohm resistive load". I don't know if this has been addressed before, but this 1.76 dB loss corresponds to a pure resistive load. Speakers are quite different from pure resistive loads, in the sense that their impedance varies with frequency, being this impedance higher than the nominal in most part of the spectrum. So, this 1.8 ohm in series with a real world speaker would definitely result in an attenuation below 1.76 dB over the whole audible spectrum on a wideband signal. Also, the attenuation will vary with frequency, so that max attenuation will be at the frequencies where the speaker impedance is minimum, and there will be little attenuation at frequencies where speaker impedance is maximum. So, the whole attenuation will depend on the spectrum of music used. There's a possibility that choral music has most of its content at frequencies where attenuation was not high, but it's difficult to know without having access to the actual music used and the speaker impedance curve. Said, that, I tried yesterday to ABX an 1.7 dB wideband (frequency constant) level attenuation on a musical sample. Result: 60/60 on a couple of minutes, Not a single miss. It is obvious to hear, but one could argue I'm trained. Despite that, I claim that, any person that does not have serious auditive problems, would be able to ABX a 1.7 dB wideband level difference on any kind of real-world music, being trained or not in ABX testing, just taking a couple of minutes to explain him the basics of ABX testing. Now, you make a point in that you have to be trained in ABXing to be good at it. I say that you have to be trained in *any* method you use to be good at it. Also, ABX testing per se requires little training. What takes more training is to learn to detect reliabily some kind of differences, whether you use ABX or not. Serious ABX training is required just for detecting very subtle differences, just like in every other area where high performance is required. And finally, an analogy: you can't evaluate driving comfort in cars without driving them, so you have to learn how to drive in order to evaluate driving comfort. Driving a car is the only reliable way to evaluate driving confort, whether you are good at it or not. And such... |
#9
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
Tom said
Actually if nominally competent components such as wires, parts, bits and amplifiers have never been shown to materially affect the sound of reproduced music in normally reverberant conditions why would ANYONE need to conduct more experimentation, or any listening test, to choose between components? Simply choose the one with the other non-sonic characteristics (features, price, terms, availability, cosmetics, style...) that suit your fancy. That is the 64,000 dollar if. Tom said Examination of the extant body of controlled listening tests available contain enough information to aid any enthusiast in making good decisions. Even IF the existing evidence shows that wire is wire (and it does) how does that preclude any person from making any purchase decision? In my way of thinking it just might be useful for a given individual to know what has gone before (and what hasn't.) Well, so far I don't see it the way you do. I must at this point thanl you for the articles on this subject you sent me when I asked for the alleged body of empirical evidence that prooved your position on the audible differences of amplifiers. The "body of evidence" you sent me that constituted actual evidence, raw data, was not much of a body. Only two articles out of the six you sent had raw data ( "Can you trust your ears" by Tom Nousiane and "Do all amplifiers sound the same" by David Clark) and only the test you conducted had it in a usful table which could allow for the examination of trends such as learning curves or fatigue curves. First, this is not much of a body of evidence. Second, if we are to draw conclusions from the results we would have to conclude that some people can hear differences between amps and some amps sound idfferent than some other amps. Of course it would be a mistake to draw conclusions from those tests by themselves because they simply are not that conclusive. If what you sent me is the best evidence out there and if what you sent me is any significant portion of the much talked about "extant body of controled listening tests available" then I don't see how anyone can draw any strong conclusions one way or another. Tom said So IMO, a person truly interested in maximizing the sonic-quality throughput of his system simply MUST examine the results of bias controlled listening tests OR fall prey to non-sonic biasing factors, even if they are inadvertent. I examined the results of contained in the articles you sent me and do not find them conclusive. Unfortunately four of the six articles you sent me had no raw data to examine and only offered conclusions. Given the fact that the two articles that did offer raw data drew conclusions that I find questionable i have trouble feeling condifent about the conclusions drawn in the other articles missing the raw data. So I find the evidence to date that I have seen less than helpful in purchase decisions. |
#10
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
(Nousaine) wrote in message ...
See his full text below: On 1 Jul 2003 15:10:59 GMT, (ludovic mirabel) wrote: _ I can't possibly answer separately and in detail all those disagreeing with me I have to live whatever remains to me of life.In particular I'll not get in between Mrs. Miyaguchi and Nousaine in the subargument of their own. So 4 interactive complementary answers for economy. First let's define what I am NOT saying. (No apologies for the capitals. Google doesn't allow italics or underlining) I'm NOT saying that the local variant of DBTs known as ABX is the wrong tool to use in research, with careful design,proper statistics, selected research topics and last but not least SELECTED, TRAINED SUBJECTS. I have no knowledge, opinion, interest in the particulars of such research because it does not research differences between real-life audio components. I'm NOT saying that those audiophiles who enjoy ABX and believe that they get valid results should not use it. As long as they do not think that their results are valid for anyone but themselves. ALSO as long as they keep in mind that their negative, null results are valid for them at this particular stage of their ABX training and musical experience. As long as they remember that if they do not revisit their negative results they may be shutting themselves off FOREVER from enlarging and enriching their hi-fi scope. I'm NOT , emphatically NOT saying that individuals shouldn't use methods of their choice to disguise the brand of components that they compare. I have one such method, myself which serves me well but may not suit others.. What I do object to is the belief that to "prove" your opinions re "difference" or "no difference"- a necessary preliminary to preference- (more about difference/preference in my answer to Audioguy.) one has to undergo a procedure known as ABX. Please, don't one of you tell me that "nobody says that". Speak for yourself. Every week somebody says just that in RAHE. Sometimes with a pseudo-objective semantical word games.: "But you have not had a "controlled test" (meaning ABX of course) so it is only your impression..".- as though it could be anything else. Or "Reviewers should undergo cotrolled test for credibility" as though it msde stupid reviewers into clever ones. And of course you Mr. Nousaine said it many times. There are a few unspoken assumptions here FIRST UNPROVEN ASSUMPTION: everyone performs just as well when ABXing and when listening for pleasure blinded or not blinded. Perhaps they do, perhaps they don't. In fact I presented evidence suggesting that there is MARKED DIFFERENCE in people's performance when listening to simple pink noise signal as opposed to listening to a musical signal I then asked Mr Miyaguchi a simple, straightforward question Is there a difference? Instead of of a simple YES= there is a difference (or NO- there isn't any- if anyone really wanted to play the fool) I got convoluted explanations of WHY there is , further beautified by you Mr. Nousaine. Incidentally I asked you the very same question one month ago and never got an answer. Why this dodging?. You can bet I'd get plain NO if there were NO DIFFERENCE. and EVERYBODY who performed flawlessly listening to pink noise would do just as well listening to music. But MOST performed abominably, (consistent with random guessing) when listening to music.. Explanation? Music is "more complex". Quite! And what else is new?.Most of us use our components to listen to music, not pink noise. If your test has problems with music is it the right one to assess the differences in MUSICAL performance of components? Where is the evidence ? Evidence where I come from ( see my answer to Audioguy concerning that) is EXPERIMENTAL evidence not circular arguments like : "Why shouldn't it be so? It is good enough for codecs isn't it?" I listen to music not codecs. And I need convincing that pink noise is the best way to test a component for its transient response to a cymbal, or rendition of a violin or a cello. I'd suggest to researchers:"Try again" All you can name since 1990 Mr. Nousaine are your own cable tests from 1995. Where I come from (again-sorry1) one gives the mag's name, the date, the page so that I can find out what was the design, who proctored, how many subjects, how many tests etc. Why so shy with details? You say: "Likewise the body of controlled listening test results can be very useful to any individual that wishes to make use of them to guide decisions (re component choice)" Where does one find that body? Buried in the pyramids or Alberta tar sands? Why not name a few ,recent representative offshoots?. Just to show that Consumer ABXing is any use other than for discomfiting the naive. Where are the current tests on current components?. You're not serious saying that there's nothing new under the sun. Any difference between the 24 bit and 16 bit cdplayers? Which dacs truly cope with jitter?, Are the Meridian and Tact room equalising systems different from any other equaliser? Any differences between the various digital sound processing systems. Come on Mr. Nousaine. Winning verbal contest against L. Mirabel is not everything. There is such a thing as putting more information on the table. Of course you are at least consistent, You truly believe that there is no difference between any components at all: By your lights such objectivist luminaries as Krueger and Pinkerton are pinkish dissidents: " Actually if nominally competent components such as wires, parts, bits and amplifiers have never been shown to materially affect the sound of reproduced music in normally reverberant conditions why would ANYONE need to conduct more experimentation, or any listening test, to choose between components?" And again. The profession of faith: " Examination of the extant body of controlled listening tests available contain enough information to aid any enthusiast in making good decisions." And what information is it?: " As I've said before; there are many proponents of high-end sound of wires, amps and parts ... but, so far, no one (in over 30 years) has ever produced a single repeatable bias controlled experiment that shows that nominally competent products in a normally reverberant environment (listening room) have any sonic contribution of their own. Nobody! Never! How about some evidence? I'll believe in BigFoot ....just show me the body! And I'll believe that there are no diferences between components when you prove that your "bias =controlled test" does not have biases of its own My other unanswered question to you one month ago was: Where is the evidence that untrained, unselected individuals perform identically, when comparing complex musical differences between components for a "test" as they do when they just listen. Reasoning that they should is plausible but reasoning that at least some of them don't is not irrational. Convincing, controlled experiment with random control subjects etc. is missing. Next consider the anti-common sense assumption that Tom and Dick do their DBT assignement equally well and both are an identical match for Harry. Should they? They are not identical in any other task aptitude, in their fingerprints or their DNA. If you agree that they would differ how do you justify YOUR challenges to all and sundry to prove their perceptions by ABX. Perhaps they are as hopeless as I'm at that task. Perhaps a violinist will hear differences in the rendition of violin tone when not bothered by a "test" but be a terrible subject for ABXing. Impossible? Where is your experimental evidence that this does not happen?. Where is the experimentation to show that the poor ABX test subjects would perform identically sitting and listening at home? Finally your telling ME that " I think Ludovic is "untrainable" because he will accept only answers he already believes are true" really takes the cake. .I'm not propunding any "test". You are. I have no faith to stick to. You BELIEVE in ABX. It is MY right to ask YOU for evidence. And it is your job to give it. You know it all perfectly well because you know what "research" and "evidence" mean. Why copy the tactics of those who have only ignorant bluster to offer? Ludovic Mirabel We're talking about Greenhill old test not because it is perfect but because no better COMPONENT COMPARISON tests are available. In fact none have been published according to MTry's and Ramplemann's bibliographies since 1990. This is simply not true. I have published 2 double blind tests personally, one of which covered 3 different wires subsequent to 1990. I frequently see the distinction being made between audio components and other audio related things (such as codecs) when it comes to talking about DBT's. What is the reason for this? In my opinion, there are two topics which should not be mixed up: 1) The effectiveness of DBT's for determining whether an audible difference exists 2) The practical usefulness of using DBT's for choosing one audio producer (component or codec) over another. Actually if nominally competent components such as wires, parts, bits and amplifiers have never been shown to materially affect the sound of reproduced music in normally reverberant conditions why would ANYONE need to conduct more experimentation, or any listening test, to choose between components? Simply choose the one with the other non-sonic characteristics (features, price, terms, availability, cosmetics, style...) that suit your fancy. Indeed 20 years ago, when I still had a day job, Radio Shack often had the "perfect" characteristic to guide purchase, which was "open on Sunday." I am not knowledgeable enough to decide on differences between your and Greenhill's interpretation of the methods and results. In my simplistic way I'd ask you to consider the following: PINK NOISE signal: 10 out of 11 participants got the maximum possible correct answers: 15 out of 15 ie. 100%. ONE was 1 guess short. He got only 14 out of 15. When MUSIC was used as a signal 1 (ONE) listener got 15 corrects, 1 got 14 and one 12. The others had results ranging from 7 and 8 through 10 to (1) 11. My question is: was there are ANY significant difference between those two sets of results? Is there a *possibility* that music disagrees with ABX or ABX with music? No: it just means with the right set of music 2 dB is at the threshold. Don't forget that listening position affects this stuff too. Also Mr Atkinson would say that perhaps the lower scoring subjects didn't have personal control of the switching. Even between two samples of music (no pink noise involved), I can certainly believe that a listening panel might have more or less difficulty in determining if they hear an audible difference. It doesn't follow that music in general is interfering with the ability to discriminate differences when using a DBT. Actually it simp,y shows that pink noise and other test signals are the most sensitive of programs. It may be possible to divulge a 'difference' with noise that would never be encountered with any known program material. It's also possible that certain programs, such as Arny Kreuger's special signals, might disclose differences that may never be encountered with commercially available music (or other) programs. So? I would aoppreciate it if would try and make it simple leaving "confidence levels" and such out of it. You're talking to ordinary audiophiles wanting to hear if your test will help them decide what COMPONENTS to buy. As before; you haven't ever been precluded from making any purchase decisions from scientific evidence before; why should any disclosure affect that now or in the future. Examination of the extant body of controlled listening tests available contain enough information to aid any enthusiast in making good decisions. Even IF the existing evidence shows that wire is wire (and it does) how does that preclude any person from making any purchase decision? In my way of thinking it just might be useful for a given individual to know what has gone before (and what hasn't.) I still don't know how this cannot do anything but IMPROVE decision making? See my first comments. It's too easy to mix up the topic of the sensitivity of DBT's as instruments for detecting audible differences with the topic of the practicality of using DBT's to choose hifi hardware. The latter is impractical for the average audiophile. No it's not. Just like 0-60 times, skid-pad and EPA mileage tests simply cannot be made by the typical individual that doesn't mean that they cannot be used to improve decision-making. Likewise the body of controlled listening test results can be very useful to any individual that wishes to make use of them to guide decisions. Otherwise the only information one has is "guidance" from sellers, anecdotal reports and "open" listening tests. The latter , of course, is quite subject to non-sonic influence. So IMO, a person truly interested in maximizing the sonic-quality throughput of his system simply MUST examine the results of bias controlled listening tests OR fall prey to non-sonic biasing factors, even if they are inadvertent. Who can argue with motherhood? The problem is that there are NO ABX COMPONENT tests being published- neither better nor worse, NONE. I heard of several audio societies considering them. No results. Not from the objectivist citadels: Detroit and Boston. Why?. Did they pan out? Given the two dozen controlled listening tests of power amplifiers published through 1991 doesn't it seem that no one needs to conduct more? Wires? The last test I published was in 1995. Not late enough? Why not? No manufacturer has EVER produced a single bias controlled experiment that showed their wires had a sound of their own in over 30 years. Why should one expect one now? I certainly can't do it; although I've given it my level (no pun intended) best. IOW, I can't produce an experiment that shows nominally competent wires ain't wires .... 'cuz they ain't. I can think of a couple of reasons: 1. It's expensive and time consuming to perfom this type of testing 2. The audible differences are, in actuality, too subtle to hear, ABX or not. Why bother with such a test? Why bother in performing a sound quality "test" that the manufacturers of the equipment can't produce? IF amps ain't amps; wires ain't wires and parts ain't parts then why haven't the makers and sellers of this stuff produced repeatable bias controlled listening tests that show this to be untrue? Then there is the possibility that you seem to be focussing on, ignoring the above two: 3. DBT's in general may be decreasing the ability to hear subtle differences. Actually they preclude the ability to "hear" non-sonic differences. Which of the the above reaons do you think are most likely? Moving away from the question Greenhill was investigating (audible differences between cables) and focusing only on DBT testing and volume differences: it is trivial to perform a test of volume difference, if the contention is being made that a DBT hinders the listener from detecting 1.75 dB of volume difference. Especially if the listeners have been trained specifically for detecting volume differences prior to the test. However, such an experiment would be exceedingly uninteresting, and I have doubts it would sway the opinion of anybody participating in this debate. The volume difference was just a by-effect of a comparison between cables. And yes, TRAINED people would do better than Greenhill's "Expert audiophiles" ie rank amateurs just like us. Would some though do better than the others and some remain untrainable? Just like us. I think Ludovic is "untrainable" because he will accept only answers he already believes are true. I have no doubt that there are some people who are unreliable when it comes to performing a DBT test. In a codec test using ABC/HR, if somebody rates the hidden reference worse than the revealed reference (both references are identical), his listening opinion is either weighted less or thrown out altogether. What you are describing is 'reverse significance' which is typically a inadvertant form of internal bias. For what it's worth, I have performed enough ABX testing to convince myself that it's possible for me to detect volume differences 0.5 dB using music, so I doubt very highly that a group test would fail to show that 1.75 dB differences on a variety of different music are not audible using a DBT. I can easily hear 1db difference between channels, and a change of 1 db. What I can't do is to have 80 db changed to 81 db, then be asked if the third unknown is 80 or 81 dbs. and be consistently correct. Perhaps I could if I trained as much as you have done. Perhaps not Some others could, some couldn't. We're all different. Produce a test which will be valid for all ages, genders, extent of training, innate musical and ABxing abilities, all kinds of musical experience and preference. Then prove BY EXPERIMENT that it works for COMPARING COMPONENTS. So that anyone can do it and if he gets a null result BE CERTAIN that with more training or different musical experience he would not hear what he did not hear before. And perhaps just get on widening his musical experience and then rcompare (with his eyes covered if he is marketing susceptible) Let's keep it simple. We're audiophiles here. We're talking about MUSICAL REPRODUCTION DIFFERENCES between AUDIO COMPONENTS. I looked at your internet graphs. They mean zero to me. I know M. Levinsohn, Quad, Apogee, Acoustat not the names of your codecs. You assure me that they are relevant. Perhaps. Let's see BY EXPERIMENT if they do. In the meantime enjoy your lab work. Ludovic Mirabel Are you really telling me that you didn't understand the gist of the group listening test I pointed you to? For one thing, it says that although people have different individual preferences about how they evaluate codec quality, as a group, they can identify trends. This, despite the variety of training, hearing acuity, audio equipment, and listening environment. Another point is that it would be more difficult to identify trends if such a study included the opinions of people who judge the hidden reference to be worse than the revealed reference (simultaneously judging the encoded signal to be the same as the revealed reference). In other words, there are people whose listening opinions can't be trusted, and the DBT is designed to identify them. That result identifies a form of experimental bias, does it not? The last point is that I can see no reason why such procedures could not (in theory, if perhaps not in practical terms) be applied to audio components. Why don't you explain to me what the difference is (in terms of sensitivity) between using DBT's for audio codecs and using DBT's for audio components? Darryl Miyaguchi There is no difference. It seems to me that this poster may have never taken a bias controlled listening test or, if he has, the results didn't fit with prior held expectations. It's much easier to argue with the existing evidence than prove that you can hear things that no human has been able to demonstrate, when not peeking. As I've said before; there are many proponents of high-end sound of wires, amps and parts ... but, so far, no one (in over 30 years) has ever produced a single repeatable bias controlled experiment that shows that nominally competent products in a normally reverberant environment (listening room) have any sonic contribution of their own. Nobody! Never! How about some evidence? I'll believe in BigFoot ....just show me the body! |
#11
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
Darryl Miyaguchi wrote in message ...
On 1 Jul 2003 15:10:59 GMT, (ludovic mirabel) wrote: We're talking about Greenhill old test not because it is perfect but because no better COMPONENT COMPARISON tests are available. In fact none have been published according to MTry's and Ramplemann's bibliographies since 1990. I frequently see the distinction being made between audio components and other audio related things (such as codecs) when it comes to talking about DBT's. What is the reason for this? Desperate grasping at straws. True believers need some way to explain away the mountains of research that have been done on human hearing perception, so they try to pretend that comparing the sound of consumer audio components is somehow unrelated to human hearing perception. bob |
#12
|
|||
|
|||
Why DBTs in audio do not deliver
Darryl Miyaguchi wrote:
Let's keep it simple. We're audiophiles here. We're talking about MUSICAL REPRODUCTION DIFFERENCES between AUDIO COMPONENTS. I looked at your internet graphs. They mean zero to me. I know M. Levinsohn, Quad, Apogee, Acoustat not the names of your codecs. You assure me that they are relevant. Perhaps. Let's see BY EXPERIMENT if they do. In the meantime enjoy your lab work. Ludovic Mirabel Are you really telling me that you didn't understand the gist of the group listening test I pointed you to? Whether he realizes it or not, he's telling you *he* doesn't comprehend them. -- -S. |
#13
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
Darryl Miyaguchi wrote in message ...
On 1 Jul 2003 15:10:59 GMT, (ludovic mirabel) wrote: Apologies for rearranging your text for my convenience. You say: In my opinion, there are two topics which should not be mixed up: 1) The effectiveness of DBT's for determining whether an audible difference exists 2) The practical usefulness of using DBT's for choosing one audio producer (component or codec) over another. Let me be quite brutal about this. My ONLY concern and interest is in part 2. I'm reading and participating in RAHE not because I'm interested in psychometric research but exactly for help to "choose one audio producer over another". And the "producer" that I'll use in my listening room is not codec but a musical reproducing device. I may just as well tackle the question of codec or any other artefact/ vs. a musical audio device. Are they different? I don't know. You challenge me below to demonstrate why they shouldn't behave identically. The shoe is on the other foot- you have first to show that they would. I'll have to explain something he Basically I come from a different corner of science from yours. Mine is applied science not basic research. You ask me why codex shouldn't act like amplifiers. Maybe they do. But till there is a convincing experiment to show that it is so, to me it is just more inference, reasoning by analogy. For millenia physicians reasoned, speculated and looked at analogies. The diseased blood is full of noxious miasmas so let's bleed poor Lord Byron to death. Sometime in the XXth century things changed. The question to ask became not: "Is it likely to work because Herr Professor thinks it should or because there are "good reasons" why it should? or what not. It became: "Can I design an experiment to show if it will or will not WORK ?" Reasoning and speculation be damned in medical research- coarse practicality culminating in Random controlled Double Blind testing rules. Patient's answers, doctor's impressions are collected for documentation- the outcome is decided by demonstrable physical changes. All that I said before, But now I come to Codex vs Amplifier. No matter how close the analogy you make it means nothing in the applied medical research. Add one hydrogen binding to a life saving drug and it becomes a killer. Of course the Rhinoceros are being exterminated because the upright horn is a cure for impotence in China. An unfair (I confess) reductio ad absurdum of reasoning by analogy: " Why shouldn't the horn work? It looks like IT doesn't it?" I said: I am not knowledgeable enough to decide on differences between your and Greenhill's interpretation of the methods and results. In my simplistic way I'd ask you to consider the following: PINK NOISE signal: 10 out of 11 participants got the maximum possible correct answers: 15 out of 15 ie. 100%. ONE was short of 100%t. He got only 14 out of 15 But when MUSIC was used as a signal 1 (ONE) listener got 15 corrects, one got 14 and one 12. The others had results ranging from 7 and 8 through 10 to 11.(one) My question is: was there are ANY significant difference between those two sets of results? Is there a *possibility* that music disagrees with ABX or ABX with music? You answered: Even between two samples of music (no pink noise involved), I can certainly believe that a listening panel might have more or less difficulty in determining if they hear an audible difference. It doesn't follow that music in general is interfering with the ability to discriminate differences when using a DBT. Sorry, but the questiion is a simple one :"Did THIS panel perform differently on pink noise and music or not?" And the answer should be simple: Yes or No. Once you answer that you can give your qualifiers, explanations and so on.. You said: It's too easy to mix up the topic of the sensitivity of DBT's as instruments for detecting audible differences with the topic of the practicality of using DBT's to choose hifi hardware. The latter is impractical for the average audiophile. As I said before the only topic I'm interested in is exactly the "practicality" of its use for comparing components. Were I interested in DBT sensitivity for other *audible differences* I'd be reading JAES or a "Journal of Psychometrics" (if there is such a thing) -not RAHE. But your last sentence certainly rings true- that's why your getting in hot water with the Old Believers is to be expected. You said that good ABX component tests are possible (paraphrase). I answered: Who can argue with motherhood? The problem is that there are NO ABX COMPONENT tests being published- neither good nor bad, NONE. I heard of several audio societies considering them. No results. Not from the objectivist citadels: Detroit and Boston. Why?. Did they not pan out? Your answer: I can think of a couple of reasons: 1. It's expensive and time consuming to perfom this type of testing 2. The audible differences are, in actuality, too subtle to hear, ABX or not. Why bother with such a test? Then there is the possibility that you seem to be focussing on, ignoring the above two: 3. DBT's in general may be decreasing the ability to hear subtle differences. Which of the the above reaons do you think are most likely? If you ask me : the last one. Enough changed in audio since 1990 to spur newer comparisons. (See my posting to Mr. Audio Guy). It is expensive but not beyond the possibilities of such as Boston Audio Socy. I saw the design of a Seattlee AUDIO Socy AbX test. Then silence. No results. Was it your reason 3.? I don't know and I never , never speculate. Especially since the stock market expired. If it works why don't people do it? You asked: Are you really telling me that you didn't understand the gist of the group listening test I pointed you to? For one thing, it says that although people have different individual preferences about how they evaluate codec quality, as a group, they can identify trends. This, despite the variety of training, hearing acuity, audio equipment, and listening environment. Another point is that it would be more difficult to identify trends if such a study included the opinions of people who judge the hidden reference to be worse than the revealed reference (simultaneously judging the encoded signal to be the same as the revealed reference). In other words, there are people whose listening opinions can't be trusted, and the DBT is designed to identify them. Re "trends and statistical averages:" How "likely" am I to hear those musical differences under ABX that I heard without it? As likely as the 72% of Greenhill's subjects who failed at a much simpler task when being ABXed?. Or do you have any other experimentally proven statistics? Let me say something about statistics as applied to prognosis ( outcome forecasting) in medicine. A patient has inoperable lung cancer. His family want to know how long he'll live. If you're a heartless fool you say: "average survival with this is 6 months." If you're a clever and humane physician you say:" Whatever I'll say you'll probably want to check in the Public Library anyway- you'll find a 6 months AVERAGE survival rate . But you husband's name is Joe Smith not Joe Average. Some die in a few weeks, some in a couple of years and some-very, very few have an unexplainable, complete disappearance of the growth. I can not tell exactly how long your husband will live, but more likely months than years." A "test" which depends on statistics can not be used as a universally recommended method of differentiation. It may be O.K. for some and worse than useless for the others. And the devil is that no one can tell if he is being deceived by it when he gets a null result because he may perform differently in 6 months time or when not bothered by ABX. Or not. Do it at your own risk. Like choosing a book to read, a movie to attend or wine to drink. Nobody pesters you with a" test" there. Sorry, "trends" are good for public opinion polls, not as guidance for inndividuals. The last point is that I can see no reason why such procedures could not (in theory, if perhaps not in practical terms) be applied to audio components. Why don't you explain to me what the difference is (in terms of sensitivity) between using DBT's for audio codecs and using DBT's for audio components? I will not repeat why I consider the above an example of reasoning by similarity and analogy without experimental evidence. And THAT YOU fail to supply. Once you do that I'll see if the experiments were well designed, properly carried out, had good controls and so on. Ludovic Mirabel |
#14
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
On Fri, 04 Jul 2003 17:40:00 GMT, (ludovic
mirabel) wrote: Darryl Miyaguchi wrote in message ... On 1 Jul 2003 15:10:59 GMT, (ludovic mirabel) wrote: Apologies for rearranging your text for my convenience. You say: In my opinion, there are two topics which should not be mixed up: 1) The effectiveness of DBT's for determining whether an audible difference exists 2) The practical usefulness of using DBT's for choosing one audio producer (component or codec) over another. Let me be quite brutal about this. My ONLY concern and interest is in part 2. I'm reading and participating in RAHE not because I'm interested in psychometric research but exactly for help to "choose one audio producer over another". And the "producer" that I'll use in my listening room is not codec but a musical reproducing device. Your claim is that DBT's reduce the ability to discriminate differences in music using audio components. This is a very specific claim which relies upon several assumptions: 1. DBT's reduce, in general, the ability to discriminate differences. Some evidence to the contrary: ABX has been successfully used to differentiate truncation vs. dithering at 16 bits: http://ff123.net/24bit/24bitanalysis.html ABX has been successfully used to discriminate volume differences in music of less than 0.5 dB (personal tests). 2. ABX may be ok for pink noise, but not for music. A controlled test (Greenhill's) showed decreased listener sensitivity when choral music was presented instead of pink noise. From this you infer that ABX is not suited for music. Again, I must point out that this inference is flawed. The simpler and more likely explanation is that all types of listening methods (included sighted listening) are affected by musical selection. 3. DBT's may be ok for audio codecs, but not for comparing audio components. We seem to disagree on this basic point, although I will point out that you can hardly claim that the onus is on me to provide evidence that the two situations are similar. If I assume a certain position (that the human ear/brain behaves similarly, according to the same psychoacoustic descriptions, regardless of the audio source), then your position is surely an assumption as well, and IMO more speculative. I choose the null hypothesis (there is no difference) until I see evidence to the contrary. Sorry, but the questiion is a simple one :"Did THIS panel perform differently on pink noise and music or not?" And the answer should be simple: Yes or No. Once you answer that you can give your qualifiers, explanations and so on.. See my separate post to this. cut 3. DBT's in general may be decreasing the ability to hear subtle differences. Which of the the above reaons do you think are most likely? If you ask me : the last one. Enough changed in audio since 1990 to spur newer comparisons. (See my posting to Mr. Audio Guy). It is expensive but not beyond the possibilities of such as Boston Audio Socy. I saw the design of a Seattlee AUDIO Socy AbX test. Then silence. No results. Was it your reason 3.? I don't know and I never , never speculate. The irony of this statement must have escaped you. As far as I can tell, your position *is* speculative, given that it is based on very specific assumptions (see above). Especially since the stock market expired. If it works why don't people do it? cut A "test" which depends on statistics can not be used as a universally recommended method of differentiation. It may be O.K. for some and worse than useless for the others. And the devil is that no one can tell if he is being deceived by it when he gets a null result because he may perform differently in 6 months time or when not bothered by ABX. Or not. Do it at your own risk. Like choosing a book to read, a movie to attend or wine to drink. Nobody pesters you with a" test" there. Sorry, "trends" are good for public opinion polls, not as guidance for inndividuals. Hearing perceptions are more similar from person to person than their preferences in books or movies or taste in wine. If they weren't, people wouldn't have been able to design audio codecs, which rely upon universal characteristics of human hearing. However, it is true that people have varying ability to hear certain things, and that this variation affects their preferences. There are two answers to the question "Which sounds best?" One answer is the one each individual gives after personal audition. The other answer is based on group results. Which answer one should choose is based on the particular circumstance. Darryl Miyaguchi |
#15
|
|||
|
|||
Why DBTs in audio do not deliver (was: Finally ... The Furutech CD-do-something)
I have no "evidence" for my perception that silver wires a la Kimber
sound better- TO ME- than copper (even when I'm blinded). None that would satisfy you and none, in truth- that would satisfy a critical peer-review. MORE- I don't believe that such "evidence" is possible outside of RAHE wishful fantasies. I don't believe that there is experimental evidence (see my answer to Nousaine) that a technique, such as the audio version of DBT has been shown to be capable of invalidating mine or anyone else's perceptions. I have to take issue with this claim. Any claims that suggest a physical manifestation of any phenomenon is a testable claim. If you claim to hear differences it is a testable claim. |
Reply |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
Crazy market saturation! | Car Audio | |||
FAQ: RAM LISTING OF SCAMMERS, SLAMMERS, AND N'EER DO WELLS! V. 8.1 | Audio Opinions | |||
A quick study in very recent RAHE moderator inconsistency | Audio Opinions | |||
System balance for LP? | Audio Opinions | |||
gps install: how to mix its audio (voice prompting) with head unit audio-out? | Car Audio |