Home |
Search |
Today's Posts |
#1
|
|||
|
|||
Validity of audio tests
I have the following worry about audio listening tests. Suppose the
meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. I am thinking by way of contrast to visual examples. I just made two prints of a photograph using different settings on my printer. I am looking at the face of the subject and I can see that the contrast is higher in one than in the other. That is a Gestalt property of a meaningful chunk of the picture, not a property of a few pixels (cf. notes). The difference with the musical case is that I can compare the contrast of the two pictures directly, whereas in music no immediate comparison is possible. At best I have to keep the property in memory, and maybe the relevant variable is something not easily retained. Is the existing empirical confirmation for tests recommended in audio based largely on visual data? If so, perhaps they rely on factors that apply to the visual domain (i.e., possibility of immediate comparison) but do not transfer easily to audio. Or are there cases in the scientific literature in which the relevant kinds of tests have been found valid to measure the detection of Gestalt properties of aural, temporally extended signals? |
#2
|
|||
|
|||
Mark DeBellis wrote:
I have the following worry about audio listening tests. Suppose the meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. Then it's not audible. End of discussion. bob |
#3
|
|||
|
|||
Mark DeBellis wrote:
I have the following worry about audio listening tests. Suppose the meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. Nobody uses short snippets to do audio listening tests or comparisons. We usually use a complete song or passage of an extended work. I am thinking by way of contrast to visual examples. I just made two prints of a photograph using different settings on my printer. I am looking at the face of the subject and I can see that the contrast is higher in one than in the other. That is a Gestalt property of a meaningful chunk of the picture, not a property of a few pixels (cf. notes). The difference with the musical case is that I can compare the contrast of the two pictures directly, whereas in music no immediate comparison is possible. At best I have to keep the property in memory, and maybe the relevant variable is something not easily retained. Not true. You can perform rapid switching as often and as many as you want. You level match the two sources so that the only difference you hear is the sound quality differences between the two. Rapid switching is the audio equivalent of a direct comparison. Your complaint would apply only to long term comparisons, where you listen first to a complete song on one source, then switch to the other source and listen all over again. Gary Eickmeier |
#4
|
|||
|
|||
Gary Eickmeier wrote:
Mark DeBellis wrote: I have the following worry about audio listening tests. Suppose the meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. Nobody uses short snippets to do audio listening tests or comparisons. We usually use a complete song or passage of an extended work. I am thinking by way of contrast to visual examples. I just made two prints of a photograph using different settings on my printer. I am looking at the face of the subject and I can see that the contrast is higher in one than in the other. That is a Gestalt property of a meaningful chunk of the picture, not a property of a few pixels (cf. notes). The difference with the musical case is that I can compare the contrast of the two pictures directly, whereas in music no immediate comparison is possible. At best I have to keep the property in memory, and maybe the relevant variable is something not easily retained. Not true. You can perform rapid switching as often and as many as you want. You level match the two sources so that the only difference you hear is the sound quality differences between the two. Rapid switching is the audio equivalent of a direct comparison. Your complaint would apply only to long term comparisons, where you listen first to a complete song on one source, then switch to the other source and listen all over again. Gary Eickmeier I second Garys comment. Some additions: There are ABX switchboxes available, I made one myself with relays to switch between two speaker cables simultaneously on both ends. You can switch any time as much as you like. The music or test signal (I use mostly pink noise) is completely at your disposition, use what you feel gives the best results. The faster and cleaner the switching action, the more subtle differences can be discovered. If you are not into electronics it is better to get a ready made box, to avoid any difference between the channels, like the sound of the relais being different making/breaking or so. -- ciao Ban Bordighera, Italy |
#5
|
|||
|
|||
On 16 Jun 2005 03:14:56 GMT, Gary Eickmeier
wrote: Mark DeBellis wrote: I have the following worry about audio listening tests. Suppose the meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. Nobody uses short snippets to do audio listening tests or comparisons. We usually use a complete song or passage of an extended work. I am thinking by way of contrast to visual examples. I just made two prints of a photograph using different settings on my printer. I am looking at the face of the subject and I can see that the contrast is higher in one than in the other. That is a Gestalt property of a meaningful chunk of the picture, not a property of a few pixels (cf. notes). The difference with the musical case is that I can compare the contrast of the two pictures directly, whereas in music no immediate comparison is possible. At best I have to keep the property in memory, and maybe the relevant variable is something not easily retained. Not true. You can perform rapid switching as often and as many as you want. OK but I am saying, suppose the meaningful variable is a property of an extended passage. So you have to listen to an uninterrupted passage in order to perceive the property. Switching back and forth will defeat the purpose, yes? Mark |
#6
|
|||
|
|||
|
#7
|
|||
|
|||
Mark DeBellis wrote:
On 16 Jun 2005 03:14:23 GMT, wrote: Mark DeBellis wrote: I have the following worry about audio listening tests. Suppose the meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. Then it's not audible. End of discussion. If you hear something but do not retain a memory of it (sufficient to carry out a certain kind of test), you still heard it. No? In a sense, I suppose, but in that case you can't carry out any kind of a comparison at all. So how could you be conscious of it under any conditions? And if you can't be conscious of it under any conditions, how can you say that you "heard" it? Though I suspect you meant something slightly different, based on another post, so I'll respond to that one. bob |
#8
|
|||
|
|||
Mark DeBellis wrote:
OK but I am saying, suppose the meaningful variable is a property of an extended passage. So you have to listen to an uninterrupted passage in order to perceive the property. Switching back and forth will defeat the purpose, yes? Yeah, but. First, the problem, if it were a problem, could easily be solved by listening to longer passages. No one's ever heard differences between competent amps/cables doing it that way, either. Second, the research demonstrates pretty clearly that our memory for subtle sonic differences is very limited. In other words, contrary to your conjecture, switching back and forth quickly and frequently really is more effective. bob |
#9
|
|||
|
|||
Mark DeBellis wrote:
OK but I am saying, suppose the meaningful variable is a property of an extended passage. So you have to listen to an uninterrupted passage in order to perceive the property. Switching back and forth will defeat the purpose, yes? I would say this is a false premise. But perhaps you could give an example of a meaningful variable that is a property of a longer passage. Gary Eickmeier |
#10
|
|||
|
|||
On 16 Jun 2005 23:53:16 GMT, "Ban" wrote:
Gary Eickmeier wrote: Mark DeBellis wrote: I have the following worry about audio listening tests. Suppose the meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. I second Garys comment. Some additions: There are ABX switchboxes available, I made one myself with relays to switch between two speaker cables simultaneously on both ends. You can switch any time as much as you like. Sorry I guess I didn't explain my idea very well. Suppose that in order to perceive the relevant property a listener has to hear an uninterrupted stretch of music from the same source. That is, suppose the relevant property is not a property that belongs to any short snippet of the signal but is rather a property that belongs only to a whole, longer passage, say 5 mins. in length or a whole movement. What I am thinking of here is the SACD vs. CD issue discussed on another thread. I am wondering if the unit over which perception can differ meaningfully can be an extended passage not a brief interval; if so, my switching back and forth between SACD and CD would not be a relevant test, because I would hear neither SACD nor CD as an unbroken extended passage. I guess I am asking basically whether the existing protocols for audio tests make room for the possibility that there can be auditory perception of properties of longer, extended passages, and are sufficient to measure such perception. Perhaps the answer would be that there could not be a difference in perceptible properties of longer passages without a detectable difference in frequency response, which could be heard in quick-switch tests; but is that obvious? Mark |
#11
|
|||
|
|||
Mark DeBellis wrote:
On 16 Jun 2005 03:14:23 GMT, wrote: If you hear something but do not retain a memory of it (sufficient to carry out a certain kind of test), you still heard it. No? I'm pretty sure I've mentioned this book here befo Daniel Dennett's _Conciousness Explained_. Probably my favorite treatese on the physical process of cognition & perception -- thought-provoking, challenging, elucidating and funny to read! Highly recomended. Anyway, I mention this book because, in a nutshell, according to Dennett, the answer to your question is "No." In a slightly bigger nutshell, Dennett goes on to explain (with far more conviction & evidence than I could possibly muster in a newsgroup posting) that there is often a significant & meaningful difference between What We Perceived, and What We Think We Perceived. |
#12
|
|||
|
|||
Mark DeBellis wrote:
Sorry I guess I didn't explain my idea very well. Suppose that in order to perceive the relevant property a listener has to hear an uninterrupted stretch of music from the same source. That is, suppose the relevant property is not a property that belongs to any short snippet of the signal but is rather a property that belongs only to a whole, longer passage, say 5 mins. in length or a whole movement. What I am thinking of here is the SACD vs. CD issue discussed on another thread. I am wondering if the unit over which perception can differ meaningfully can be an extended passage not a brief interval; if so, my switching back and forth between SACD and CD would not be a relevant test, because I would hear neither SACD nor CD as an unbroken extended passage. I guess I am asking basically whether the existing protocols for audio tests make room for the possibility that there can be auditory perception of properties of longer, extended passages, and are sufficient to measure such perception. Yes. There's nothing that would make a DBT involving full 5-minute samples invalid. However, there's also no reason to think they would work better, as I noted yesterday. (To be completely accurate, the protocols DO require that the subject have the ability to switch any time he wants. But there is nothing that requires him to switch more often than once every 5 minutes if he so chooses.) Perhaps the answer would be that there could not be a difference in perceptible properties of longer passages without a detectable difference in frequency response, which could be heard in quick-switch tests; but is that obvious? Yep. And you're more likely to notice it if you switch quickly and frequently between choices. bob |
#13
|
|||
|
|||
"Mark DeBellis" wrote in message
... On 16 Jun 2005 23:53:16 GMT, "Ban" wrote: Gary Eickmeier wrote: Mark DeBellis wrote: I have the following worry about audio listening tests. Suppose the meaningful variable is a property of an extended passage, not a short snippet. Then a subject's failure to accurately distinguish or re-identify may be due to an inability to retain the property in memory. I second Garys comment. Some additions: There are ABX switchboxes available, I made one myself with relays to switch between two speaker cables simultaneously on both ends. You can switch any time as much as you like. Sorry I guess I didn't explain my idea very well. Suppose that in order to perceive the relevant property a listener has to hear an uninterrupted stretch of music from the same source. That is, suppose the relevant property is not a property that belongs to any short snippet of the signal but is rather a property that belongs only to a whole, longer passage, say 5 mins. in length or a whole movement. What I am thinking of here is the SACD vs. CD issue discussed on another thread. I am wondering if the unit over which perception can differ meaningfully can be an extended passage not a brief interval; if so, my switching back and forth between SACD and CD would not be a relevant test, because I would hear neither SACD nor CD as an unbroken extended passage. I guess I am asking basically whether the existing protocols for audio tests make room for the possibility that there can be auditory perception of properties of longer, extended passages, and are sufficient to measure such perception. Perhaps the answer would be that there could not be a difference in perceptible properties of longer passages without a detectable difference in frequency response, which could be heard in quick-switch tests; but is that obvious? There is a very simple, very powerful way to determine this. But it is not practical or possible for one individual. It is called monadic testing. It requires listening to the segment of music, and rating that musical reproduction *immediately afterwards* using a series of rating criteria. Such criteria might include, for example, a five point scale ranging from: "bass sounded extrememly punchy" to "bass sounded flabby and undynamic". When hundreds of people do this, statistics can be applied to determine if there are in fact perceivable differences, and if so, on what criteria. If I were Harmon Industries, I might design and sponsor such a test on occasion. Frankly, Sony blew an opportunity to do such a test (it would be expensive) for their SACD launch. Imagine if the introductory campaign had included "proof" that SACD sounded better. We'd now have a viable second format. If I were the AES, I might sponsor such a test as a "control test" for single-person tests such as the much bally-hooed ABX test, to advance the state of the art.. But for a given individual it is not a practical test. |
#16
|
|||
|
|||
On 17 Jun 2005 23:48:24 GMT, "Buster Mudd"
wrote: Mark DeBellis wrote: On 16 Jun 2005 03:14:23 GMT, wrote: If you hear something but do not retain a memory of it (sufficient to carry out a certain kind of test), you still heard it. No? I'm pretty sure I've mentioned this book here befo Daniel Dennett's _Conciousness Explained_. Probably my favorite treatese on the physical process of cognition & perception -- thought-provoking, challenging, elucidating and funny to read! Highly recomended. Anyway, I mention this book because, in a nutshell, according to Dennett, the answer to your question is "No." That is an interesting book, and I do remember reading it a while back, and I second your recommendation, but what exactly is the reason to think "No"? There are cases all the time when people perceive things and then forget them. In a slightly bigger nutshell, Dennett goes on to explain (with far more conviction & evidence than I could possibly muster in a newsgroup posting) that there is often a significant & meaningful difference between What We Perceived, and What We Think We Perceived. True, but how does that difference play a role here? |
#17
|
|||
|
|||
|
#18
|
|||
|
|||
Mark DeBellis wrote:
It would not solve the problem because once you had listened to the first passage you would have to remember the property for the duration of the second passage, and I am hypothesizing that you don't have reliable memory for that. That is the problem. OK, so let me get this straight: You listen to one component, hear certain properties, then listen to another component, and hear certain other properties, but by the time it's all over with you can't remember which was which? This pretty much dooms any listening test, doesn't it? Not sure I see the point of your question. Gary Eickmeier |
#19
|
|||
|
|||
Mark DeBellis wrote:
On 17 Jun 2005 03:11:07 GMT, wrote: Mark DeBellis wrote: OK but I am saying, suppose the meaningful variable is a property of an extended passage. So you have to listen to an uninterrupted passage in order to perceive the property. Switching back and forth will defeat the purpose, yes? Yeah, but. First, the problem, if it were a problem, could easily be solved by listening to longer passages. No one's ever heard differences between competent amps/cables doing it that way, either. It would not solve the problem because once you had listened to the first passage you would have to remember the property for the duration of the second passage, and I am hypothesizing that you don't have reliable memory for that. That is the problem. Then how do you know it's a meaningful variable? FWIW, here I am thinking of SACD vs. CD rather than amps or cables. I don't know if it makes a difference, but the intuition is about music not white noise (say). Second, the research demonstrates pretty clearly that our memory for subtle sonic differences is very limited. In other words, contrary to your conjecture, switching back and forth quickly and frequently really is more effective. Is the research that demonstrates this based entirely on the tests that I am saying would not be sensitive to such possibilities? Isn't that a circular argument? If not, what is the relevant research? It's based on tests of human hearing. Your ability to remember partial loudness differences lasts a couple of seconds, tops. You are speculating that there exists something that violates this established fact. What is it, and how do you know? bob |
#20
|
|||
|
|||
Mark DeBellis wrote:
On 18 Jun 2005 02:27:00 GMT, wrote: Yes. There's nothing that would make a DBT involving full 5-minute samples invalid. OK, then please explain to me where I'm going wrong. I am hypothesizing This is where you're going wrong. You are NOT hypothesizing. You are engaging in idle speculation. "There might be something" is not a hypothesis. If you can tell us what you think that something is, and give us some reason to believe it might be a factor, then you have a hypothesis. that there are properties (1) that can only be perceived over long stretches and (2) are not retained in memory. Aren't these two things mutually contradictory? Certainly you are relying on memory when you perceive something over long stretches. And, to repeat myself, what is it? We're still waiting. If there are such properties, the kind of test I'm thinking of won't be sufficient to measure the perception of them, because at the end of the second 5-minute sample, the subject won't remember the first one well enough to make an accurate comparison. A test of this sort will not be sensitive to the phenomenon. Please tell me why the reasons I have given for my conclusion are not good ones. However, there's also no reason to think they would work better, as I noted yesterday. (To be completely accurate, the protocols DO require that the subject have the ability to switch any time he wants. But there is nothing that requires him to switch more often than once every 5 minutes if he so chooses.) Perhaps the answer would be that there could not be a difference in perceptible properties of longer passages without a detectable difference in frequency response, which could be heard in quick-switch tests; but is that obvious? Yep. Well, it's not obvious to me, so if you could give me some indication why I should think it's true, that would be most appreciated! Can you name some sonic distinction that isn't a partial loudness difference? bob |
#21
|
|||
|
|||
On 18 Jun 2005 22:46:39 GMT, Gary Eickmeier
wrote: Mark DeBellis wrote: It would not solve the problem because once you had listened to the first passage you would have to remember the property for the duration of the second passage, and I am hypothesizing that you don't have reliable memory for that. That is the problem. OK, so let me get this straight: You listen to one component, hear certain properties, then listen to another component, and hear certain other properties, but by the time it's all over with you can't remember which was which? This pretty much dooms any listening test, doesn't it? Not sure I see the point of your question. It is that the test would be inadequate to measure the detection of said properties. The background to this is that I said I thought SACD sounded better than CD, and Chung suggested that I try a simple blind test, to see if I could reliably identify which was SACD and which was CD (after matching levels; SACD and CD layers of same disc). Indeed I could not, at least on one set of trials. Should the conclusion be that SACD sounds the same as CD? Or is it possible that the test I applied is inadequate in some way? How *could* it be possible that the test is inadequate? My question is basically an attempt to explain how that might be. Suppose when I am listening to recorded music (1) I hear properties of temporally extended passages and (2) I can't retain a memory of those properties long enough to make a comparison, if at all. Then I could have perceived different things, although this difference would not show up in the kind of test I performed. Mark |
#22
|
|||
|
|||
Mark DeBellis wrote:
On 17 Jun 2005 23:48:24 GMT, "Buster Mudd" wrote: Mark DeBellis wrote: On 16 Jun 2005 03:14:23 GMT, wrote: If you hear something but do not retain a memory of it (sufficient to carry out a certain kind of test), you still heard it. No? I'm pretty sure I've mentioned this book here befo Daniel Dennett's _Conciousness Explained_. Probably my favorite treatese on the physical process of cognition & perception -- thought-provoking, challenging, elucidating and funny to read! Highly recomended. Anyway, I mention this book because, in a nutshell, according to Dennett, the answer to your question is "No." That is an interesting book, and I do remember reading it a while back, and I second your recommendation, but what exactly is the reason to think "No"? Because "hearing" is a cognitive process; it takes place in the brain, not in the ear. So if your brain tells you you didn't hear it, even if soundwaves did strike your eardrum...and even (!) if at an earlier time your brain told you that you did hear it...for all intents & purposes, you didn't hear it. Saying "I heard it" is only useful if you can access the perception in order to make subsequent discriminations. There are cases all the time when people perceive things and then forget them. At which point any information they may have gleaned from perceiving that thing is lost to them. Hence, the distinction between whether they actually perceived it & then forgot it, or never perceived it in the first place, is moot. In a slightly bigger nutshell, Dennett goes on to explain (with far more conviction & evidence than I could possibly muster in a newsgroup posting) that there is often a significant & meaningful difference between What We Perceived, and What We Think We Perceived. True, but how does that difference play a role here? It goes to the core of your initial question: How valid can an audio test be if it's measuring the perception of phenomena which may not actually exist? |
#23
|
|||
|
|||
Mark DeBellis wrote:
On 18 Jun 2005 22:46:39 GMT, Gary Eickmeier wrote: Mark DeBellis wrote: It would not solve the problem because once you had listened to the first passage you would have to remember the property for the duration of the second passage, and I am hypothesizing that you don't have reliable memory for that. That is the problem. OK, so let me get this straight: You listen to one component, hear certain properties, then listen to another component, and hear certain other properties, but by the time it's all over with you can't remember which was which? This pretty much dooms any listening test, doesn't it? Not sure I see the point of your question. It is that the test would be inadequate to measure the detection of said properties. The background to this is that I said I thought SACD sounded better than CD, and Chung suggested that I try a simple blind test, to see if I could reliably identify which was SACD and which was CD (after matching levels; SACD and CD layers of same disc). Indeed I could not, at least on one set of trials. Should the conclusion be that SACD sounds the same as CD? Or is it possible that the test I applied is inadequate in some way? How *could* it be possible that the test is inadequate? My question is basically an attempt to explain how that might be. Suppose when I am listening to recorded music (1) I hear properties of temporally extended passages and (2) I can't retain a memory of those properties long enough to make a comparison, if at all. Then I could have perceived different things, although this difference would not show up in the kind of test I performed. IOW, you did a test, you didn't like the result, so now you're demanding that we give you some basis for rejecting the result of the test. The test is adequate, assuming you did it with reasonable care. That's why scientists have used it to test just about everything we know about hearing. Why should a test become inadequate just because some hobbyist wants to believe something that isn't true? bob |
#24
|
|||
|
|||
wrote in message ...
Mark DeBellis wrote: On 18 Jun 2005 22:46:39 GMT, Gary Eickmeier wrote: Mark DeBellis wrote: It would not solve the problem because once you had listened to the first passage you would have to remember the property for the duration of the second passage, and I am hypothesizing that you don't have reliable memory for that. That is the problem. OK, so let me get this straight: You listen to one component, hear certain properties, then listen to another component, and hear certain other properties, but by the time it's all over with you can't remember which was which? This pretty much dooms any listening test, doesn't it? Not sure I see the point of your question. It is that the test would be inadequate to measure the detection of said properties. The background to this is that I said I thought SACD sounded better than CD, and Chung suggested that I try a simple blind test, to see if I could reliably identify which was SACD and which was CD (after matching levels; SACD and CD layers of same disc). Indeed I could not, at least on one set of trials. Should the conclusion be that SACD sounds the same as CD? Or is it possible that the test I applied is inadequate in some way? How *could* it be possible that the test is inadequate? My question is basically an attempt to explain how that might be. Suppose when I am listening to recorded music (1) I hear properties of temporally extended passages and (2) I can't retain a memory of those properties long enough to make a comparison, if at all. Then I could have perceived different things, although this difference would not show up in the kind of test I performed. IOW, you did a test, you didn't like the result, so now you're demanding that we give you some basis for rejecting the result of the test. The test is adequate, assuming you did it with reasonable care. That's why scientists have used it to test just about everything we know about hearing. Why should a test become inadequate just because some hobbyist wants to believe something that isn't true? Sorry to sound like a broken record, but since you assert this constantly I can only do the same in response....the testing you favor has never been validated for the open-ended evaluation of reproduced music. Period. If it had been and could be demonstrated to have been so, it would be widely used and accepted by most every audiophile. The fact that it has not been, is not accepted, and flies in the face of so much otherwise different consensus means that to continue asserting it as you do, is an act of faith, nothing else. |
#25
|
|||
|
|||
Gary Eickmeier wrote:
Mark DeBellis wrote: OK but I am saying, suppose the meaningful variable is a property of an extended passage. So you have to listen to an uninterrupted passage in order to perceive the property. Switching back and forth will defeat the purpose, yes? I would say this is a false premise. But perhaps you could give an example of a meaningful variable that is a property of a longer passage. I missed your post until now because, for some reason, it didn't download (I'm using Free Agent) from my news server. An example of a property that belongs to a temporally extended passage without belonging to short slices of it is being a descending C major scale, one octave long. That is a property a listener can perceive the passage as having, but it is a property is one that belongs to the whole, not short parts. Another perceivable property that belongs to a temporal whole is the property, belonging to a spoken sentence, of being syntactically well formed. Mark |
#26
|
|||
|
|||
|
#27
|
|||
|
|||
"Buster Mudd" wrote:
Because "hearing" is a cognitive process; it takes place in the brain, not in the ear. So if your brain tells you you didn't hear it, even if soundwaves did strike your eardrum...and even (!) if at an earlier time your brain told you that you did hear it...for all intents & purposes, you didn't hear it. Saying "I heard it" is only useful if you can access the perception in order to make subsequent discriminations. I agree with the basic idea that perception is something that influences behavior, but why does the behavior have to be restricted to comparison and identification? Suppose a listener gives higher approval ratings to one set of (blind) stimuli than another, without ever trying to say which stimuli were the same and which were different. This would be an influence on behavior, but of a weaker sort than is required by the "can you reliably identify" type of test. There are cases all the time when people perceive things and then forget them. At which point any information they may have gleaned from perceiving that thing is lost to them. Hence, the distinction between whether they actually perceived it & then forgot it, or never perceived it in the first place, is moot. Not if the information is still doing work somewhere in your cognitive economy, even though it can't be brought to consciousness, or is not specific enough to enable one to perform the identification task. Mark |
#28
|
|||
|
|||
When I said the other day that audio subjectivism is the rejection of
the scientific method, this is what I meant: Harry Lavo wrote: Sorry to sound like a broken record, but since you assert this constantly I can only do the same in response....the testing you favor has never been validated for the open-ended evaluation of reproduced music. Period. If it had been and could be demonstrated to have been so, it would be widely used and accepted by most every audiophile. The fact that it has not been, is not accepted, and flies in the face of so much otherwise different consensus means that to continue asserting it as you do, is an act of faith, nothing else. To believe this, one must ignore reams of scientific data that conflict with what you believe. In particular, one must assert, against all evidence, that human hearing operates differently when listening to reproduced music than it does at all other times, and that generations of scientists have just been deluding themselves. This is how Creation Science operates. Let's hope the audio field never sinks so low. bob |
#29
|
|||
|
|||
wrote:
Mark DeBellis wrote: On 17 Jun 2005 03:11:07 GMT, wrote: Mark DeBellis wrote: OK but I am saying, suppose the meaningful variable is a property of an extended passage. So you have to listen to an uninterrupted passage in order to perceive the property. Switching back and forth will defeat the purpose, yes? Yeah, but. First, the problem, if it were a problem, could easily be solved by listening to longer passages. No one's ever heard differences between competent amps/cables doing it that way, either. It would not solve the problem because once you had listened to the first passage you would have to remember the property for the duration of the second passage, and I am hypothesizing that you don't have reliable memory for that. That is the problem. Then how do you know it's a meaningful variable? It seems his point is that yo don't know it isn't either. FWIW, here I am thinking of SACD vs. CD rather than amps or cables. I don't know if it makes a difference, but the intuition is about music not white noise (say). Second, the research demonstrates pretty clearly that our memory for subtle sonic differences is very limited. In other words, contrary to your conjecture, switching back and forth quickly and frequently really is more effective. Is the research that demonstrates this based entirely on the tests that I am saying would not be sensitive to such possibilities? Isn't that a circular argument? If not, what is the relevant research? It's based on tests of human hearing. Your ability to remember partial loudness differences lasts a couple of seconds, tops. Now that is interesting given that small barely audible level differences can lead one to form a prefernce. How can that be? You are speculating that there exists something that violates this established fact. What is it, and how do you know? How do you explain the fact that small level diferences can lead to prefeences if we can't remember them in our comparisons? Scott Wheeler |
#30
|
|||
|
|||
Harry Lavo said:
.....the testing you favor has never been validated for the open-ended evaluation of reproduced music. Period. Which evalutaion method that you approve of has been so validated? What did the BBC use when they did their ABX comparisons for the purpose of upgrading their speakers? If it had been and could be demonstrated to have been so, it would be widely used and accepted by most every audiophile. The fact that double blind level matched comparisons are the standard for those doing the serious research on all other forms of sound hasn't seemed to convince audiophiles that it is valid, so, it would seem that many of them simply refuse to accept what is known, probably because it doesn't give the results that they believe they should get. The fact that it has not been, is not accepted, and flies in the face of so much otherwise different consensus means that to continue asserting it as you do, is an act of faith, nothing else. And accepting less reliable, non-bias controlled tests, is not? The problem is not consensus, since clearly audiophiles are in the minority when it comes to ABX and it's reliability. |
#31
|
|||
|
|||
Mark DeBellis wrote:
.... The background to this is that I said I thought SACD sounded better than CD, and Chung suggested that I try a simple blind test, to see if I could reliably identify which was SACD and which was CD (after matching levels; SACD and CD layers of same disc). Indeed I could not, at least on one set of trials. Should the conclusion be that SACD sounds the same as CD? Or is it possible that the test I applied is inadequate in some way? How *could* it be possible that the test is inadequate? My question is basically an attempt to explain how that might be. ... Another possibility is suggested by an informal experiment on smells that I did on myself. I took a selection of spices that were all in the same type of bottle, closed my eyes, shuffled the bottles around, and opened and sniffed them one by one, trying to identify the spice by name. I was sure I could do it but was quite amazed to discover that I couldn't do it at all. With my eyes closed, I could tell the differences perfectly well, but I just couldn't connect the names with the smells. So I think it's *possible* that in unsighted comparisons, the part of the brain that does symbolic analysis and associated judgment is not in perfect communication with other parts. -- Greg Lee |
#32
|
|||
|
|||
A couple of further ideas to toss out, from an onlooker's perspective
of course. Quick-switch tests are said to be the most sensitive, therefore best, yes? Because they permit finer discriminations. An observation: the ultimate purpose of an audio test is not discrimination per se. That is, it's not as if the job that has to be performed here is to discriminate two sources if they can be, to do the best job of discrimination we can, as if that were the real goal. The real purpose of the test is rather to find out what information is available to the listener in the context of use, or perhaps to estimate an upper bound on that information. It is possible that two sources in the ordinary context of use do not present different information to the listener, even if there are ways to set up testing situations (e.g., at higher volume) where a person could discriminate the sources. All well and good if we do, but it's not like having a test that permits such discrimination is a valuable achievement in itself; what we are basically interested in is making sure that if there are differences of information presented in the ordinary context of use, then they will show up and get discriminated in the test. Question: for all the resolving power quick-switch tests have, for all the power they have to put the stimulus under a microscope and discern small differences of detail, are there certain sorts of properties they are *not* so good at picking up? Is there perhaps a forest-for-trees phenomenon lurking somewhere out there? Here's an off the cuff example. Suppose I have two digital photographs that are identical except that one is 1.01 the size of the other. First I compare them (this is the analogue of the quick-switch test) by comparing small portions of one with the other. The comparison is set up in such a way that when I compare a square portion of one with a square portion of the other, one of them is 1.01 as large as the other. However, I cannot see the difference because these are small areas and the difference in size is below my threshold of discrimination. However, when I compare the wholes I can see the difference in size, because the difference is now greater than the threshold, since the whole is much larger than any of those parts. An auditory example would be tempo. Suppose I am listening to two sources, where the only difference is that one of them has a speed of 1.01 times the other. If I listen to short excerpts any difference is below the just-noticeable-difference, but if the whole example is the Ring cycle, I will notice that one finishes before dark and the other doesn't, I get hungry during one but not the other, etc. So even if quick-switch tests, on balance, are the most sensitive, that doesn't mean there can't be things out there that don't get caught in their net (though they may be detectable in other ways). To come back to the SACD/CD example, my concern is whether, even if the quick-switch test were a "null," there could be differences that the test does not do a good job of proving the existence of. Rather than feel assured that science tells us there could not be such differences, it seems to me pretty apparent that every test has its limitations. Sound plausible? Mark |
#33
|
|||
|
|||
Sorry, one of my previous posts had a couple of extraneous words in
one place; here is a corrected version. Gary Eickmeier wrote: Mark DeBellis wrote: OK but I am saying, suppose the meaningful variable is a property of an extended passage. So you have to listen to an uninterrupted passage in order to perceive the property. Switching back and forth will defeat the purpose, yes? I would say this is a false premise. But perhaps you could give an example of a meaningful variable that is a property of a longer passage. I missed your post until now because, for some reason, it didn't download (I'm using Free Agent) from my news server. An example of a property that belongs to a temporally extended passage without belonging to short slices of it is being a descending C major scale, one octave long. That is a property a listener can perceive the passage as having, but it is a property that belongs to the whole, not short parts. Another perceivable property that belongs to a temporal whole is the property, belonging to a spoken sentence, of being syntactically well formed. Mark |
#34
|
|||
|
|||
"Mark DeBellis" wrote in message
... "Buster Mudd" wrote: Because "hearing" is a cognitive process; it takes place in the brain, not in the ear. So if your brain tells you you didn't hear it, even if soundwaves did strike your eardrum...and even (!) if at an earlier time your brain told you that you did hear it...for all intents & purposes, you didn't hear it. Saying "I heard it" is only useful if you can access the perception in order to make subsequent discriminations. I agree with the basic idea that perception is something that influences behavior, but why does the behavior have to be restricted to comparison and identification? Suppose a listener gives higher approval ratings to one set of (blind) stimuli than another, without ever trying to say which stimuli were the same and which were different. This would be an influence on behavior, but of a weaker sort than is required by the "can you reliably identify" type of test. Yet if 200 people do the same thing, you can apply statistical measures of difference and determine with high accuracy whether or not subjectively there is a difference, and if so, in what characteristics (assuming the scalar data is pertinent to the differences heard). That is exactly the kind of validation that is missing that would prove (or prove in the negative) whether quick-switch, "short form" tests such as ABX can measure the same thing. There are cases all the time when people perceive things and then forget them. At which point any information they may have gleaned from perceiving that thing is lost to them. Hence, the distinction between whether they actually perceived it & then forgot it, or never perceived it in the first place, is moot. Not if the information is still doing work somewhere in your cognitive economy, even though it can't be brought to consciousness, or is not specific enough to enable one to perform the identification task. Certainly. If you hear it again and respond the same way, it can influence either favorably or unfavorably your reception to the music being played / your evaluation of the system it is being played on. Thus most audiophiles emphasis on long term listening. |
#35
|
|||
|
|||
wrote in message ...
When I said the other day that audio subjectivism is the rejection of the scientific method, this is what I meant: Harry Lavo wrote: Sorry to sound like a broken record, but since you assert this constantly I can only do the same in response....the testing you favor has never been validated for the open-ended evaluation of reproduced music. Period. If it had been and could be demonstrated to have been so, it would be widely used and accepted by most every audiophile. The fact that it has not been, is not accepted, and flies in the face of so much otherwise different consensus means that to continue asserting it as you do, is an act of faith, nothing else. To believe this, one must ignore reams of scientific data that conflict with what you believe. In particular, one must assert, against all evidence, that human hearing operates differently when listening to reproduced music than it does at all other times, and that generations of scientists have just been deluding themselves. This is how Creation Science operates. Let's hope the audio field never sinks so low. I repeat, the test has never been directly validated for the purposes espoused here. And the vast majority of the scientific work has *NOT* been on music and certainly virtually none has been dedicated to the worth of abx testing as a means of open-ended evaluation of audio components. |
#36
|
|||
|
|||
Mark DeBellis wrote:
"Buster Mudd" wrote: Because "hearing" is a cognitive process; it takes place in the brain, not in the ear. So if your brain tells you you didn't hear it, even if soundwaves did strike your eardrum...and even (!) if at an earlier time your brain told you that you did hear it...for all intents & purposes, you didn't hear it. Saying "I heard it" is only useful if you can access the perception in order to make subsequent discriminations. I agree with the basic idea that perception is something that influences behavior, but why does the behavior have to be restricted to comparison and identification? It doesn't. You can easily do a DBT as, say, a preference test. If the subject reports the same preference at a statistically significant rate, we can assume that the two are different. Suppose a listener gives higher approval ratings to one set of (blind) stimuli than another, without ever trying to say which stimuli were the same and which were different. This would be an influence on behavior, but of a weaker sort than is required by the "can you reliably identify" type of test. There are cases all the time when people perceive things and then forget them. At which point any information they may have gleaned from perceiving that thing is lost to them. Hence, the distinction between whether they actually perceived it & then forgot it, or never perceived it in the first place, is moot. Not if the information is still doing work somewhere in your cognitive economy, even though it can't be brought to consciousness, or is not specific enough to enable one to perform the identification task. But it isn't, because of our short aural memory for partial loudness differences. bob |
#37
|
|||
|
|||
Greg Lee wrote:
Mark DeBellis wrote: ... The background to this is that I said I thought SACD sounded better than CD, and Chung suggested that I try a simple blind test, to see if I could reliably identify which was SACD and which was CD (after matching levels; SACD and CD layers of same disc). Indeed I could not, at least on one set of trials. Should the conclusion be that SACD sounds the same as CD? Or is it possible that the test I applied is inadequate in some way? How *could* it be possible that the test is inadequate? My question is basically an attempt to explain how that might be. ... Another possibility is suggested by an informal experiment on smells that I did on myself. I took a selection of spices that were all in the same type of bottle, closed my eyes, shuffled the bottles around, and opened and sniffed them one by one, trying to identify the spice by name. I was sure I could do it but was quite amazed to discover that I couldn't do it at all. With my eyes closed, I could tell the differences perfectly well, but I just couldn't connect the names with the smells. So I think it's *possible* that in unsighted comparisons, the part of the brain that does symbolic analysis and associated judgment is not in perfect communication with other parts. Or maybe it's possible that when you make the test tougher, you don't do as well? You're doing an open identification test. That would be the equivalent of listening to one amp and deciding whether it was a Rotel, Adcom, Krell, or Hafler. To succeed at such a test, even if it were possible, would require intense practice. All evidence suggests, however, that it would not be possible--that even if there were some slight sonic differences between those amps, which you could tell apart in a quick-switching comparison, you probably wouldn't be able to remember what is distinctive about each of them long enough to identify a single example. Whereas I'll bet lots of professional chefs, with a fair bit more training than you, would have no trouble putting the right labels on your spice rack. Olfactory analogies, like visual ones, don't work. bob |
#38
|
|||
|
|||
Mark DeBellis wrote:
A couple of further ideas to toss out, from an onlooker's perspective of course. Quick-switch tests are said to be the most sensitive, therefore best, yes? Because they permit finer discriminations. An observation: the ultimate purpose of an audio test is not discrimination per se. No, but if you can't discriminate between two things, then the differences between them are irrelevant. That is, it's not as if the job that has to be performed here is to discriminate two sources if they can be, to do the best job of discrimination we can, as if that were the real goal. The real purpose of the test is rather to find out what information is available to the listener in the context of use, or perhaps to estimate an upper bound on that information. Well, we know that. That's what the science of psychoacoustics is all about. Check it out sometime. It is possible that two sources in the ordinary context of use do not present different information to the listener, even if there are ways to set up testing situations (e.g., at higher volume) where a person could discriminate the sources. All well and good if we do, but it's not like having a test that permits such discrimination is a valuable achievement in itself; what we are basically interested in is making sure that if there are differences of information presented in the ordinary context of use, then they will show up and get discriminated in the test. Question: for all the resolving power quick-switch tests have, for all the power they have to put the stimulus under a microscope and discern small differences of detail, are there certain sorts of properties they are *not* so good at picking up? Is there perhaps a forest-for-trees phenomenon lurking somewhere out there? No there is not, according to all experts in the field. Or do you think you know more than the experts? Here's an off the cuff example. Suppose I have two digital photographs that are identical except that one is 1.01 the size of the other. First I compare them (this is the analogue of the quick-switch test) by comparing small portions of one with the other. The comparison is set up in such a way that when I compare a square portion of one with a square portion of the other, one of them is 1.01 as large as the other. However, I cannot see the difference because these are small areas and the difference in size is below my threshold of discrimination. However, when I compare the wholes I can see the difference in size, because the difference is now greater than the threshold, since the whole is much larger than any of those parts. Irrelevant and off-point visual analogy. Visual analogies don't work. Ever. An auditory example would be tempo. Suppose I am listening to two sources, where the only difference is that one of them has a speed of 1.01 times the other. If I listen to short excerpts any difference is below the just-noticeable-difference, but if the whole example is the Ring cycle, I will notice that one finishes before dark and the other doesn't, I get hungry during one but not the other, etc. But you're not discriminating between the two by listening to them. You're discriminating between them by looking at the clock. (And I don't know what the threshold is for speed variation, but at some point you really would be able to distinguish between them in a standard DBT.) So even if quick-switch tests, on balance, are the most sensitive, that doesn't mean there can't be things out there that don't get caught in their net (though they may be detectable in other ways). So far you haven't come up with a single one. That's because there aren't any. To come back to the SACD/CD example, my concern is whether, even if the quick-switch test were a "null," there could be differences that the test does not do a good job of proving the existence of. Rather than feel assured that science tells us there could not be such differences, it seems to me pretty apparent that every test has its limitations. Sound plausible? No. It sounds like you're grasping at straws because you don't like what the science is telling you. If you live in Kansas, I suggest you run for the state board of ed. bob |
#39
|
|||
|
|||
|
#40
|
|||
|
|||
If I understand you right, it follows from what you are saying that
there will a perceptible difference between two passages only if there are short corresponding portions of those passages which, when juxtaposed, will exhibit a perceptible loudness difference. OK, yes, that does seem plausible (*). What I am getting from this then is that the relevant task is not necessarily "Which one is SACD and which is CD?" but "Are A and B the same or different?" (This is for me the most important point.) And for the latter, yes, I can see, intuitively at least, why quick-switch tests are the most sensitive. The quick-switch test will catch a difference that exists between the passages only if the samples include points of divergence; if too few samples are taken, they might just miss them. (*) About the principle stated above which, if true, is a fact of psychology. It seems to be saying more or less that differences are perceptible in context only if they are perceptible in isolation. OK, suppose I am looking at a photograph with a continuous gradation from light to dark. And I can see the difference between that and a patch of constant tone. Then the principle would say that I can do this only if I can see the difference between small patches where the sample taken from the first photograph is more or less constant in tone. But not too small, because once the areas get very small I can't reliably compare them any more. OK now the auditory case. There is a signal that gets louder, and I can hear the difference between that and a signal of constant loudness. The principle says that I can do this only if I can hear the difference in loudness between short corresponding portions. Seems plausible ... and if the portions get to be too short then would reliability go down, just as in the visual case? Does it matter how the short portions are "juxtaposed"? Separated by silence or one followed continously by the other? Mark |
Reply |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
common mode rejection vs. crosstalk | Pro Audio | |||
Topic Police | Pro Audio |