Home |
Search |
Today's Posts |
#1
|
|||
|
|||
Blind listening test!
I found someone to help me. We are starting tonight. I'm going to do
some training of my ears first. I would appreciate any suggestions that people can give me for improving the test. Here's the system: I'm connecting a Marantz CD player to a CAL DAC to an Antique Sound Lab MG Head OTL Mk III headphone amplifier to AKG K501 headphones. The cables to test will run between the DAC and the headphone amplifier. They a Rat Shack 2M gold-plated, about $5 Transparent 2M, the one without network boxes, about $75 Making it blind ------------------ I will hang a sheet over most of the system. I still need to access the CD player to start and stop things and put them in and out. The CD player is at the bottom of my rack, so I will tuck the sheet into the component above it, blocking everything else from my view. I will make sure that there is no way I can see the cables themselves, but I will have to sit in front of the rack. My friend will hook up cables following my instructions, either A, B or X as I instruct. He will determine X with a coin toss and write it down. I will leave the room, he will hook up the cable and call me back into the room. We will need to have some interaction as he will call me, and we will pass each other coming in and out of the room. I will keep my eyes closed so at least I won't pick up any verbal clues. Hey it occurs to me that I can have him knock on something instead of calling me so I can't pick up on any voice cues. Level matching ------------------ I don't have a good way to control levels.. I don't have a CD with a test tone. Hmm, it occurs to me that I can burn one. I have a basic voltmeter. Okay, I will do some basic measurements on a test tone. Is a 1Khz test tone good? Also, it is a good assumption that two straight wires into a 100K ohm load aren't going to differ by more than 0.1 dB? But in any case I will leave the amplifier's volume control in one position and not touch it during each trial. Documentation --------------- I type everything that happens in time order into my laptop and publish it on the web. Protocol -------------- Basic ABX, but not with a switcher box-- instead I will ask my friend to hook up one of the cables each time. This will take a while, obviously, and I won't be able to do many auditions of each cable, nor will I be able to switch quickly. Also, if I get tired or confused, I will simply not give an answer for the current trial, and start with that trial again later or another day. Does that sound acceptable? It seems to me that there's a small chance this could be a way to defeat the test; i.e. refuse to answer when I'm not sure, implying that I'm only sure when I've picked up the answer by non-sonic means. However, I will write down everything that happens including the times that I refuse to answer so people can review it later. Training --------------- For the initial tests, I will ask my friend whether I guessed right after each trial. When I'm confident that I've learned well the sound of each cable, then the "test proper" will start. I will determine a number of trials and write down my answer after each trial, but not compare answers to the answer sheet until the proper number of trials has been reached. -Mike |
#2
|
|||
|
|||
Michael Mossey wrote:
I found someone to help me. We are starting tonight. I'm going to do some training of my ears first. I would appreciate any suggestions that people can give me for improving the test. Here's the system: I'm connecting a Marantz CD player to a CAL DAC to an Antique Sound Lab MG Head OTL Mk III headphone amplifier to AKG K501 headphones. Is that a tube amp? I'd want somebody to confirm that there aren't impedance problems here that might create FR anomalies. The cables to test will run between the DAC and the headphone amplifier. They a Rat Shack 2M gold-plated, about $5 Transparent 2M, the one without network boxes, about $75 Making it blind ------------------ I will hang a sheet over most of the system. I still need to access the CD player to start and stop things and put them in and out. The CD player is at the bottom of my rack, so I will tuck the sheet into the component above it, blocking everything else from my view. I will make sure that there is no way I can see the cables themselves, but I will have to sit in front of the rack. My friend will hook up cables following my instructions, either A, B or X as I instruct. He will determine X with a coin toss and write it down. I will leave the room, he will hook up the cable and call me back into the room. We will need to have some interaction as he will call me, and we will pass each other coming in and out of the room. I will keep my eyes closed so at least I won't pick up any verbal clues. Hey it occurs to me that I can have him knock on something instead of calling me so I can't pick up on any voice cues. Level matching ------------------ I don't have a good way to control levels.. I don't have a CD with a test tone. Hmm, it occurs to me that I can burn one. I have a basic voltmeter. Okay, I will do some basic measurements on a test tone. Is a 1Khz test tone good? Also, it is a good assumption that two straight wires into a 100K ohm load aren't going to differ by more than 0.1 dB? Hey, you're the engineer. You tell us. But if you can level-match, I'd check it at 100 and 10k as well. But in any case I will leave the amplifier's volume control in one position and not touch it during each trial. Documentation --------------- I type everything that happens in time order into my laptop and publish it on the web. Protocol -------------- Basic ABX, but not with a switcher box-- instead I will ask my friend to hook up one of the cables each time. This will take a while, obviously, and I won't be able to do many auditions of each cable, nor will I be able to switch quickly. Also, if I get tired or confused, I will simply not give an answer for the current trial, and start with that trial again later or another day. Does that sound acceptable? Uh, no. You should make your best judgment, and move on to the next trial. I'd also set a limit in advance: Do eight trials today, or do as many trials as you can fit in two hours, or something. (You migth want to do a dry run to see how many trials you can reasonably do.) The temptation to finagle when the numbers aren't going your way can be strong, so you want fixed rules going in. It seems to me that there's a small chance this could be a way to defeat the test; i.e. refuse to answer when I'm not sure, implying that I'm only sure when I've picked up the answer by non-sonic means. However, I will write down everything that happens including the times that I refuse to answer so people can review it later. Training --------------- For the initial tests, I will ask my friend whether I guessed right after each trial. This really isn't necessary for training. All you need to do is listen to each enough that you feel you can tell the two apart. If you wait until you get a string of correct answers, you may never start! When I'm confident that I've learned well the sound of each cable, then the "test proper" will start. I will determine a number of trials and write down my answer after each trial, but not compare answers to the answer sheet until the proper number of trials has been reached. -Mike |
#3
|
|||
|
|||
On 7 Apr 2005 01:12:22 GMT, "Michael Mossey"
wrote: Level matching ------------------ I don't have a good way to control levels.. I don't have a CD with a test tone. Hmm, it occurs to me that I can burn one. I have a basic voltmeter. Okay, I will do some basic measurements on a test tone. Is a 1Khz test tone good? Also, it is a good assumption that two straight wires into a 100K ohm load aren't going to differ by more than 0.1 dB? But in any case I will leave the amplifier's volume control in one position and not touch it during each trial. It's almost certainly the case that you won't need to worry about level-matching for this test, but if you are going to do it, then a 15kHz tests tone is better than 1kHz (any differences will most likely be due to cable capacitance, which will show up at high frequencies). Documentation --------------- I type everything that happens in time order into my laptop and publish it on the web. Protocol -------------- Basic ABX, but not with a switcher box-- instead I will ask my friend to hook up one of the cables each time. This will take a while, obviously, and I won't be able to do many auditions of each cable, nor will I be able to switch quickly. You will need to do at least ten trials, and preferably twenty, to get statistical significance. The usual standard around here is fifteen out of twenty for a 'success'. Also, if I get tired or confused, I will simply not give an answer for the current trial, and start with that trial again later or another day. Does that sound acceptable? It seems to me that there's a small chance this could be a way to defeat the test; i.e. refuse to answer when I'm not sure, implying that I'm only sure when I've picked up the answer by non-sonic means. However, I will write down everything that happens including the times that I refuse to answer so people can review it later. Best to document everything, including 'don't knows'. Training --------------- For the initial tests, I will ask my friend whether I guessed right after each trial. When I'm confident that I've learned well the sound of each cable, then the "test proper" will start. That's fine. I will determine a number of trials and write down my answer after each trial, but not compare answers to the answer sheet until the proper number of trials has been reached. That is important, although you if you plan a twenty trial test, you might want to check your results after ten trials, to see if there's any point in continuing. Get more than four out of ten wrong, you might as well save your time and grab a beer! :-) -- Stewart Pinkerton | Music is Art - Audio is Engineering |
#4
|
|||
|
|||
"Michael Mossey" wrote in message
... I found someone to help me. We are starting tonight. I'm going to do some training of my ears first. I would appreciate any suggestions that people can give me for improving the test. Here's the system: I'm connecting a Marantz CD player to a CAL DAC to an Antique Sound Lab MG Head OTL Mk III headphone amplifier to AKG K501 headphones. The cables to test will run between the DAC and the headphone amplifier. They a Rat Shack 2M gold-plated, about $5 Transparent 2M, the one without network boxes, about $75 Making it blind ------------------ I will hang a sheet over most of the system. I still need to access the CD player to start and stop things and put them in and out. The CD player is at the bottom of my rack, so I will tuck the sheet into the component above it, blocking everything else from my view. I will make sure that there is no way I can see the cables themselves, but I will have to sit in front of the rack. My friend will hook up cables following my instructions, either A, B or X as I instruct. He will determine X with a coin toss and write it down. I will leave the room, he will hook up the cable and call me back into the room. We will need to have some interaction as he will call me, and we will pass each other coming in and out of the room. I will keep my eyes closed so at least I won't pick up any verbal clues. Hey it occurs to me that I can have him knock on something instead of calling me so I can't pick up on any voice cues. Level matching ------------------ I don't have a good way to control levels.. I don't have a CD with a test tone. Hmm, it occurs to me that I can burn one. I have a basic voltmeter. Okay, I will do some basic measurements on a test tone. Is a 1Khz test tone good? Also, it is a good assumption that two straight wires into a 100K ohm load aren't going to differ by more than 0.1 dB? But in any case I will leave the amplifier's volume control in one position and not touch it during each trial. Documentation --------------- I type everything that happens in time order into my laptop and publish it on the web. Protocol -------------- Basic ABX, but not with a switcher box-- instead I will ask my friend to hook up one of the cables each time. This will take a while, obviously, and I won't be able to do many auditions of each cable, nor will I be able to switch quickly. Also, if I get tired or confused, I will simply not give an answer for the current trial, and start with that trial again later or another day. Does that sound acceptable? It seems to me that there's a small chance this could be a way to defeat the test; i.e. refuse to answer when I'm not sure, implying that I'm only sure when I've picked up the answer by non-sonic means. However, I will write down everything that happens including the times that I refuse to answer so people can review it later. Training --------------- For the initial tests, I will ask my friend whether I guessed right after each trial. When I'm confident that I've learned well the sound of each cable, then the "test proper" will start. I will determine a number of trials and write down my answer after each trial, but not compare answers to the answer sheet until the proper number of trials has been reached. That sounds like a good test, Mike. I would only suggest that you make sure your assistant disconnects the cables after every trial, and BEFORE he flips the coin. There's a tendency to just leave the cables in place if there's no change from the previous trial. This is bad news. Also, I would be surprised if there was any significant level change between cables. If there is, I'd investigate the setup thoroughly; there shouldn't be. Norm Strong |
#5
|
|||
|
|||
Follow up to this:
I did some blind listening to cables last night and did not feel confident that I could tell the difference. I also observed some facts about blind listening. I agree with the objectivists that expectation during sighted listening can affect what we hear. I observed that even in blind listening, I formed "expectations." For example, on the basis of an initial impression I might immediately make a conclusion about the sound as a whole, and it was extremely difficult to separate the sound from those expectations. At the moment I'm probably not going to follow up with a more extended blind test because I don't feel confident I can tell the difference between cables. I may get some ideas about how to make the test conditions more sensitive, and maybe I'll reconsider. I already use my cheapest interconnect in my main system, anyway. -Mike |
#6
|
|||
|
|||
wrote:
"Michael Mossey" wrote in message ... I found someone to help me. We are starting tonight. I'm going to do some training of my ears first. I would appreciate any suggestions that people can give me for improving the test. Here's the system: I'm connecting a Marantz CD player to a CAL DAC to an Antique Sound Lab MG Head OTL Mk III headphone amplifier to AKG K501 headphones. The cables to test will run between the DAC and the headphone amplifier. They a Rat Shack 2M gold-plated, about $5 Transparent 2M, the one without network boxes, about $75 Making it blind ------------------ I will hang a sheet over most of the system. I still need to access the CD player to start and stop things and put them in and out. The CD player is at the bottom of my rack, so I will tuck the sheet into the component above it, blocking everything else from my view. I will make sure that there is no way I can see the cables themselves, but I will have to sit in front of the rack. My friend will hook up cables following my instructions, either A, B or X as I instruct. He will determine X with a coin toss and write it down. I will leave the room, he will hook up the cable and call me back into the room. We will need to have some interaction as he will call me, and we will pass each other coming in and out of the room. I will keep my eyes closed so at least I won't pick up any verbal clues. Hey it occurs to me that I can have him knock on something instead of calling me so I can't pick up on any voice cues. Level matching ------------------ I don't have a good way to control levels.. I don't have a CD with a test tone. Hmm, it occurs to me that I can burn one. I have a basic voltmeter. Okay, I will do some basic measurements on a test tone. Is a 1Khz test tone good? Also, it is a good assumption that two straight wires into a 100K ohm load aren't going to differ by more than 0.1 dB? But in any case I will leave the amplifier's volume control in one position and not touch it during each trial. Documentation --------------- I type everything that happens in time order into my laptop and publish it on the web. Protocol -------------- Basic ABX, but not with a switcher box-- instead I will ask my friend to hook up one of the cables each time. This will take a while, obviously, and I won't be able to do many auditions of each cable, nor will I be able to switch quickly. Also, if I get tired or confused, I will simply not give an answer for the current trial, and start with that trial again later or another day. Does that sound acceptable? It seems to me that there's a small chance this could be a way to defeat the test; i.e. refuse to answer when I'm not sure, implying that I'm only sure when I've picked up the answer by non-sonic means. However, I will write down everything that happens including the times that I refuse to answer so people can review it later. Training --------------- For the initial tests, I will ask my friend whether I guessed right after each trial. When I'm confident that I've learned well the sound of each cable, then the "test proper" will start. I will determine a number of trials and write down my answer after each trial, but not compare answers to the answer sheet until the proper number of trials has been reached. That sounds like a good test, Mike. I would only suggest that you make sure your assistant disconnects the cables after every trial, and BEFORE he flips the coin. There's a tendency to just leave the cables in place if there's no change from the previous trial. This is bad news. Also, I would be surprised if there was any significant level change between cables. If there is, I'd investigate the setup thoroughly; there shouldn't be. Norm Strong Yes, I had my assistant disconnect and reconnect the cables on every trial. My initial impressions of the sound did not correlate with the cables. I.e., my friend hooked up cables a few times, and I just listened to them blind and wrote down observations. Then he revealed the order, I checked my observations against that, and there was no match. I did not find any aspect of the sound that would let me tell the cables apart. Except at the very beginning. I had strong opinions about the first three trials and I was right about them. Easily a chance phenomenon, but it is interesting to hypothesize that my ear was less discrinimating after listening to many things--which is something I observe, that the more stuff I listen to in an hour, the more it all sounds the same. Of course that could also be because it is all the same. -Mike |
#7
|
|||
|
|||
Michael Mossey wrote:
Follow up to this: I did some blind listening to cables last night and did not feel confident that I could tell the difference. Aww, does this mean you're not going to tell us how you scored? We were really looking forward to that, you know. I also observed some facts about blind listening. I agree with the objectivists that expectation during sighted listening can affect what we hear. I observed that even in blind listening, I formed "expectations." That's how pernicious it is. You don't have to know anything more than that the two things you are listening to are different (not how, or what they are), and they can sound different. For example, on the basis of an initial impression I might immediately make a conclusion about the sound as a whole, and it was extremely difficult to separate the sound from those expectations. A reason why longer listening might not help. Our brains can be so stubborn. At the moment I'm probably not going to follow up with a more extended blind test because I don't feel confident I can tell the difference between cables. I may get some ideas about how to make the test conditions more sensitive, and maybe I'll reconsider. I already use my cheapest interconnect in my main system, anyway. Keep in mind that you did not use a particularly sensitive test protocol. I know you don't want to believe that quick switching is the best way to do these things, but it is. If there were an audible difference between these interconnects, that would be the only way to determine it. bob |
#8
|
|||
|
|||
On 8 Apr 2005 22:33:25 GMT, "Michael Mossey"
wrote: My initial impressions of the sound did not correlate with the cables. I.e., my friend hooked up cables a few times, and I just listened to them blind and wrote down observations. Then he revealed the order, I checked my observations against that, and there was no match. I did not find any aspect of the sound that would let me tell the cables apart. Except at the very beginning. I had strong opinions about the first three trials and I was right about them. Easily a chance phenomenon, but it is interesting to hypothesize that my ear was less discrinimating after listening to many things--which is something I observe, that the more stuff I listen to in an hour, the more it all sounds the same. Of course that could also be because it is all the same. Fine, so *test* your new hypothesis. Do say three trials a day for a week, *then* check your score for the 21 trials. -- Stewart Pinkerton | Music is Art - Audio is Engineering |
#9
|
|||
|
|||
Stewart Pinkerton wrote:
On 8 Apr 2005 22:33:25 GMT, "Michael Mossey" wrote: My initial impressions of the sound did not correlate with the cables. I.e., my friend hooked up cables a few times, and I just listened to them blind and wrote down observations. Then he revealed the order, I checked my observations against that, and there was no match. I did not find any aspect of the sound that would let me tell the cables apart. Except at the very beginning. I had strong opinions about the first three trials and I was right about them. Easily a chance phenomenon, but it is interesting to hypothesize that my ear was less discrinimating after listening to many things--which is something I observe, that the more stuff I listen to in an hour, the more it all sounds the same. Of course that could also be because it is all the same. Fine, so *test* your new hypothesis. Do say three trials a day for a week, *then* check your score for the 21 trials. -- Stewart Pinkerton | Music is Art - Audio is Engineering Of course I want to test it. Why would you think otherwise? Actually I have something I want to do first. My plan is to have my friend hook up one of two cables at random. I will listen with different selections of music, and in different ways. And I will write down my impressions on a scale. For example, "harshness" vs. "smooth" on a scale of 1 to 5. Sometimes I will do long listening. Sometimes I will make an instant impression. Then my friend will reveal the order to me. I will look for any correlation in my data. Does there seem to be a pattern in long listening tests, but not in quick impressions? Or vice-versa? Is "harshness" a good place to listen, or is something else better? Basically I'm looking for the best listening strategy. If there appears to be no way to tell the cables apart after this experiment it is pointless to continue. However, if I find a pattern, then I will set up the 20 blind ABX trials. I only have one friend who is only occasionally free to help me with this, so it's going to be a while. -Mike |
#10
|
|||
|
|||
"Michael Mossey" wrote in message
... Of course I want to test it. Why would you think otherwise? Actually I have something I want to do first. My plan is to have my friend hook up one of two cables at random. I will listen with different selections of music, and in different ways. And I will write down my impressions on a scale. For example, "harshness" vs. "smooth" on a scale of 1 to 5. Sometimes I will do long listening. Sometimes I will make an instant impression. What you're proposing here is an "investigation." That's a good idea, but why not start by replicating some of the experiments already run by others, using a variable that's easily controlled. I would advise changing nothing but the volume level. Can you tell the difference between 2 signals that differ by 1db, and nothing else? Try it using music, sine waves of different frequencies, white noise and pink noise. My guess is that you'll have difficulty hearing a difference with music as a source; less difficulty using a steady signal, either sine or noise. Once you find the most sensitive signal, change the reference level. Try 60/61db, then 80/81db, then 100/101db. After you've found the signal that makes a 1db level difference most obvious, reduce the difference from 1db to 0.5db, etc. etc. What I'm suggesting here is that you investigate your own hearing ability before proceeding to something that is entirely terra incognita. At least that's the way I'd approach the problem. Cheers, Norm Strong |
#11
|
|||
|
|||
Michael Mossey wrote:
I only have one friend Isn't that always the case with us audio geeks? |
#12
|
|||
|
|||
|
#13
|
|||
|
|||
Buster Mudd wrote:
wrote: Keep in mind that you did not use a particularly sensitive test protocol. I know you don't want to believe that quick switching is the best way to do these things, but it is. If there were an audible difference between these interconnects, that would be the only way to determine it. I thought the most compelling defense against the Subjectivist Camp's insistence that quick switching wasn't practical, realistic, or ultimately revealing of sonic differences was that you DIDN'T have to do quick switching! You don't. But if you don't, there will be some audible differences that you miss. OTOH, for interconnects it really won't make a difference either way. bob |
#14
|
|||
|
|||
Buster Mudd wrote:
wrote: Keep in mind that you did not use a particularly sensitive test protocol. I know you don't want to believe that quick switching is the best way to do these things, but it is. If there were an audible difference between these interconnects, that would be the only way to determine it. I thought the most compelling defense against the Subjectivist Camp's insistence that quick switching wasn't practical, realistic, or ultimately revealing of sonic differences was that you DIDN'T have to do quick switching! Haven't DBT proponents (including the many on RAHE) repeatedly argued that quick switching is NOT a requirement of these tests? That's what makes "Audio Objectivism" so appealing to me: theoretically, Mr. Audiophile can use his own equipment in his own room playing his own material at his own pace on his own schedule...he can take hours, days, weeks, months to reach his conclusion...the only difference between his normal music listening activities & his participation in this hypothetical double-blind ABX test are A) he can't ever know whether A or B is in place at any given time, and B) at some point he does have to make a guess as to whether X is A or B. If that's not the case, where's the fun? (And besides, regardless of whether or not quick switching is demonstrably better for identifying audible differences, wouldn't slow switching be just as good for identifying the inability to identify audible differences?) No, because extant psychoacoustic data suggest that 'slow switching' (long intervals between sound samples) is a less sensitive means for detecting subtle differences between the samples, than 'quick switching'. Ditto long samples versus short samples. So an 'inability' identified in this way might simply be due to the insensitivity of the protocol. That needs to be addressed by trying the more sensitive protocol. So, yes, Mr. Audiophile is free to do the test any way he wants, as long as A and B are adhered to. But if Mr. A. *fails* to detect difference using his own switching protocols (e.g., 'slow switching' using long samples), Mr. Objectivist will suggest he try protocols considered to be more sensitive, that involve short samples and quick switching. He might even suggest some progressive ABX 'ear training' for differences. In doing so Mr. O is trying to *help* Mr. A substantiate the differences he wants so badly to hear, not hinder him. ; -- -S It's not my business to do intelligent work. -- D. Rumsfeld, testifying before the House Armed Services Committee |
Reply |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
OPINIONS: Just How Good Are MP3's? | Car Audio | |||
James Randi gets clarified on audio biz | High End Audio | |||
Do any DVD receivers play MP3s on DVD-/+R or DVD-/+RW yet? | Car Audio | |||
FW: Yet another DBT post | High End Audio | |||
Some serious cable measurements with interesting results. | High End Audio |