| Home |
| Search |
| Today's Posts |
|
#161
|
|||
|
|||
Does anyone know of this challenge?
|
|
#162
|
|||
|
|||
|
Does anyone know of this challenge?
S888Wheel wrote:
From: Howard Ferstler Date: 5/14/2004 11:01 AM Pacific Standard Time Message-id: wP7pc.49177$xw3.2938368@attbi_s04 I have come into this one a bit late, but I do want to interject my two-cents worth. I mean it is obvious that the old "amps have a definable sound" argument lives on in different forms, with different rationalizations, even if the amps measure the same and are built by competent designers, and sound the same during blind comparisons. Do you know of any examples of amps actually measuring "the same" that are claimed to sound different? Few amps (well, let's be candid, no amps) measure exactly the same. However, they often measure very close, indeed; close enough to be subjective-performance clones of each other. Yet reviewers often compare such amps (not level matched, and not in such a way that they can compare quickly) and then go off and describe differences that are simply not going to exist. I believe they do so for two reasons: First, they are often psychologically dependent upon audio being an esoteric, mysterious, and fascinating hobby. Going the brass tacks route would undermine those motivations. In other words, they like the mystery of it all as much as their readers. Second, they depend upon a magazine readership that shares the views noted in the first reason, above. If they delivered a brass-tacks review those readers would be likely to protest to the editor and possibly cancel subscriptions. When I started writing for Fanfare years ago and pretty much stated my opinions regarding the so-called "sound" of amplifiers and CD players the editor got a substantial number of "I'm canceling my subscription" letters from audio buffs, even though the magazine was one of the best recording-review publications in business. I mean, the reason to subscribe to the magazine was to get good record reviews, not to indulge in audio fantasies. Unfortunately, the editor (who was not an audiophile) had for some time been employing one or two equipment reviewers and equipment-oriented commentators who had managed to build up a following of true-believer audio enthusiasts. The introduction of my skepticism into the mix tended to rile those people, and the editor contacted me and expressed serious concern about the potential for a "subscription cancellation" problem to get out of hand. Incidentally, I left the magazine after a year of working for the editor for a number of reasons, but the lack of support was not one of them. One involved just getting tired of writing rebuttal letters. Another involved the low pay, since like Mickey Spillane I write for money. OK, we read about this amp "sound" thing all the time, both in manufacturer ads and in enthusiast magazine test reports, and of course we hear it proclaimed by numerous high-end boutique sales people. Certain reviewers are particularly bad. One may get hold of an esoteric, "super-duper" (and expensive or even super-expensive) amp, and after discussing its sometimes arcane features and maybe even doing some rudimentary measurements (or spouting often bizarre manufacturer specifications), What would constitute a "bizarre manufacturer specification?" It is not unusual at all for a high-end manufacturer to make statements about soundstaging, depth, focus, etc., although they usually leave that sort of thing to reviewers. While such statements are not specifications, per se, they come across as such to impressionable consumers. Probably, we see more hyperbole with upscale CD players than we do with amplifiers. that reviewer may engage in an almost poetic monologue regarding its sound quality. Nothing wrong with "poetry" in the relm of editorial comment. It is a chosen form of communication. Some people enjoy it. Ah, now we get to the gist of your typical high-end journalism technique. Yep, many of those who read fringe-audio publications want a proto-mystical analysis of products. They DO NOT want measurements, and they also DO NOT want some hardware geek discussing precise listening/comparing techniques. They want to have the reviewer discuss how a particular piece of esoteric and expensive equipment transported him to another realm. They most definitely do not want the reviewer to fail to play what I like to call "the game." Often, he will do this while comparing it to other units (expensive and super-expensive) that he has on hand or has had on hand. He may not "hear" profound differences, but he rhapsodizes about the subtle ones he does hear. So? Not everyone wants subjective review to be clinical. Some find that boring. This is odd. They want to know what gear works well or at least what gear will do the job with the smallest amount of cash outlay (at least sharp consumers behave this way), and yet according to you what really interests some of those high-end types is a review that does not bore them. So, if the choice is between a non-boring review that spouts hyperbole and a brass-tacks review that delivers the goods, you claim that some guys will opt for the hyperbole? OK, but in that case they get sub-par sound (or at least par sound at inflated prices) for their lack of intellectual effort. If they want that, I suppose it is their business. However, I tend to believe that many such individuals are not inherently born that way. They are created by certain, misleading audio-journalism techniques. There is nothing wrong with different writers writing editorial with different styles. There is more to it than style. Certainly, it is possible to deliver a well-written and accurate review that would satisfy someone who is really interested in knowing the facts - as opposed to someone who simply wants to be entertained with a piece of fluff literature. Comparing the test unit to one he "had on hand" in the past is of course absurd, particularly when it comes to subtle differences, because there is no way anyone could do a meaningful comparison between units that were not set up and listened to at the same time. That is your opinion. You are entitled to it. But lets get down to the underlying meaning. Would you say this is true in the case of speakers that exhibit subtle differences? Are such observations meaningless? Speakers tend to be gross enough in their differences for a non-blind approach to work. I mean, the differences are there and what it boils down to is a matter of taste to a degree. I have compared wide-dispersion speaker designs to those that deliver a clean, narrowly focussed first-arrival signal and have little in the way of overpowering room reverb, and I can tell you that both approaches will work. Indeed, I have two AV systems in my house for reviewing purposes, with one set up making use of wide-dispersion main speakers (and center and surround speakers, too) and with the other being captained by a pair of speakers designed to deliver a phase-coherent first-arrival signal. This second system also has wide-dispersion surround speakers, however, because you need that sort of thing with surround sound. Anyway, both sound quite good, but each has its strong points. Now, the fact is that when comparing speakers it is still important to level match and do quick switching. I do this when I compare for product reviews and it is obvious as hell that the approach is as vital when comparing speakers as when comparing anything else. I may measure speakers (doing a room curve) and compare them that way over a period of time (I did this in issues 94 and 95 of The Sensible Sound a while back), but I make a point of noting that the curves are only starting points. Rough curves will indicate spectral-balance problems, but two systems with very similar room curves (at least in my rooms) may sound quite different in terms of spaciousness and soundstaging. They sound very similar in terms of spectral balance, however, and I rate that parameter very high. Incidentally, level matching with speakers is rather tricky, since there may be overlaps with each system's response curves that make it impossible to set up a balance point at a single frequency: i.e. doing it at 1 kHz, for example. One speaker may have a slight dip there and the other may have a moderate peak. Once level matched that way, they will not actually be balanced well at all. My technique involves doing two integrated moving-microphone RTA curves (one for each speaker) and then adjusting levels so that the curves overlap as much as possible. However, even when he has another "reference" unit on hand to compare with the device being reviewed the confrontation may be seriously flawed, mainly because levels are not matched and the comparison procedures are such that quick switching is impossible. One does not have to use quich switching to make relevant observations about what they hear. What they "think" they hear. They need to do the work level matched and with quick switching to validate what they "think" they hear when doing sloppy comparisons. I have done that sort of thing with speakers and the results were revealing. Needless to say, with amps (and CD players, too) the revelations were even more profound. Actually, few of those reviewers who get involved with amp reviewing do a blind comparison even once at the beginning of their reviewing careers - just to see just how revealing it will be of similarities. How do you know what other reviewers have and have not done? Well, when I read reviews that have the reviewers going on and on about the different sound of two amps they have on hand (or even with the second amp not on hand, due to its being reviewed some time back), or the sound of two CD players, or the sound of two sets of speaker wires or interconnects, I pretty much conclude that the guy either does not know what he is talking about or else is in the business of entertaining readers instead of informing them. If they did do some careful, level-matched, quick-switch comparisons between amps (or between wires or between CD players) they might change their tune - if their motivations involved speculation for its own sake. I firmly believe that some reviewers have done this sort of thing and rejected the results. They did so not because they did not believe them, but because they DID believe them and realized how bad such news would be for the high-end product-reviewing business. There is no audio-fun romanticism to be had in that kind of brass-tacks behavior. Are you saying there is no fun to be had in the "objectivist" approach to audio? There is plenty of fun, but only for those who realize that brass-tacks thinking about audio can be fun. For those who romanticize the hobby such behavior may be the very definition of dull. For a lot of people, audio involves a lot more than sound quality and accurately reproducing input signals. Interestingly, some "reviewers" go beyond commenting upon imagined subtle differences and will instead make proclamations about the vast differences between the amp under test and one or more reference units. The comments are often absurd in the extreme, with the commentary going on and on about soundstaging, depth, focus, transparency, dynamics, and the like. Do you really believe comentary on the characteristics of any given playback system in terms of imaging soundstage, depth and focus are inherently absurd? They are when the commentary involves amplifier sound. Actually, if someone did an improper level-match comparison between two amps (that is, they did a global level match and did not realize that the two amps might have channel-balance differences that would make them be balanced differently - even though the average levels from all channels were the same) they might hear soundstaging differences. However, if amps (or CD players, and certainly wires) are not screwed up in some way, and are balanced and level matched properly, soundstaging, depth, and focus should not be an issue. It would be an issue with speakers, of course, as I noted above. Radiation-pattern differences could have a huge impact with those items. Do you think that comments on a playback system's dynamic range and transparency are absurd? Well, you can have dynamic-range differences with amps, because one of two involved in a comparison might hit its clipping limit before the other. Then, you would hear differences. However, I have never said that was not possible. I said that up to their clipping limits, all properly designed amps (and most mainstream jobs are designed properly) sound the same. Of course, I should probably further qualify that and say that with some wild and weird speaker loads a weak-kneed amp might have problems. I think that Pinkerton has pointed that out, because he has done some comparisons with some pretty demanding speaker loads. The problem is that without close, level-matched comparing, opinions of that kind are not only a big joke they are also misleading the readers, and misleading readers, no matter how entertaining the report's often flowery text, is not the job of a product reviewer. The job of a product reviewer is determined by the editorial staff of any given publication. Not by you. The job of a product reviewer is to tell the truth. I have done a fair amount of comparing between amps, using some pretty good ancillary hardware, and with careful level matching. Let me tell you that although some amps might be very, very slightly different sounding from the mainstream (I found one that did, but shortly after the comparison it went up in smoke), nobody is going to be able to pinpoint such differences without doing some very close listening and precise comparing. You are unfortunately making universal proclamations based on your personal experience here. What may be a slight difference to your sensibilities may be a substantial difference to someone else's sensibilities and what you can or cannot do in terms of comparing things may or may not be universal. That is why it is best for reviewers to do comparisons level matched, with quick switching, and probably blind or double blind if they feel that they are likely to be biased. They certainly ought to do it that way during the initial part of their reviewing career, in order to see just how much alike amps (and CD players and wires, needless to say) sound. They certainly owe their readers more than poetic claptrap. What's more, an amp that does sound a tad different from mainstream models (here I am talking about some tube units, particularly single-ended versions) is probably going to not be as accurate an amplifying device as those others. Ironically, many of those good performing mainstream amps can be found contained inside of modestly priced AV receivers, at places like Best Buy and Circuit City. OK, now fantasy is sometimes fun and I do not begrudge any reader who wants to fantasize about his audio components. It seems you do IME. In what way? While I have some strong ideas about things like speaker radiation patterns and the need for a sensible approach to low-bass reproduction, I certainly do not attempt to offer up a one-sided slant to those topics when I write for my readers. And I do NOT begrudge any reader who wants to fantasize about his components, even though I kind of wonder about his motivations for being involved with the hobby. What I do begrudge are reviewers who capitalize on the naive approach some enthusiasts have when it comes to purchasing components. Hell, I do not even begrudge the reviewer who has prejudices that he keeps to himself. It is when he allows those prejudices to fool readers that I get a bit up in arms. I rather enjoy fantasizing myself when I am off line. However, when reviewing, reviewers should be different. They should deal with brass tacks and not speculations - even if speculations make for more poetic literature. Again, it is up to the editorial staff of the journal to determine what recviewers should and should not be doing. Reviewers owe it to their readers to be honest. The editorial staff also owes the same approach to those readers. Sure, some people are happy being suckered. However, those who see that as an OK thing and capitalize on it by creating more suckers through baloney journalism are not the kind of journalists that audio needs. Not if audio wants to be a viable, long-term hobby. Howard Ferstler |
|
#163
|
|||
|
|||
|
Does anyone know of this challenge?
"Harry Lavo" wrote in message
... I'm trying to gain enough intellectual support on this forum to spend a year of my time going to the trouble to set up the control test with Tom, since it obviously cannot be done by one person but must be done by many. My taking a test I don't believe in up front would show me nothing at this point. I said I would do it when it is part of a control test. And BTW, I have done blind testing in the past on my own, but not rigorously. And sometimes I can hear differences. And sometimes not. Same as when testing sighted. But those are anecdotes...I am trying to help the group see what is needed to scientifically separate the two variables...comparative/evaluative and blind/non-blind. Let's try this on for size: Suppose you have 2 speaker cables which appear to have quite different sonic signatures. You have essentially unlimited time to evaluate them in any way you feel necessary. All of this is sighted, of course. (I recommend writing down your thoughts as you evaluate the cables for future reference.) Is it your claim that even this is not enough to be able to identify which cable is connected without seeing it? At some point, you're going to have to bite the bullet and say, "This is Cable A. I reecognize the characteristics that I wrote down during the evaluative period." If not, I think we're wasting everybody's time--Harry's as well--and talking past each other. I've only done one speaker cable test, and it was essentially a biwiring test, not a straight cable quality test. Nonetheless, the results clearly indicated that the only characteristic that made any audible difference was the resistance of the wire. I still have the cables used in this test. If anyone would like to replicate my test (and document the results this time!) I would be most happy to mail all the wires to you. They consist of 4 pairs of AWG12, 18 & 24; all 33 feet in length. In practice I found it better to do a single channel test, but there's enough wire for stereo, if one prefers. I'm getting bored with the whole subject, so I'm not going to replicate my own test. Norm Strong |
|
#164
|
|||
|
|||
|
Does anyone know of this challenge?
|
|
#166
|
|||
|
|||
|
Does anyone know of this challenge?
On 5/17/04 6:39 PM, in article Fabqc.68734$536.11120778@attbi_s03,
"S888Wheel" wrote: From: Stewart Pinkerton Date: 5/17/2004 8:52 AM Pacific Standard Time Message-id: 7d5qc.68404$xw3.3855476@attbi_s04 IME, those who apply less rigorous attention to their hobby than to their work, are not really interested in that hobby. -- Funny. IME those who apply as much attention to their hobby/hobbies as their work are not really interested in thier work. Good point - but my point is that work pays much better than hobbies - so if you pay more attention to your hobbies - you may not have work for very long!! |
|
#168
|
|||
|
|||
|
Does anyone know of this challenge?
"normanstrong" wrote in message
news:khbqc.15167$gr.1357885@attbi_s52... "Harry Lavo" wrote in message ... I'm trying to gain enough intellectual support on this forum to spend a year of my time going to the trouble to set up the control test with Tom, since it obviously cannot be done by one person but must be done by many. My taking a test I don't believe in up front would show me nothing at this point. I said I would do it when it is part of a control test. And BTW, I have done blind testing in the past on my own, but not rigorously. And sometimes I can hear differences. And sometimes not. Same as when testing sighted. But those are anecdotes...I am trying to help the group see what is needed to scientifically separate the two variables...comparative/evaluative and blind/non-blind. Let's try this on for size: Suppose you have 2 speaker cables which appear to have quite different sonic signatures. You have essentially unlimited time to evaluate them in any way you feel necessary. All of this is sighted, of course. (I recommend writing down your thoughts as you evaluate the cables for future reference.) Is it your claim that even this is not enough to be able to identify which cable is connected without seeing it? At some point, you're going to have to bite the bullet and say, "This is Cable A. I reecognize the characteristics that I wrote down during the evaluative period." If not, I think we're wasting everybody's time--Harry's as well--and talking past each other. In all honesty, it doesn't make a difference. If I did the tests months apart (which would be the best way), I wouldn't even expect to remember my ratings accurately. What I would want to do is to listen once again, the same way, and rate the two components again the same way. Only this time I wouldn't know which was which. And if I did either remember or come up independently with a similar sonic signature, and accurately duplicate my ratings under blind conditions, and the majority of other testees did the same, then statistically you'd have to say that the sighted differences were real and that blinding per se did not invalidate them. If on the other hand, my initial ratings were "random" because the differences did not really exist, then I could not duplicate them except by chance and over the group of testees the results would be random and would not correlate statistically. And I could do all this without ever making a "choice". Then move on to the next stage of the control test ....evaluative vs. comparative. You may recall my discussion of food testing. Final testing was always done monadically, or proto-mondically (less frequent). Consumers were not "comparing", they were evaluating. The statistical analysis between the two (or more) sets of testees/variables was what determined if there was in fact a difference/preference. And on individual attributes as well as some overall satisfaction ratings, so the results could be understood in depth. I've only done one speaker cable test, and it was essentially a biwiring test, not a straight cable quality test. Nonetheless, the results clearly indicated that the only characteristic that made any audible difference was the resistance of the wire. I still have the cables used in this test. If anyone would like to replicate my test (and document the results this time!) I would be most happy to mail all the wires to you. They consist of 4 pairs of AWG12, 18 & 24; all 33 feet in length. In practice I found it better to do a single channel test, but there's enough wire for stereo, if one prefers. I'm getting bored with the whole subject, so I'm not going to replicate my own test. |
|
#169
|
|||
|
|||
|
Does anyone know of this challenge?
On 5/17/04 11:52 AM, in article 7d5qc.68404$xw3.3855476@attbi_s04, "Stewart
Pinkerton" wrote: No need to be an audio curmudgeon ! Sorry, but *you* claim to be a professional engineer, yet you apply extreme sloppiness to your listening tests regarding cable, and you refuse even to discuss the detail of your 'experiment'. IME, those who apply less rigorous attention to their hobby than to their work, are not really interested in that hobby. Listen. I keep my profession and my hobbies seprate. I never claimed my test was supposed to be definitive. As a professional, you should realize that sometimes a quick and dirty test will tell you if something is worth pursuing further. Engineering is as much about rigor as the proper allocation of resources towards a task. I take umbridge at your claims of sloppiness. And, yes, I pay much closer to work than this hobby - it pays better! |
|
#170
|
|||
|
|||
|
Does anyone know of this challenge?
On 5/17/04 6:44 PM, in article cfbqc.16222$qA.2005223@attbi_s51, "Howard
Ferstler" wrote: First, they are often psychologically dependent upon audio being an esoteric, mysterious, and fascinating hobby. Going the brass tacks route would undermine those motivations. In other words, they like the mystery of it all as much as their readers. I do think there is an element of romance to it all. I can see how someone might be really into the romance of it. |
|
#171
|
|||
|
|||
|
Does anyone know of this challenge?
|
|
#172
|
|||
|
|||
|
Does anyone know of this challenge?
On Mon, 17 May 2004 23:46:11 GMT, Bromo wrote:
On 5/17/04 6:39 PM, in article Fabqc.68734$536.11120778@attbi_s03, "S888Wheel" wrote: From: Stewart Pinkerton Date: 5/17/2004 8:52 AM Pacific Standard Time Message-id: 7d5qc.68404$xw3.3855476@attbi_s04 IME, those who apply less rigorous attention to their hobby than to their work, are not really interested in that hobby. -- Funny. IME those who apply as much attention to their hobby/hobbies as their work are not really interested in thier work. In the real world, not that many people are lucky enough to be paid for doing something that they really enjoy. This does not of course prevent them from doing it well. Good point - but my point is that work pays much better than hobbies - so if you pay more attention to your hobbies - you may not have work for very long!! Perhaps this is a world view which does not cross the Atlantic very well. -- Stewart Pinkerton | Music is Art - Audio is Engineering |
|
#173
|
|||
|
|||
|
Does anyone know of this challenge?
"Nousaine" wrote in message
news:9hqqc.962$zw.585@attbi_s01... "Harry Lavo" wrote: "normanstrong" wrote in message news:khbqc.15167$gr.1357885@attbi_s52... "Harry Lavo" wrote in message ... I'm trying to gain enough intellectual support on this forum to spend a year of my time going to the trouble to set up the control test with Tom, since it obviously cannot be done by one person but must be done by many. My taking a test I don't believe in up front would show me nothing at this point. I said I would do it when it is part of a control test. And BTW, I have done blind testing in the past on my own, but not rigorously. And sometimes I can hear differences. And sometimes not. Same as when testing sighted. But those are anecdotes...I am trying to help the group see what is needed to scientifically separate the two variables...comparative/evaluative and blind/non-blind. Let's try this on for size: Suppose you have 2 speaker cables which appear to have quite different sonic signatures. You have essentially unlimited time to evaluate them in any way you feel necessary. All of this is sighted, of course. (I recommend writing down your thoughts as you evaluate the cables for future reference.) Is it your claim that even this is not enough to be able to identify which cable is connected without seeing it? At some point, you're going to have to bite the bullet and say, "This is Cable A. I reecognize the characteristics that I wrote down during the evaluative period." If not, I think we're wasting everybody's time--Harry's as well--and talking past each other. In all honesty, it doesn't make a difference. If I did the tests months apart (which would be the best way), I wouldn't even expect to remember my ratings accurately. What I would want to do is to listen once again, the same way, and rate the two components again the same way. Only this time I wouldn't know which was which. And if I did either remember or come up independently with a similar sonic signature, and accurately duplicate my ratings under blind conditions, and the majority of other testees did the same, then statistically you'd have to say that the sighted differences were real and that blinding per se did not invalidate them. If on the other hand, my initial ratings were "random" because the differences did not really exist, then I could not duplicate them except by chance and over the group of testees the results would be random and would not correlate statistically. And I could do all this without ever making a "choice". OK; and if your results were not statistically confirmed by your second listening then what would your conclusions be? You'll say that "blinding" caused the difference, whereas most everbody else would conclude that the subject was unreliable (which would be true) and that he/she didn't really "hear" definable acoustical differences the first time. Tom, do you comprehend what you read? Or do you not read carefully? Please re-read the last two sentences of my paragraph above yours, and tell me it differs from what you just said....how? Then move on to the next stage of the control test ....evaluative vs. comparative. You may recall my discussion of food testing. Final testing was always done monadically, or proto-mondically (less frequent). Consumers were not "comparing", they were evaluating. The statistical analysis between the two (or more) sets of testees/variables was what determined if there was in fact a difference/preference. And on individual attributes as well as some overall satisfaction ratings, so the results could be understood in depth. OK; and what if those subjects were unable to reliably distinguish between the samples? Didn't you have built-in controls to test that too? If there is no statistical difference, then there is no difference. It's as simple as that. I've only done one speaker cable test, and it was essentially a biwiring test, not a straight cable quality test. Nonetheless, the results clearly indicated that the only characteristic that made any audible difference was the resistance of the wire. I still have the cables used in this test. If anyone would like to replicate my test (and document the results this time!) I would be most happy to mail all the wires to you. They consist of 4 pairs of AWG12, 18 & 24; all 33 feet in length. In practice I found it better to do a single channel test, but there's enough wire for stereo, if one prefers. I'm getting bored with the whole subject, so I'm not going to replicate my own test. Isn't it interesting that people have so much time to "argue" about controlled listening tests when they are fairly easy to implement? Easy does not equate to useful if they are not accurate. At best, then, they would be a waste of time. At worst, they would lead to misleading conclusions. The decision not to do it is very logical if you have reason to doubt the validity of the test, until and unless the test is validated. |
|
#174
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Harry, every point you want to make about testing, all of them, can be
accomplished by an abx test. Using it you can know what a and b are and have them visable at all times. You can "evaluate" and jot notes and do whatever you wish for as long as you wish listening to either a or b alone or in comparsion for as many times as you wish. Once you havd down pat what you think clearly different, preference can be formed but is irrelevant, any difference at all you think exist should be clearly there when you hit the x choice. You can listen to x as long as you wish, consult your notes etc. by which to identify if it is a or b. What part of the above doesn't in fact accomplish your goals? In fact it could take less time as the x choice is done at the end of an "evaluation" period and the entire period need not be repeated, unless you choose to do so. We can go further, there have been abx tests where folk took as long as they wished, we can consult these results to get at least a preliminary look at the test you are really proposing; as an indication if it is worth redoing. What you propose is the above without the abx being a part of the "evaluation" period, but it should make no difference at all if doing the "evaluation" the abx box is sitting there already for the x choice to be made. Once a minimal number of "evaluation" and x have been done, you can compare notes and results without regard to the stats if you wish, but of course the stats will say if x choices were at a level different then that of random guessing. |
|
#175
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
wrote in message news:Udrqc.1168$zw.477@attbi_s01...
Harry, every point you want to make about testing, all of them, can be accomplished by an abx test. Using it you can know what a and b are and have them visable at all times. You can "evaluate" and jot notes and do whatever you wish for as long as you wish listening to either a or b alone or in comparsion for as many times as you wish. Once you havd down pat what you think clearly different, preference can be formed but is irrelevant, any difference at all you think exist should be clearly there when you hit the x choice. You can listen to x as long as you wish, consult your notes etc. by which to identify if it is a or b. What part of the above doesn't in fact accomplish your goals? In fact it could take less time as the x choice is done at the end of an "evaluation" period and the entire period need not be repeated, unless you choose to do so. We can go further, there have been abx tests where folk took as long as they wished, we can consult these results to get at least a preliminary look at the test you are really proposing; as an indication if it is worth redoing. What you propose is the above without the abx being a part of the "evaluation" period, but it should make no difference at all if doing the "evaluation" the abx box is sitting there already for the x choice to be made. Once a minimal number of "evaluation" and x have been done, you can compare notes and results without regard to the stats if you wish, but of course the stats will say if x choices were at a level different then that of random guessing. You seem to miss the part about having to make a conscious "choice" (a left brain function) versus simply evaluating the musicality of the equipment (a right brain choice). I am not convinced (because I have never been shown a shred of evidence) that they measure the same thing. That is why I have proposed the test the way I have. You cannot "assume" that a test that is valid for determining if a specific non-musical artifact can be heard is also valid for open-end evaluation of audio components. They are two different things entirely. I personally think the a-b-x test is even more suspect than a straight a-b. |
|
#176
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Harry Lavo wrote:
wrote in message news:Udrqc.1168$zw.477@attbi_s01... Harry, every point you want to make about testing, all of them, can be accomplished by an abx test. Using it you can know what a and b are and have them visable at all times. You can "evaluate" and jot notes and do whatever you wish for as long as you wish listening to either a or b alone or in comparsion for as many times as you wish. Once you havd down pat what you think clearly different, preference can be formed but is irrelevant, any difference at all you think exist should be clearly there when you hit the x choice. You can listen to x as long as you wish, consult your notes etc. by which to identify if it is a or b. What part of the above doesn't in fact accomplish your goals? In fact it could take less time as the x choice is done at the end of an "evaluation" period and the entire period need not be repeated, unless you choose to do so. We can go further, there have been abx tests where folk took as long as they wished, we can consult these results to get at least a preliminary look at the test you are really proposing; as an indication if it is worth redoing. What you propose is the above without the abx being a part of the "evaluation" period, but it should make no difference at all if doing the "evaluation" the abx box is sitting there already for the x choice to be made. Once a minimal number of "evaluation" and x have been done, you can compare notes and results without regard to the stats if you wish, but of course the stats will say if x choices were at a level different then that of random guessing. You seem to miss the part about having to make a conscious "choice" (a left brain function) versus simply evaluating the musicality of the equipment (a right brain choice). I am not convinced (because I have never been shown a shred of evidence) that they measure the same thing. That is why I have proposed the test the way I have. You cannot "assume" that a test that is valid for determining if a specific non-musical artifact can be heard is also valid for open-end evaluation of audio components. They are two different things entirely. I personally think the a-b-x test is even more suspect than a straight a-b. I've often considered the objectivist viewpoint that "all competent amplifiers operating within their power ranges with appropriate speakers sound the same", etc. possibly true *for the measurable variables that they are interested in*, but nonetheless possibly not true - nor measurable by a-b or a-b-x tests - for the sound qualities that subjectivists are interested in. No doubt I'll be challenged on this view, but let me explain. When one reads a subjective review, or perhaps does one's own review either in a showroom or in one's home, one *might* be perceiving sonic qualities either not measured nor easily defined by the usual objectivist standards. For example, Harry has used the word "musicality". And I might use the same term, and others might make refernce to the imaging, soundstaging or *depth of field" qualities associated with a particular piece of equiopment. Still others may simply say "this sounds more realistic me" (than another component being compared). While it may be perfectly acceptable to the objectivists to consider only variables that can be measured in terms of frequency response or various kinds of distortion, I would be reluctant - as I think would be most subjectivists - to attribute the various variables I've mentioned above to specific, capable of replication, measurements to measure these things. Also, how often, even within the frequency response realm, are complete graphs presented that *might* account for a particular component being perceived as relatively analytical, dark, or lean - all terms frequently used by subjectivists? This is one of the reasons that I feel the 2 "camps" are really operating from almost totally different frames-of-reference and the endless challenges and disputes about the value of double blind testing, are, in practical terms, unlikely to convince anybody of anything they don't already strongly believe. Chacun a son gout!  Bruce J. Richman |
|
#177
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
No, I didn't miss it at all. For the sake of your "left" brain, let's do
it another way. Using abx never hit a or b, only do x. While there in that random choice do all the "evaluation" and left musical assessment you think prudent for as long and using notes and any other approach you want; you just don't know; you just don't know. Now after a series of x only, compare the results to your notes and experience of musicality and see if they match what x really was. There is no choice needed on your part, just your regular "evaluation" and left brain involvement. If each time x is b then the notes etc. should be obvious. You see, we really need to validate the "evaluation" test, if we can call it that, or perhaps more in line with your choice oflanguage we want to validate your "experience" of the musical event as presented by the gear at question. In either case and no matter how worded, the abx already can do what you want to do, as a pure coincidence, an objective test is being done at the same time. But you never need consult the statistical results, only the comparsion of your notes to the actual x chosen each time. You can even do a "subjective" language assessment before hand to gauge how closely your notes matched each example of a or b you experienced. You seem to miss the part about having to make a conscious "choice" (a left brain function) versus simply evaluating the musicality of the equipment (a right brain choice). I am not convinced (because I have never been shown a shred of evidence) that they measure the same thing. That is why I have proposed the test the way I have. You cannot "assume" that a test that is valid for determining if a specific non-musical artifact can be heard is also valid for open-end evaluation of audio components. They are two different things entirely. I personally think the a-b-x test is even more suspect than a straight a-b. |
|
#178
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Bruce J. Richman wrote:
I've often considered the objectivist viewpoint that "all competent amplifiers operating within their power ranges with appropriate speakers sound the same", etc. possibly true *for the measurable variables that they are interested in*, but nonetheless possibly not true - nor measurable by a-b or a-b-x tests - for the sound qualities that subjectivists are interested in. No doubt I'll be challenged on this view, but let me explain. When one reads a subjective review, or perhaps does one's own review either in a showroom or in one's home, one *might* be perceiving sonic qualities either not measured nor easily defined by the usual objectivist standards. For example, Harry has used the word "musicality". And I might use the same term, and others might make refernce to the imaging, soundstaging or *depth of field" qualities associated with a particular piece of equiopment. Still others may simply say "this sounds more realistic me" (than another component being compared). While it may be perfectly acceptable to the objectivists to consider only variables that can be measured in terms of frequency response or various kinds of distortion, I would be reluctant - as I think would be most subjectivists - to attribute the various variables I've mentioned above to specific, capable of replication, measurements to measure these things. Also, how often, even within the frequency response realm, are complete graphs presented that *might* account for a particular component being perceived as relatively analytical, dark, or lean - all terms frequently used by subjectivists? This is one of the reasons that I feel the 2 "camps" are really operating from almost totally different frames-of-reference and the endless challenges and disputes about the value of double blind testing, are, in practical terms, unlikely to convince anybody of anything they don't already strongly believe. Chacun a son gout! Whether one describes audible diffences in something as vague as 'musicality' or as specific and measurable as 'frequency response in the 1-2 kHz range' , it remains the case that a claim of having *heard* the difference can be best tested by employing controls for known sighted biases. Also, determining that a measurable differnece exists, is no guarantee that a given person will hear it. But it is virtually assured that when a given person's perception of audible difference is confirmed via testing, there will be a measurable difference as a reasonable cause. -- -S. "They've got God on their side. All we've got is science and reason." -- Dawn Hulsey, Talent Director |
|
#179
|
|||
|
|||
|
Does anyone know of this challenge?
Bromo wrote in message news:aRcqc.15561$gr.1402907@attbi_s52...
The power of a critic is truly awesome. People turn to others to analyze and judge the work of others all the time. Ask any actor, producer, car manufacturer, computer maker, artist or anyone. There are people that make it their living in passing these judgements. And if they do a good job - the people that follow their advice can save a lot of time AND make their lives better! Yes, but now you are assuming that these critics compare something with real differences. When differences can be easily found, then consumers can compare their own perceptions against critics findings. But in this thread the real differences are questioned. If there are no real differences then the lack of stimulus can lure the consumer to imagine the supposed diffrences. This is well recorded phenomenon and it has been studied with beers, cigarettes, bottled water and so on. Critics themselves are not immune to this either and if they receive positive feedback from hearing the differences they then have no motive to not hear the differences. If our brains do not receive the signals they are expecting then they will often create false signals of their own. However, when brains receive something to work with, then there is no need to imagine things. The only way to reliably calibrate your senses is to do blind tests. This is why wine tastings are often blind. No one would take seriously a wine critic who needs to see the bottle before writing his review. Lasse |
|
#180
|
|||
|
|||
|
Does anyone know of this challenge?
Harry Lavo wrote:
You may recall my discussion of food testing. Final testing was always done monadically, or proto-mondically (less frequent). Consumers were not "comparing", they were evaluating. The statistical analysis between the two (or more) sets of testees/variables was what determined if there was in fact a difference/preference. And on individual attributes as well as some overall satisfaction ratings, so the results could be understood in depth. The problem with comparing food testing with audio testing is that food has no standards for accuracy. Audio has always had well defined standards for accuracy and always will, and some people just don't want to deal with that fact. That's okay, except when bogus technical claims are made to justify a preference. |
|
#181
|
|||
|
|||
|
Does anyone know of this challenge?
Harry Lavo" wrote:
All original post retained "Nousaine" wrote in message ... "Harry Lavo" wrote: "Bob Marcus" wrote in message news:soupc.16766$6f5.1472957@attbi_s54... Harry Lavo wrote: Had you really followed my oft-repeated discussions of test (latest with Steven) you would note that I said explicitly that: 1) If the equipment under test were felt by subjectivists to have differences (and thought not by objectivists) and if the blind vs. sighted *evaluative* testing showed those differences to go away/be greatly mitigated by blinding then I would then I would gladly accept blinding with no reservations To be honest, Harry, I can't recall you ever stating this explicitly before. I'm not accusing you of being disingenuous here. I'm only suggesting that perhaps you weren't as clear as you thought you were. I said it that explicitly, albeit not that succinctly, here within the last month. And it was apparent that you, Tom, and others were not reading what I said at the time, but rather going into a "riff" based on what you think I said, or thought I would say, or wished I would say. But that is what I said. (and I further said that if real differences existed then I expected they would show up even if blinded in the evaluative test). 2) I further said that if the quick-switch comparative blind test showed the same results as the blind evaluative test, then I would swing over and support your test as validated. Pray tell, what fault can you possibly find with a test that allows those definitive conclusions to be reached (by me, and presumably by many other subjectivists here.) I and others have found numerous faults with it. Purely as a practical matter, it's impossible to pull off. I even suggested an alternative that would be far more straightforward, and meets every requirement you have insisted on, and you rejected that. Under the circumstances, I can understand Tom's suspicion that you were merely throwing up smoke. Your test did not "meet..every requirement I have insisted on". For it continued to be based on quick swith a-b testing, a technique that is of itself being questioned as possibly contributing to erroneous conclusions. That is why I proposed a test that started where the subjectivists live...with extended evaluative listening, and changed only the condiditon of "blindness" not the listening techniques themselves. That is why I rejected the approach as I said at the time. It is inviting in its simplicity but it would not sway subjectivists including myself because of this flaw. As for the "many other subjectivists here," I think you are being presumptuous. Finding flaws with bias-controlled tests seems to be part of what makes one a subjectivist. I see no reason to believe that your test, even if you could pull it off, would be any different. bob That's not how I read it/them at all. They may have some differences with me / my way of thinking, but the main problem they have is the *assumption* (unverified) that a test that is good for picking out small level differences in codec artecfact and other known acoustic anomalies is a suitable technique for open ended evaluation of audio components. Once that criticism is addressed, I think you will find most objections melt away if your assumptions prove true. Please. Those folks addressing codec artifacts are dealing with exactly the same issues ..... subtle acoustical differences no matter what the cause are what the more interested parties have interest in. As far as I can see the "only" reason you don't like bias-controlled tests is that they do not support your prior held beliefs. And you hold out the idea that until bias controlled tests can support your prior held beliefs they will remain "unvalidated." And your "proposed" validation test won't be considered as conclusive until the results would be the "same" as those obtained under un-bias controlled conditions. Excuse me if I wonder why? But since the very idea of the test possibly being flawed is so threatening that even acknowledging the possibility seems beyond the objectivist ken, there is little movement on either side. It seems to me the 'threatening' part is validating open-ended uncontrolled bias evaluation. One should be able to do that easily by showing the ability to come to the same conclusions under listener-bias controlled conditions following open-ended evaluation. Why won't you? Why has no one else done so? Again, Tom, you give evidence that you either don't read or don't want to understand what I say. That is exactly the purpose of the evaluative sighted-evaluative blind leg of my proposed test. And I just repeated the reasons for that in the lates post you are just now responding to. Please re-read paragraph #1 above and tell me what about believing "blind results" (if they support your position) I am above believing? Please read your proposal. "When" bias controls are shown to give the same results that open tests would give then you'll accept them. No? What has not been "validated" are the conclusions you gain from non-controlled open listening as to acoustical cause. To imply that simply removing the sight of I/O terminals or otherwise hiding the answers somehow changes the acoustics of the situation (inducing masking) is a misdirection. If on the other hand, conclusions you hold are simply the result of a myriad of influences many of which have no acoustical cause why should anyone else but you care? As I have said repeatedly, one casual, undocumented, anecdotal case does not prove your case, however interesting. How about over two dozen performed prior to May 1990? There is good reason for starting with sighted, open-ended evaluative listening...because that is where most audiophiles and reviewers start and make their judgments. Sure and that's where all Urban Legends start too. So? Of course, I'm not dismissing all listener observations either. It's just that those about amp/wire sound have been put to the test. Harry has amplifiers he feels meet the criteria of "better" sonics but he won't put himself to the experimental test of a simple bias controlled experiment. Instead he postulates a lengthy, costly experiment that is based on having the results "conform" to the results he already "knows" to be true before he'll accept them. In other words if a long listening biased controlled test doesn't support amp differences then he'll just say that it was masking real difference. This is the classic experimental dodge. Invent a needless experiment that will be unlikely to be conducted (with pre-held results that have to be confirned or the experiment would be "invalid" or "unconclusive") and you have the idea here. So if that is erroneous, you have to *prove* it with rigorous testing...and you do that by doing the exact same test but "blind" instead of sighted. You do not do it by changing the test technique as well as going blind. My case all along is that you have switched two variables at once, yet you impute the difference in results to only one of those variables, while in fact the other variable may be at fault. How much clearer can I make it!! Oh you've made your case very clear. As long as you can avoid dealing with other bias controlled experimental results you'll be quite happy to continue debating. |
|
#182
|
|||
|
|||
|
Does anyone know of this challenge?
|
|
#183
|
|||
|
|||
|
Does anyone know of this challenge?
Nousaine wrote:
"Harry Lavo" wrote: "normanstrong" wrote in message news:khbqc.15167$gr.1357885@attbi_s52... "Harry Lavo" wrote in message ... I'm trying to gain enough intellectual support on this forum to spend a year of my time going to the trouble to set up the control test with Tom, since it obviously cannot be done by one person but must be done by many. My taking a test I don't believe in up front would show me nothing at this point. I said I would do it when it is part of a control test. And BTW, I have done blind testing in the past on my own, but not rigorously. And sometimes I can hear differences. And sometimes not. Same as when testing sighted. But those are anecdotes...I am trying to help the group see what is needed to scientifically separate the two variables...comparative/evaluative and blind/non-blind. Let's try this on for size: Suppose you have 2 speaker cables which appear to have quite different sonic signatures. You have essentially unlimited time to evaluate them in any way you feel necessary. All of this is sighted, of course. (I recommend writing down your thoughts as you evaluate the cables for future reference.) Is it your claim that even this is not enough to be able to identify which cable is connected without seeing it? At some point, you're going to have to bite the bullet and say, "This is Cable A. I reecognize the characteristics that I wrote down during the evaluative period." If not, I think we're wasting everybody's time--Harry's as well--and talking past each other. In all honesty, it doesn't make a difference. If I did the tests months apart (which would be the best way), I wouldn't even expect to remember my ratings accurately. What I would want to do is to listen once again, the same way, and rate the two components again the same way. Only this time I wouldn't know which was which. And if I did either remember or come up independently with a similar sonic signature, and accurately duplicate my ratings under blind conditions, and the majority of other testees did the same, then statistically you'd have to say that the sighted differences were real and that blinding per se did not invalidate them. If on the other hand, my initial ratings were "random" because the differences did not really exist, then I could not duplicate them except by chance and over the group of testees the results would be random and would not correlate statistically. And I could do all this without ever making a "choice". OK; and if your results were not statistically confirmed by your second listening then what would your conclusions be? You'll say that "blinding" caused the difference, whereas most everbody else would conclude that the subject was unreliable (which would be true) and that he/she didn't really "hear" definable acoustical differences the first time. Also, I wonder if Harry would be willing to make a more modest claim, if his second listening yielded a 'no difference' result: namely, that *his* 'first listening' perception of difference between the two DUTs was probably imaginary. And having done so, would that experience temper/inform his other claims of having heard a difference? Would he, in effect, become more of an 'objectivist'? You may recall my discussion of food testing. Final testing was always done monadically, or proto-mondically (less frequent). Consumers were not "comparing", they were evaluating. The statistical analysis between the two (or more) sets of testees/variables was what determined if there was in fact a difference/preference. And on individual attributes as well as some overall satisfaction ratings, so the results could be understood in depth. OK; and what if those subjects were unable to reliably distinguish between the samples? Didn't you have built-in controls to test that too? Seem to me that any 'monadic' evaluation is also a sort of comparison -- indeed, any sensation taht we have to describe involves comparing, in the sense of asking yourself , e.g., does this taste like my *memory* of salty, sweet, bitter, etc. If the evulative report form is multiple-choice, this 'choosing' is all the more explicit. If the evaluative report form is scalar ('on a scale of 1-10 , with 1 being sweet and 10 being bitter') there's still choice involved. There is always some sort of real or virtual reference that one is comparing the sensation to. I would posit that the same is true for a 'monadic' evaluation of, say, a cable. You aren't directly comparing it to another real cable, but you are comparing what you hear to your store of memories of what 'smoothness', 'bass articulation', or whatever, sound like. Otherwise you could not make an 'evaluation'. -- -S. "They've got God on their side. All we've got is science and reason." -- Dawn Hulsey, Talent Director |
|
#184
|
|||
|
|||
|
"Evaluation" test or chicken fat test, how to decide?
"Easy does not equate to useful if they are not accurate. At best, then,
they would be a waste of time. At worst, they would lead to misleading conclusions. The decision not to do it is very logical if you have reason to doubt the validity of the test, until and unless the test is validated." In another place you said the "evaluation" approach was often used by reviewers and that is why anything else needs to be "validated" by "evaluation" first. Leaving aside that dubious bit of logic and precondition, how do we "evaluate" the "validity" of the "evaluation" approach? This assumes the "evaluation" experience is in fact a valid test or is it just sometime spent listening carefully. Frankly I see your continuing drumbeat about the "evaluation" test a bit too much a strawman. I offered my chicken fat test as an example of what you propose on this ng. In it I said any other test must be validated while having one's left hand submerged in chicken fat because that is how I listen and any other test must first be done thusly to make sure it too is valid. Sure it is easier not to use chicken fat, but not doing so is just a waste of time if another test is not valid, and we willn't know until and unless it is done with chicken fat. |
|
#185
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Harry Lavo wrote:
wrote in message news:Udrqc.1168$zw.477@attbi_s01... Harry, every point you want to make about testing, all of them, can be accomplished by an abx test. Using it you can know what a and b are and have them visable at all times. You can "evaluate" and jot notes and do whatever you wish for as long as you wish listening to either a or b alone or in comparsion for as many times as you wish. Once you havd down pat what you think clearly different, preference can be formed but is irrelevant, any difference at all you think exist should be clearly there when you hit the x choice. You can listen to x as long as you wish, consult your notes etc. by which to identify if it is a or b. What part of the above doesn't in fact accomplish your goals? In fact it could take less time as the x choice is done at the end of an "evaluation" period and the entire period need not be repeated, unless you choose to do so. We can go further, there have been abx tests where folk took as long as they wished, we can consult these results to get at least a preliminary look at the test you are really proposing; as an indication if it is worth redoing. What you propose is the above without the abx being a part of the "evaluation" period, but it should make no difference at all if doing the "evaluation" the abx box is sitting there already for the x choice to be made. Once a minimal number of "evaluation" and x have been done, you can compare notes and results without regard to the stats if you wish, but of course the stats will say if x choices were at a level different then that of random guessing. You seem to miss the part about having to make a conscious "choice" (a left brain function) versus simply evaluating the musicality of the equipment (a right brain choice). As presented, this dichotomy is simplistic and does not reflect the last 30 years of progress in brain lateralization research. -- -S. "They've got God on their side. All we've got is science and reason." -- Dawn Hulsey, Talent Director |
|
#186
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"I've often considered the objectivist viewpoint that "all competent
amplifiers operating within their power ranges with appropriate speakers sound the same", etc. possibly true *for the measurable variables that they are interested in*, but nonetheless possibly not true - nor measurable by a-b or a-b-x tests - for the sound qualities that subjectivists are interested in. No doubt I'll be challenged on this view, but let me explain." There is no restriction or demand any particular kind of difference be identified, just a difference, any difference. As with Harry, listen using any techneque desired in whatever manner chosen for as long as desired searching for any quality/quanity of the sound one feels illuminating and make notes accordingly. Then compare the notes to the actual x choice and see if there is any match at all that shows the sound quality/quantity sought was there or not. |
|
#187
|
|||
|
|||
|
"Evaluation" test or chicken fat test, how to decide?
wrote in message ...
"Easy does not equate to useful if they are not accurate. At best, then, they would be a waste of time. At worst, they would lead to misleading conclusions. The decision not to do it is very logical if you have reason to doubt the validity of the test, until and unless the test is validated." In another place you said the "evaluation" approach was often used by reviewers and that is why anything else needs to be "validated" by "evaluation" first. Leaving aside that dubious bit of logic and precondition, how do we "evaluate" the "validity" of the "evaluation" approach? This assumes the "evaluation" experience is in fact a valid test or is it just sometime spent listening carefully. Frankly I see your continuing drumbeat about the "evaluation" test a bit too much a strawman. I offered my chicken fat test as an example of what you propose on this ng. In it I said any other test must be validated while having one's left hand submerged in chicken fat because that is how I listen and any other test must first be done thusly to make sure it too is valid. Sure it is easier not to use chicken fat, but not doing so is just a waste of time if another test is not valid, and we willn't know until and unless it is done with chicken fat. Let me put it this way. I didn't say just reviewers. I said reviewers and most audiophiles. They take equipment home. They listen to it. They say things like " the soundstage just opened up", etc. The objectivists here say it is all just imaginary, the result of sighted bias. So how do you prove that. You can't do it by substituting another test. You have to do it by simply "blinding" the listener leaving the test as close as possible to what they do naturally. That is what the evaluate sighted vs. blind leg of the control test is designed to do...with just enough analytic rigor to make statistical analysis possible. You can't just switch to comparative testing, because that is yet a second variable in addition to blinding, versus the original sighted listening. Why is this so hard to understand? Especially for those of you who are scientists. I'm not saying evaluative listening is "validated". I'm saying that is what is done. If you want to say it is all imaginary and will go away with blinding, then *that* is what you add as a variable...blinding. Not another, completely different type of test. |
|
#188
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"Steven Sullivan" wrote in message
news:4gwqc.22942$qA.2544878@attbi_s51... Bruce J. Richman wrote: I've often considered the objectivist viewpoint that "all competent amplifiers operating within their power ranges with appropriate speakers sound the same", etc. possibly true *for the measurable variables that they are interested in*, but nonetheless possibly not true - nor measurable by a-b or a-b-x tests - for the sound qualities that subjectivists are interested in. No doubt I'll be challenged on this view, but let me explain. When one reads a subjective review, or perhaps does one's own review either in a showroom or in one's home, one *might* be perceiving sonic qualities either not measured nor easily defined by the usual objectivist standards. For example, Harry has used the word "musicality". And I might use the same term, and others might make refernce to the imaging, soundstaging or *depth of field" qualities associated with a particular piece of equiopment. Still others may simply say "this sounds more realistic me" (than another component being compared). While it may be perfectly acceptable to the objectivists to consider only variables that can be measured in terms of frequency response or various kinds of distortion, I would be reluctant - as I think would be most subjectivists - to attribute the various variables I've mentioned above to specific, capable of replication, measurements to measure these things. Also, how often, even within the frequency response realm, are complete graphs presented that *might* account for a particular component being perceived as relatively analytical, dark, or lean - all terms frequently used by subjectivists? This is one of the reasons that I feel the 2 "camps" are really operating from almost totally different frames-of-reference and the endless challenges and disputes about the value of double blind testing, are, in practical terms, unlikely to convince anybody of anything they don't already strongly believe. Chacun a son gout! Whether one describes audible diffences in something as vague as 'musicality' or as specific and measurable as 'frequency response in the 1-2 kHz range' , it remains the case that a claim of having *heard* the difference can be best tested by employing controls for known sighted biases. Yep...which is exactly what the evaluative stage of the control test....sighted vs blind..is intended to do. You can't do it by substituing the actual test under question as a matter of "faith". Also, determining that a measurable differnece exists, is no guarantee that a given person will hear it. But it is virtually assured that when a given person's perception of audible difference is confirmed via testing, there will be a measurable difference as a reasonable cause. Let's see..a difference may exist but I miss it. On the other hand, if I don't hear it you conclude.....what? |
|
#189
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
You seem to miss the part about having to make a conscious "choice" (a
left brain function) versus simply evaluating the musicality of the equipment (a right brain choice). I am not convinced (because I have never been shown a shred of evidence) that they measure the same thing. That is why I have proposed the test the way I have. You cannot "assume" that a test that is valid for determining if a specific non-musical artifact can be heard is also valid for open-end evaluation of audio components. They are two different things entirely. I personally think the a-b-x test is even more suspect than a straight a-b. wrote in message news:Azvqc.22582$qA.2523392@attbi_s51... No, I didn't miss it at all. For the sake of your "left" brain, let's do it another way. Using abx never hit a or b, only do x. While there in that random choice do all the "evaluation" and left musical assessment you think prudent for as long and using notes and any other approach you want; you just don't know; you just don't know. Now after a series of x only, compare the results to your notes and experience of musicality and see if they match what x really was. There is no choice needed on your part, just your regular "evaluation" and left brain involvement. If each time x is b then the notes etc. should be obvious. You see, we really need to validate the "evaluation" test, if we can call it that, or perhaps more in line with your choice oflanguage we want to validate your "experience" of the musical event as presented by the gear at question. In either case and no matter how worded, the abx already can do what you want to do, as a pure coincidence, an objective test is being done at the same time. But you never need consult the statistical results, only the comparsion of your notes to the actual x chosen each time. You can even do a "subjective" language assessment before hand to gauge how closely your notes matched each example of a or b you experienced. And how is this better than doing the evaluations and then using statistical analysis to determine if "differences" hold up under blind conditions? I don't understand how you are going to do any statistical analysis without making a left brain "choice". |
|
#190
|
|||
|
|||
|
Does anyone know of this challenge?
"Nousaine" wrote in message
... Harry Lavo" wrote: All original post retained "Nousaine" wrote in message ... "Harry Lavo" wrote: "Bob Marcus" wrote in message news:soupc.16766$6f5.1472957@attbi_s54... Harry Lavo wrote: Had you really followed my oft-repeated discussions of test (latest with Steven) you would note that I said explicitly that: 1) If the equipment under test were felt by subjectivists to have differences (and thought not by objectivists) and if the blind vs. sighted *evaluative* testing showed those differences to go away/be greatly mitigated by blinding then I would then I would gladly accept blinding with no reservations To be honest, Harry, I can't recall you ever stating this explicitly before. I'm not accusing you of being disingenuous here. I'm only suggesting that perhaps you weren't as clear as you thought you were. I said it that explicitly, albeit not that succinctly, here within the last month. And it was apparent that you, Tom, and others were not reading what I said at the time, but rather going into a "riff" based on what you think I said, or thought I would say, or wished I would say. But that is what I said. (and I further said that if real differences existed then I expected they would show up even if blinded in the evaluative test). 2) I further said that if the quick-switch comparative blind test showed the same results as the blind evaluative test, then I would swing over and support your test as validated. Pray tell, what fault can you possibly find with a test that allows those definitive conclusions to be reached (by me, and presumably by many other subjectivists here.) I and others have found numerous faults with it. Purely as a practical matter, it's impossible to pull off. I even suggested an alternative that would be far more straightforward, and meets every requirement you have insisted on, and you rejected that. Under the circumstances, I can understand Tom's suspicion that you were merely throwing up smoke. Your test did not "meet..every requirement I have insisted on". For it continued to be based on quick swith a-b testing, a technique that is of itself being questioned as possibly contributing to erroneous conclusions. That is why I proposed a test that started where the subjectivists live...with extended evaluative listening, and changed only the condiditon of "blindness" not the listening techniques themselves. That is why I rejected the approach as I said at the time. It is inviting in its simplicity but it would not sway subjectivists including myself because of this flaw. As for the "many other subjectivists here," I think you are being presumptuous. Finding flaws with bias-controlled tests seems to be part of what makes one a subjectivist. I see no reason to believe that your test, even if you could pull it off, would be any different. bob That's not how I read it/them at all. They may have some differences with me / my way of thinking, but the main problem they have is the *assumption* (unverified) that a test that is good for picking out small level differences in codec artecfact and other known acoustic anomalies is a suitable technique for open ended evaluation of audio components. Once that criticism is addressed, I think you will find most objections melt away if your assumptions prove true. Please. Those folks addressing codec artifacts are dealing with exactly the same issues ..... subtle acoustical differences no matter what the cause are what the more interested parties have interest in. As far as I can see the "only" reason you don't like bias-controlled tests is that they do not support your prior held beliefs. And you hold out the idea that until bias controlled tests can support your prior held beliefs they will remain "unvalidated." And your "proposed" validation test won't be considered as conclusive until the results would be the "same" as those obtained under un-bias controlled conditions. Excuse me if I wonder why? Once again you are deliberately and totally misrepresenting what I have said, Tom. I have said *ABSOLUTELY* no such thing. If you continue this, I can only conclude that you are deliberately falsehooding. I have told you *three times* to stop and check what I actually have said. You have not. I expect an apology!! But since the very idea of the test possibly being flawed is so threatening that even acknowledging the possibility seems beyond the objectivist ken, there is little movement on either side. It seems to me the 'threatening' part is validating open-ended uncontrolled bias evaluation. One should be able to do that easily by showing the ability to come to the same conclusions under listener-bias controlled conditions following open-ended evaluation. Why won't you? Why has no one else done so? Again, Tom, you give evidence that you either don't read or don't want to understand what I say. That is exactly the purpose of the evaluative sighted-evaluative blind leg of my proposed test. And I just repeated the reasons for that in the lates post you are just now responding to. Please re-read paragraph #1 above and tell me what about believing "blind results" (if they support your position) I am above believing? Please read your proposal. "When" bias controls are shown to give the same results that open tests would give then you'll accept them. No? Not at all. What my proposal said was if they validate the result, then the differences existed. If the differences go away under blind but otherwise identical evaluative testing, then clearly sighted bias was at work. But I also said that we would know that for sure because you weren't mixing in a second variable - that being a change in test. I have said this over and over since I first set out the proposed test, and you simply keep misrepresenting my view to fit your own bias about my position. What has not been "validated" are the conclusions you gain from non-controlled open listening as to acoustical cause. To imply that simply removing the sight of I/O terminals or otherwise hiding the answers somehow changes the acoustics of the situation (inducing masking) is a misdirection. If on the other hand, conclusions you hold are simply the result of a myriad of influences many of which have no acoustical cause why should anyone else but you care? As I have said repeatedly, one casual, undocumented, anecdotal case does not prove your case, however interesting. How about over two dozen performed prior to May 1990? Two dozen what? I'm talking about long, extended listening tests without comparative listening, but rather evaluative listening. You've only talked about one case that even comes close, and you've never shared details of that one. There is good reason for starting with sighted, open-ended evaluative listening...because that is where most audiophiles and reviewers start and make their judgments. Sure and that's where all Urban Legends start too. So? Of course, I'm not dismissing all listener observations either. It's just that those about amp/wire sound have been put to the test. You want to prove that it is sighted bias, you have to keep everything the same including the way they reach their conclusion, only changing the "blind". You haven't rigorously done that. Harry has amplifiers he feels meet the criteria of "better" sonics but he won't put himself to the experimental test of a simple bias controlled experiment. Instead he postulates a lengthy, costly experiment that is based on having the results "conform" to the results he already "knows" to be true before he'll accept them. Where did amplifiers enter into this? I talked about a sighted listening test done seven years ago when I chose an amplifier to replace my D90B. A choice I have been happy with ever since. So I'm to go out and buy two amps I don't like seven years later to prove to you that I can do it again, blind? Get real! In other words if a long listening biased controlled test doesn't support amp differences then he'll just say that it was masking real difference. Nope, I said I would do exactly that and support your conclusion *when* it is is part of a carefully designed control test. The one I outlined. This is the classic experimental dodge. Invent a needless experiment that will be unlikely to be conducted (with pre-held results that have to be confirned or the experiment would be "invalid" or "unconclusive") and you have the idea here. Who else has done this to make it a classic, Tom? Who else has the background in test design to have proposed this here on RAHE. I don't think so. So if that is erroneous, you have to *prove* it with rigorous testing...and you do that by doing the exact same test but "blind" instead of sighted. You do not do it by changing the test technique as well as going blind. My case all along is that you have switched two variables at once, yet you impute the difference in results to only one of those variables, while in fact the other variable may be at fault. How much clearer can I make it!! Oh you've made your case very clear. As long as you can avoid dealing with other bias controlled experimental results you'll be quite happy to continue debating. Nope, I'm not happy debating. I'd like to start setting up a test. But so far I can't even get serious suggestions to what to test (i.e. two component DUT's with fairly universal concurrence on both sides, i.e. objectivists universally accept that there will be no difference; subjectivists universally believe that there will be a difference.) |
|
#191
|
|||
|
|||
|
Does anyone know of this challenge?
wrote in message ...
Harry Lavo wrote: You may recall my discussion of food testing. Final testing was always done monadically, or proto-mondically (less frequent). Consumers were not "comparing", they were evaluating. The statistical analysis between the two (or more) sets of testees/variables was what determined if there was in fact a difference/preference. And on individual attributes as well as some overall satisfaction ratings, so the results could be understood in depth. The problem with comparing food testing with audio testing is that food has no standards for accuracy. Audio has always had well defined standards for accuracy and always will, and some people just don't want to deal with that fact. That's okay, except when bogus technical claims are made to justify a preference. We're not talking about accuracy here at all. We're talking about different ways of determining if their are statistically significant differences, and in what direction, between two DUT's. |
|
#192
|
|||
|
|||
|
Does anyone know of this challenge?
"Nousaine" wrote in message
... "Harry Lavo" wrote: ....many snips.... Bob Marcus" wrote in message My approach would let them listen however they want, for whatever they want, and then decide which component they prefer. No, it proposes that they have to make a left-brain "choice" as opposed to an evaluation. Big difference. In my test, I let the statistics make the "choice"...the audiophile never has to chose...he may prefer one on one characteristic and the other on another characteristic. If a statistically significant sampling indicates one way or the other, then that piece of equipment can be said to "differ" on that characteristic. Regardless of choice. What a good idea. But if the subject is unable to reliably identify which is which what difference would it make? If the subjects (there are sixteen-twenty, remember) are unable to reliably identify, then you won't get this result...you'll get random results and not statistically significant differences. We use this technique where blind tests are not possible (car audio systems) but ONLY when blind tests cannot be done. That's your choice. Why couldn't you have two identical cars with different systems, but the systems hidden? That is why I rejected the approach as I said at the time. It is inviting in its simplicity but it would not sway subjectivists including myself because of this flaw. What flaw? That people aren't able to "hear" acoustical differences when they are exposed to them? As far as I can tell the only complaint against bias controls is that "subjectivists" can no longer "hear" those things that are plain as day when they have the answer sheet in front of them. Then apply the bias controls to an evaluative test to see if the sighted differences perceived actually go away with blinding. That's what you need to do to prove your point. If it is sighted bias and not the change in test technique itself causing the descrepancy between common belief that there are differences and you own belief that most everything sounds the same (within...blah blah blah). Then it's left brained activity or some other supposed mechanism that is a problem. No subjectivist ever wonders why not a single one of them has ever been able to supply a single replicable experiment that demostrates the effects when some other party with a different or uninterested perspective is present and even modest bias controls are implemented. If I were an "audiophile" (which I'm not ... I'm an ardent enthusiast) I'd be wondering why my herd hasn't delivered any evidence instead of argument. Because it is difficult and time-consuming and requires many people to set up a definitive evaluative blind test, and for most audio hobbiests it simply isn't worth the effort. A good example of a subjectivist concocting a "flaw" out of thin air. No, you keep assuming the validity of your test when in fact what is needed is validation of it before you promulgate it to the world as "the answer". Actually Harry YOU keep assuming the validity of "your" test, which has never been validated to acoustical cause. Indeed many experiments have shown it to be quite unreliable. My test is only a test in that I take what people and reviewers are already doing that leads them to believe there are differences, and I change *one* variable -- blinding. I make no claims otherwise. I'll wait for the results. On the other hand, you and the other objectivists here are the one's pushing blind comparative testing as *the* definitive test whose "nulls" lead you to believe that differences for the most part do not exist. So it is incumbent upon you to show that this technique is beyond reproach...even for the open-ended evaluation of audio components. Which you simply have failed to do, instead relying on the assumption that it is appropriate because it is used in scientific audiology research. That leap of faith is just that..faith. Not science. Nor is introducing two variables simultaneously science. As for the "many other subjectivists here," I think you are being presumptuous. Finding flaws with bias-controlled tests seems to be part of what makes one a subjectivist. I see no reason to believe that your test, even if you could pull it off, would be any different. bob That's not how I read it/them at all. They may have some differences with me / my way of thinking, but the main problem they have is the *assumption* (unverified) that a test that is good for picking out small level differences in codec artecfact and other known acoustic anomalies is a suitable technique for open ended evaluation of audio components. The unverified assumption here is yours: that the human ear works differently when comparing sound from high-end components than when comparing sounds from other sources. No, that the ear-brain combination works differently when asked to evaluate a components sound on an open-ended basis versus having to choose between two components in a short timeframe. And my test is exactly designed to separate the effects of "blindness" from the effects of two different test techniques. Yours is not. Actually "your" proposed test is designed to obfuscate. First it can't be executed without a long expensive ritual. Furthermore you have repeated refused to use yourself as a first subject (who would be more appropriate?) Furthermore you have stated that unless "results" conform to those which you already hold (amps do sound different under long term open evaluation) then you won't accept them as validating blind tests. My test is not a one person test. It is a sixteen - twenty-five person or more test. I have said repeatedly that if this group is willing to participate in and help set up the test with me (and you) I will be first in line to be part of the test. Couldn't be clearer. IOW you've said that IF they give the same results as uncontrolled tests that you already hold to be "true" then you'll accept them. I have said no such thing -- *EVER* . It is your own misrepresentation repeated ad infinitum. Once that criticism is addressed, I think you will find most objections melt away if your assumptions prove true. But since the very idea of the test possibly being flawed is so threatening that even acknowledging the possibility seems beyond the objectivist ken, there is little movement on either side. We've posted our tests. Where are yours? Science is about more than quibbling over methodology, Harry. It's about actually doing experiments. Your standing in this debate will rise immeasurably when you actually do one. I'm trying to gain enough intellectual support on this forum to spend a year of my time going to the trouble to set up the control test with Tom, since it obviously cannot be done by one person but must be done by many. Why not 'start' with one? You seem to be avoiding the possibility that you cannot "hear" what you claim to hear when even modest bias controls are implemented. I've already answered that question. My taking a test I don't believe in up front would show me nothing at this point. What? That you can't "hear" your own amplifier with a cloth placed over the input/output terminals in your own reference system? Steve Zipser challenged people to visit Sunshine Stereo to demonstrate same. If the test is not a valid test, the results would be misleading now, wouldn't they. Once you agree to validate the test, I will agree to take part in it as part of the whole control test execution. I said I would do it when it is part of a control test First subject in your own test. How's that?. If you don't know by now then you clearly have not read my posts well enough to even understand what I have proposed. If that's the case, everything since is just posturing on your part. And BTW, I have done blind testing in the past on my own, but not rigorously. And sometimes I can hear differences. And sometimes not. Same as when testing sighted. But those are anecdotes...I am trying to help the group see what is needed to scientifically separate the two variables...comparative/evaluative and blind/non-blind. OK; what can you sometimes "hear" and what "not"? Not going to get suckered into that in this post....it is anecdotal and irrelevant to the subject at hand. |
|
#193
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"And how is this better than doing the evaluations and then using
statistical analysis to determine if "differences" hold up under blind conditions? I don't understand how you are going to do any statistical analysis without making a left brain "choice"." We are doing both at the same time. You do all your left brain stuff and record experiences of musicality etc. or whatever as notes. When finished you compare the record of what x was each timeyou did your "evaluation" thing. If there is a pattern to indicate difference was heard then the language of the notes should tend to match the record of a or b being heard. You can do a total non statistical analysis, using literary content, or word frequency, or whatever each note was to record as the experience. Now, if you are curious, a statical analysis of the record of each x can also be done. You are doing your "evaluation" and the objective test at the same time. You never have to make a choice, only make "evaluations " and record them at each x and then see if the content of the "evaluation" matches the record of x. Nodecision, no choice, no diversion, no stress, no forced answers; just the "meaning" of your "evaluation" for each x. A pattern clearly in the content of the notes should emerge. If item a is always "musically involving" then we have learned something, we have validated the "evaluation", as would be equally the case if that term happened all over the place with no clear connection. Again, if you want to do a statistical analysis, it can be done using your no choice left brain notion. |
|
#194
|
|||
|
|||
|
Subjectivist and Objectivist -- Are these misnomers? (WAS:
On 5/18/04 8:43 PM, in article D4yqc.22117$gr.1808882@attbi_s52, "Harry
Lavo" wrote: Oh you've made your case very clear. As long as you can avoid dealing with other bias controlled experimental results you'll be quite happy to continue debating. Nope, I'm not happy debating. I'd like to start setting up a test. But so far I can't even get serious suggestions to what to test (i.e. two component DUT's with fairly universal concurrence on both sides, i.e. objectivists universally accept that there will be no difference; subjectivists universally believe that there will be a difference.) Except I think most Audiophiles would fall in a spectrum between the two camps - the extremes being those that think that we have learned and can measure everything there is to know - and that system integration is not more difficult than comparing specification sheets (What we call "objectivists")- and those that feel that spec sheets are not what you hear - and that testing and analysis is useless unless it is done with listening to music (we call these folks "subjectivists"). It seems to me that the titles are misnomers to a large degree - and it is rare that people will be extreme to one degree or other. It is a little like engineering design - there are people that tend toward extensive simulation and those that iterate on the bench. The most talented engineers tend to be able to work in both worlds - since simulations generally tech you a lot about the principles and tend to show trends rather well- but more complicated simulations generally fall short of exact predictions (at least at RF frequencies) - at the end, on the bench, you need to get the circuits or system to behave as per our design targets. Without one or the other the design is incomplete! Kind of like getting a sheaf of data on a speaker - and buying it based solely upon those sheets without listening to it. Like plunking down a non refundable $2000/pr or something. So in a roundabout way -- my querstion is: Are the terms "subjectivists" and "objectivists" misnomers? Can a so-called "objectivist" get "lost in the woods" as badly as a so-called "subjectivist" but in a different way? |
|
#195
|
|||
|
|||
|
"Evaluation" test or chicken fat test, how to decide?
"Why is this so hard to understand? Especially for those of you who are
scientists. I'm not saying evaluative listening is "validated". I'm saying that is what is done. If you want to say it is all imaginary and will go away with blinding, then *that* is what you add as a variable...blinding. Not another, completely different type of test." And I say I always listen using chicken fat, that is what is done. To validate your "evaluation" test it must first be done with chicken fat, why change the variable I use by removing it, how can we know your new test gives completely spurious results in the absence of chicken fat? We need to go through a complete series with and then without chicken fat to know if your test is valid in the least or a blunted experience without benefit of chicken fat? |
|
#196
|
|||
|
|||
|
Subjectivist and Objectivist -- Are these misnomers? (WAS:
Bromo wrote:
On 5/18/04 8:43 PM, in article D4yqc.22117$gr.1808882@attbi_s52, "Harry Lavo" wrote: Oh you've made your case very clear. As long as you can avoid dealing with other bias controlled experimental results you'll be quite happy to continue debating. Nope, I'm not happy debating. I'd like to start setting up a test. But so far I can't even get serious suggestions to what to test (i.e. two component DUT's with fairly universal concurrence on both sides, i.e. objectivists universally accept that there will be no difference; subjectivists universally believe that there will be a difference.) Except I think most Audiophiles would fall in a spectrum between the two camps - the extremes being those that think that we have learned and can measure everything there is to know - and that system integration is not more difficult than comparing specification sheets (What we call "objectivists") That's not what we called objectivists. I would postulate that an objectivist, as far as this newsgroup is concerned, is one who believes in the validity of (a)standard controlled-bias testing like DBT's, and (b) measurements. - and those that feel that spec sheets are not what you hear - and that testing and analysis is useless unless it is done with listening to music (we call these folks "subjectivists"). I suggest you get your definitions straight. Check this webpage: http://www.dself.dsl.pipex.com/ampin...o/subjectv.htm In particular, pay attention to this: *** A short definition of the Subjectivist position on power amplifiers might read as follows: * Objective measurements of an amplifier's performance are unimportant compared with the subjective impressions received in informal listening tests. Should the two contradict the objective results may be dismissed out of hand. * Degradation effects exist in amplifiers that are unknown to engineering science, and are not revealed by the usual measurements. * Considerable latitude may be used in suggesting hypothetical mechanisms of audio impairment, such as mysterious capacitor shortcomings and subtle cable defects, without reference to the plausibility of the concept, or gathering any evidence to support it . *** |
|
#197
|
|||
|
|||
|
Subjectivist and Objectivist -- Are these misnomers? (WAS:
chung wrote in message news:LMBqc.22804$gr.1936664@attbi_s52...
Bromo wrote: On 5/18/04 8:43 PM, in article D4yqc.22117$gr.1808882@attbi_s52, "Harry Lavo" wrote: Oh you've made your case very clear. As long as you can avoid dealing with other bias controlled experimental results you'll be quite happy to continue debating. Nope, I'm not happy debating. I'd like to start setting up a test. But so far I can't even get serious suggestions to what to test (i.e. two component DUT's with fairly universal concurrence on both sides, i.e. objectivists universally accept that there will be no difference; subjectivists universally believe that there will be a difference.) Except I think most Audiophiles would fall in a spectrum between the two camps - the extremes being those that think that we have learned and can measure everything there is to know - and that system integration is not more difficult than comparing specification sheets (What we call "objectivists") That's not what we called objectivists. I would postulate that an objectivist, as far as this newsgroup is concerned, is one who believes in the validity of (a)standard controlled-bias testing like DBT's, and (b) measurements. - and those that feel that spec sheets are not what you hear - and that testing and analysis is useless unless it is done with listening to music (we call these folks "subjectivists"). I suggest you get your definitions straight. Check this webpage: http://www.dself.dsl.pipex.com/ampin...o/subjectv.htm In particular, pay attention to this: *** A short definition of the Subjectivist position on power amplifiers might read as follows: 1.* Objective measurements of an amplifier's performance are unimportant compared with the subjective impressions received in informal listening tests. Should the two contradict the objective results may be dismissed out of hand. 2.* Degradation effects exist in amplifiers that are unknown to engineering science, and are not revealed by the usual measurements. 3.* Considerable latitude may be used in suggesting hypothetical mechanisms of audio impairment, such as mysterious capacitor shortcomings and subtle cable defects, without reference to the plausibility of the concept, or gathering any evidence to support it . *** May I revise? 1. Measurements of an amplifier's performance are generally unimportant compared with the impressions received in extended listening tests informed by familiarity with live music of the same kind. Comparative listening sessions pitting amp against amp (when possible, using the acknowledged best available) should reveal the overall quality level of the product under consideration in relation to the state of the art. [Reasoning: All that matters is how it sounds in comparison to 'reality'.] 2. Degradation effects MAY exist in amplifiers that are unknown to engineering science, which are not revealed by the usual measurements. [Reasoning: Complex waveforms may behave in ways that are not entirely described by the usual methods.] I would not agree with point #3. I would also disagree with the adjective 'objective' (measurements) in point 1. There is no basis for claiming measurements are 'objective'. |
|
#198
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Harry Lavo wrote:
You seem to miss the part about having to make a conscious "choice" (a left brain function) versus simply evaluating the musicality of the equipment (a right brain choice). I am not convinced (because I have never been shown a shred of evidence) that they measure the same thing. Neither test 'measures' anything. Zilch. They are both subjective tests. That is why I have proposed the test the way I have. You cannot "assume" that a test that is valid for determining if a specific non-musical artifact can be heard is also valid for open-end evaluation of audio components. They are two different things entirely. I personally think the a-b-x test is even more suspect than a straight a-b. Something doesn't have to be audible to be subjectively musical. Right now, I just came up with a 'musical' idea in my head. A little motif if you will. I want to hear what it actually sounds like. I should try it out on the piano --- Ooops, when I did, I had to use the analytical part of my brain to do so. (you got to 'play the right notes at the right time'--- to quote J.S. Bach) So it must not be musical when I hear it for that reason. (according to Harry Lavo's hypothesis) And heavens, writing down music is SO 'left brain' --- that's even worse! Let's just be content with hearing what's in our heads --- everything else is probably not musical... According to your definition, it is logical to conclude that nothing is 'musical' because both composers and performers are constatnly switching 'left brain' and 'right brain' modes (at near light speed, most likely) when doing their activities. The fact is, it would be damn near impossible for them to do what they do if they DIDN'T. |
|
#199
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#200
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Stewart Pinkerton wrote:
On Tue, 18 May 2004 21:46:37 GMT, (Bruce J. Richman) wrote: I've often considered the objectivist viewpoint that "all competent amplifiers operating within their power ranges with appropriate speakers sound the same", etc. possibly true *for the measurable variables that they are interested in*, but nonetheless possibly not true - nor measurable by a-b or a-b-x tests - for the sound qualities that subjectivists are interested in. No doubt I'll be challenged on this view, but let me explain. First, explain which part of the quoted 'objectivist' standpoint has *anything* to do with anything *measurable*. These are based on controlled *listening* tests, and have nothing to do with measurements. The "controlled listening tests" obviously involve the listeners determining whether the DUT's sound the same or different. This is a form of measurement, although on a dichotomous basis rather than an interval scale. Every data point recorded in an ABX test or even in a more simple A/B comparison is obviously a measurement of the observer's ability to differentiate or not differentiate between the the 2 components being evaluated. When one reads a subjective review, or perhaps does one's own review either in a showroom or in one's home, one *might* be perceiving sonic qualities either not measured nor easily defined by the usual objectivist standards. For example, Harry has used the word "musicality". And I might use the same term, and others might make refernce to the imaging, soundstaging or *depth of field" qualities associated with a particular piece of equiopment. Still others may simply say "this sounds more realistic me" (than another component being compared). Fine - but does it actually sound *different* from the other component. If not, then expressions of preference based on sound quality are hardly relevant................ Again, for those that consider various technical specifrications or bias-controlled testing to be the one and only determinant of differences, I'm sure that this is *not* relevant. Hence, my comments about different frames-of=reference. However, for those subjectivists who choose to let their perceptions of difference play a role in choosing what components they use or purchase - of course, preferences are relevant. The cerntral point is that these components *may* sound different to *them*, and if asked how or why, they may describe perceptions that cannot easily be disregarded and/or ridiculed by traditional frequency or distortion variable measurements. I don't profess to know exactly how one would go about measuring, for example, differences in "imaging" or perceptions of "more body" in the sound of a particular component, for example, but it may well be that certain types of measurements might be available that could answer these questions. I suspect that for most subjectivists, however, following through on their preferences will remain preferable and all that is needed. Attempts at conversion via derision of their positions has, at least on RAHE, largely been a waste of time IMHO. While it may be perfectly acceptable to the objectivists to consider only variables that can be measured in terms of frequency response or various kinds of distortion, I would be reluctant - as I think would be most subjectivists - to attribute the various variables I've mentioned above to specific, capable of replication, measurements to measure these things. Also, how often, even within the frequency response realm, are complete graphs presented that *might* account for a particular component being perceived as relatively analytical, dark, or lean - all terms frequently used by subjectivists? This is a total strawman. I repeat, the 'objectivist' standpoint is based on *listening* tests, not measurements. In fact, I have always preferred the term 'reliable and repeatable subjectivist' for my own position, but that's rather long-winded! :-) It is certainly no strawman whatsoever. See my comments above re. listening tests themselves being binary measurements in which data is collected and used to support and promote their use. See also my comments about derision of subjectivist positions re. how components are preferred and presumably used. This is one of the reasons that I feel the 2 "camps" are really operating from almost totally different frames-of-reference and the endless challenges and disputes about the value of double blind testing, are, in practical terms, unlikely to convince anybody of anything they don't already strongly believe. While your premise is incorrect, I fear that your conclusion is accurate! :-( Both the premise and conclusion are quite likely correct. Different frames-of-reference, use of different variables in many cases (although not all), and above all, respect for individual preferences that are respected for what they are - no more, no less - are among the significant differences that apply to the 2 camps in this endless, useless debate (IMHO, of course).  Chacun a son gout! Jack's son should drink less port, then he wouldn't have gout.... -- Stewart Pinkerton | Music is Art - Audio is Engineering Bruce J. Richman |
| Reply |
| Thread Tools | |
| Display Modes | |
|
|
Similar Threads
|
||||
| Thread | Forum | |||
| ALL amps are equal?? | Car Audio | |||
| Light weight system challenge | Car Audio | |||
| Note to the Idiot | Audio Opinions | |||
| Mechanic blames amplifier for alternator failing?? Help>>>>>>>>>>> | Car Audio | |||
| Southeast Invitational Sound Challenge | Car Audio | |||
Linear Mode
