| Home |
| Search |
| Today's Posts |
|
#401
|
|||
|
|||
Doing an "evaluation" test, or has it already been done?
"John Atkinson" wrote in message
news:di7wc.41752$pt3.19036@attbi_s03... You claimed that 12% of the products reviewed in Stereophile were cables. As I showed, if you analyze the entire list of products reviewed by Stereophile, the actual figure is much lower. Correcting an error in my earlier posting -- I had inadvertently not included digital datalinks -- there were 175 cables reviewed out of a total of 3,557 components in those 41 years, meaning that "wires" account for 4.9%, not 12%. It might be informative to calculate the percentage of cable reviews for the last 30, 20 and 10 years. I'd be more interested in the trend. IOW, are cable reviews becoming increasingly or decreasingly frequent as time goes on? Norm Strong |
|
#402
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
On 6/5/04 1:31 PM, in article 7snwc.6115$4S5.4336@attbi_s52, "normanstrong"
wrote: 3,557 components in those 41 years, meaning that "wires" account for 4.9%, not 12%. It might be informative to calculate the percentage of cable reviews for the last 30, 20 and 10 years. I'd be more interested in the trend. IOW, are cable reviews becoming increasingly or decreasingly frequent as time goes on? Does it really matter, except if the number of cable reviews increases from today, I may reevaluate my subscriptions. |
|
#403
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"Bromo" wrote in message
news:Tvtwc.45414$pt3.23196@attbi_s03... On 6/5/04 1:31 PM, in article 7snwc.6115$4S5.4336@attbi_s52, "normanstrong" wrote: 3,557 components in those 41 years, meaning that "wires" account for 4.9%, not 12%. It might be informative to calculate the percentage of cable reviews for the last 30, 20 and 10 years. I'd be more interested in the trend. IOW, are cable reviews becoming increasingly or decreasingly frequent as time goes on? Does it really matter, except if the number of cable reviews increases from today, I may reevaluate my subscriptions. Hell, they practically give away Stereophile now anyway. Last offer was $9.95 for a year. I haven't bitten on the offers - got very burned out with Stereophile in the nineties with all the sorbathane pads, and TipToes, Peter Belt crapola, Tice Clocks and Directional Interconnects, and all the other High End Goofy ****. Whatta load of crap. I trust my ears, not their reviewers. Sam Tellig, Corey Greenburg, and a few columns from JGH were the only things that kept me hanging in there for four or five years. |
|
#404
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#406
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"normanstrong" wrote in message
news:7snwc.6115$4S5.4336@attbi_s52... "John Atkinson" wrote in message news:di7wc.41752$pt3.19036@attbi_s03... there were 175 cables reviewed out of a total of 3,557 components in those 41 years, meaning that "wires" account for 4.9%, not 12%. It might be informative to calculate the percentage of cable reviews for the last 30, 20 and 10 years. I'd be more interested in the trend. IOW, are cable reviews becoming increasingly or decreasingly frequent as time goes on? Hi Norm, I do not have the time to break out the data for the decades as you suggest, but my feeling is that the 1990 issues saw the greatest proportion of cables reviewed as a percentage of the total. You can find the raw data at http://www.stereophile.com/images/ma...dex/index.html if you want to take a crack at the statistical analysis. John Atkinson Editor, Stereophile |
|
#407
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"Bruce J. Richman" wrote in message
news:Ogmwc.44069$pt3.38393@attbi_s03. That said, I see nothing wrong with listing the components used as part of a product review. If anything, it serves as an implicit caveat that should suggest that those reading the review consider the test results in the context of the sources, electronics, and speakers involved. I would hope that those describing the results of level-controlled, bias-controlled testing would also list the components involved in their experiments. It is also fairly standard practice to list and describe the equipment in use when conducting evaluations, and indeed, most peer-reviewed journals would expect this to be done. The fact that it is also done for consumer-oriented publications such as Stereophile should be applauded, not condemned. I agree. There is nothing wrong with listing the ancillary equipment used by the reviewer. But it should stop at that point. The reviewer should never comment on the other components, or insinuate that they have an effect on the test results. Norm Strong |
|
#408
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
(John Atkinson)
Date: 6/6/2004 6:45 AM Pacific Standard Time Message-id: (Bruce J. Richman) wrote in message news:Ogmwc.44069$pt3.38393@attbi_s03... That said, I see nothing wrong with listing the components used as part of a product review. If anything, it serves as an implicit caveat that should suggest that those reading the review consider the test results in the context of the sources, electronics, and speakers involved. I would hope that those describing the results of level-controlled, bias-controlled testing would also list the components involved in their experiments. It is also fairly standard practice to list and describe the equipment in used when conducting evaluations, and indeed, most peer-reviewed journals would expect this to be done. The fact that is also done for consumer-oriented publications such as Stereophile and others should be applauded, not condemned. Amen to that Bruce. Thank you. I have yet to see any arguments from, for example, Tom Nousaine why a reviewer listing the ancillary components used when preparing a review is a bad thing. All he has claimed is that it "pleases advertisers." John Atkinson Editor, Stereophile I would think it might not please advertisers so much as Tom suspects. For every piece of equipment cited. One advertiser/potential advertiser is pleased while several competing advertisers/potential advertisers are left out and perhaps a bit displeased. |
|
#409
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"t.hoehler" wrote in message
... I...got very burned out with Stereophile in the nineties with all the sorbathane pads, and TipToes, Peter Belt crapola, Tice Clocks and Directional Interconnects, and all the other High End Goofy ****. You can find Stereophile's coverage of the Belt devices at http://www.stereophile.com/asweseeit/110/ Similarly, you can find Stereophile's coverage of the Tice clock at http://www.stereophile.com/asweseeit/787/ Whatta load of crap. Tice thought so, They canceled their advertising in Stereophile following our publication of these articles. But I'd be interested in learning what you found so dissatisfying about what we wrote. I trust my ears, not their reviewers. This is something we urge on our readers. What we offer is to be used as a guide, nort as received wisdom. John Atkinson Editor, Stereophile |
|
#411
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#413
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#414
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#415
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
But hifi mags aren't scientific in the least, most profoundly so in the
"audition" type of "review". Not only does it allow more exposure to similar "high end" gear, it promotes a sense that the user is really in the know and experienced by having the gear at hand; it's all part of the "mystic" that great pains are taken to cultivate. It is all part of the "real" product the mags sell. So too is the mindless listing of the music sources, ie. the "kid napper cover of the king macoo on the bando label ba 435x71d", in other words see how clever and in the know about the really cool insider stuff I display? As has been suggusted, a standard set of reference bits of various music and other sound sources used everytime would be so much more meaningful, other then the display of one's "hep" factor. Unless one is using gear not competent in the most crude way, the "night and day" revelations should jump right out and bite the "reviewer", unless it is a blind listening alone experience being reported; now we are talking the nbasement provisions for science that really make a difference. On that basis if there is a question the other gear is not competent, a simple test will reveal it so. Thank goodness you don't edit a magazine! To show the equipment used is a basic premise in any sort of scientific record keeping. Whether you agree with their conclusions, test conditions, etc, I cannot believe you are advising them to stop listing equipment used to "measure" the stuff! I could listen to a CD through the built in speakers of my iMac - and not liste the equipment and pass judgement. If people didn't see that I had good sound quality reproduction equipment - where is the credibility? |
|
#416
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
(Nousaine) wrote in message
news:rU9xc.14227$HG.12445@attbi_s53... (John Atkinson) wrote: Now you are asking that I limit my analysis to just the issues for which I have been responsible, though it is fair to point out that you made no such qualification when you claimed that "12%" of the products reviewed in Stereophile were cables, nor did you clarify until pressed that you were basing your "12%" figure on just one year's worth of Stereophile issues (2001). I specifically said that I examined one January list of prior years products and found that 12% of them were cables. Yes. I pointed out that you happened to choose a year where more cables had been written about than usual to derive your typical figure. You then cherry-picked another year... No. I mentioned another year's figure in passing to demonstrate that the year you chose was not representative. I made no claim that this other year was typical or representative. and then just added up all the issues back to day one which included 15 years of material which was not on your watch. As I said, if you are going to talk about the overall percentage of reviews in Stereophile that are devoted to cables, the overall statistics seeme a good point to start. As I also said, the correct figure for _all_ reviews is 5%, not the 12% you claimed. You then asked about the statistics for the reviews that had been published during my tenure as editor of Stereophile. This raises the proportion of cables reviews to 6%. Hardly a big difference, I suggest. It looks to me from your reference that all but one or two of the cable reviews occured during your watch. This is what I said. That the first cable review to appear in Stereophile was in 1978, 16 years after the magazine's first issue, and that the highest proportion of cable reviews appeared in the 1990s. So what? As I have offered the statistical analysis of the reviews that were published "during [my] watch," and that difference between that period and the overall period is 6% vs 5%, both significantly less than the 12% you claimed, I fail to understand your point. I have snipped most of the rest of your response Mr. Nousaine as it involves questions that have already been answered. But there are two points that should be addressed: 1: On the matter of what Larry Klein said at the 1990 AES Conference, I made notes at the time of his words and am confident that those notes reflect what he said. However, when you wrote in the following exchange that there was "a negotiation between Hirsch and a manufacturer": Mr. Klein specifically mentioned Julian Hirsch's reviews, where he would discuss negative comments made by Julian with the manufacturer of the product after they had been sent a preprint. If Larry and the manufacturer could not reach agreement on how the review should be worded, then Larry would cancel publication of the review. I don't recall him saying exactly that. That there was a negotiation between Hirsch and a manufacturer. Please note that Mr. Klein made it clear that Julian was not involved in these discussions, that it was Mr. Klein who would negotiate the final wording of Julian's review with the manufacturer, not Julian himself. And 2: You keep trying to pretend you were talking about Stereophile's Recommended Components listing when Dr. Richman's original statement and your original response concerned _reviews_. But on that subject, you then wrote: publishing a Recommended Components List that includes products that might not even exist or have not been evaluated (NR) I don't know which magazine you are referring to here, Mr. Nousaine. Every product recommended in Stereophile's listing exists and has been evaluated. The "NR" to which you refer means that while the product has indeed been evaluated, a review has not been published. as far as I know Hachette magazines don't recommend products they've only "heard about." :-) I don't [know] that Stereo Review did either. And neither does Stereophile, Mr. Nousaine. And whom are you quoting when you put the words "heard about" in quotes? Certainly not me, if that is the impression you wished to leave. Look, Mr. Nousaine, if you are serious about making it your mission to criticize magazines that compete with your own, why don't you investigate the fact that some reviewers (not Stereophile's) act as paid consultants for manufacturers whose products they review? Why don't you investigate a magazine (not Stereophile) whose current policy is to sell its cover to an advertiser? Surely these subjects are more worthy of your attention than the the proportion of cables that have been reviewed in Stereophile? John Atkinson Editor. Stereophile |
|
#417
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#418
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Bromo wrote:
On 6/7/04 6:34 PM, in article , "Nousaine" wrote: I've thought about this. IMO unless the ancillary equipment has a bearing on the performance of the product being tested (and there's no evidence that competent electronic devices and accessories such as cabling) then such a listing reduces the space available for the product actually being reviewed (or others, or other content.) So from an outsiders view such a listing can only serve three possible purposes: 1. It's a subliminal advertisement for the listed accessories. 2. It attempts to raise the status and self-image of the reviewer. and finally 3. It helps keep the wire/amp/tweak mythology going. I think it's a waste of valuable space. Thank goodness you don't edit a magazine! To show the equipment used is a basic premise in any sort of scientific record keeping. Whether you agree with their conclusions, test conditions, etc, I cannot believe you are advising them to stop listing equipment used to "measure" the stuff! I could listen to a CD through the built in speakers of my iMac - and not liste the equipment and pass judgement. If people didn't see that I had good sound quality reproduction equipment - where is the credibility? But what does listing these accessories in a speaker review ( Richard Gray's Power Company Substation isolation transformer, 1200s, 600s, & Pole Pig power conditioners; Sounds of Silence Vibraplane active isolation platform; Symposium Ultrashelf, Finite Elemente equipment stands' Audioharma Cable Cooker; Walker Precision Isolated Motor Drive, ACS Tube Traps, Shakti Stones & On-Lines; PRG, BAD, Abbfusor panels as accessories; in addition to 7 assorted cables) do for anyone's credibility? What does the the multiple cartridged, 2 digital sources, 2 preamplifiers and 2 different power amplifiers tell us except raise questions about how long each piece was actually used in listening? It appears to me that 1) most of this physical equipment has nothing to do with the sound of the speakers being evaluated 2) much of it may not have even been used for any length of time with the speakers; and 3) much of it detracts from credibility and is intended to keep the mythology. alive. So I agree with you. I'm glad competing publications employ this window dressing :-) |
|
#419
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#421
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Bromo wrote:
On 6/8/04 6:29 PM, in article , " wrote: Thank goodness you don't edit a magazine! To show the equipment used is a basic premise in any sort of scientific record keeping. Whether you agree with their conclusions, test conditions, etc, I cannot believe you are advising them to stop listing equipment used to "measure" the stuff! I could listen to a CD through the built in speakers of my iMac - and not list the equipment and pass judgment. If people didn't see that I had good sound quality reproduction equipment - where is the credibility? But hifi mags aren't scientific in the least, most profoundly so in the "audition" type of "review". Not so important. If I read someone who says "the recording was bass light" - and it was Jaco Pastorious - I would wonder if the reviewer had a system incapable of producing bass, or if the recording really was bad - the list of equipment would remove the former wonder. Same thing - if someone has a SET amp - and said the same thing about a record player - by seeing the equipment, I will be able to judge for myself if the system would be capable of reproducing the kind of music I like -- i.e. How relevant is the opinion. To me, the conditions of the review as well as the author are pretty important - even if the goal is not supposed to be scientific. In addition to the uses of equipment listing you mention, I think there is another that is of potential value to some readers. To the extent that the equipment listed may be somewhat similar to that of the reader, the reader might be able to at least make some educated guesses - admittedly subjective and hypothetical as they may be - as to how a particular piece of gear might work in his *own* system. I, too, have one gripe with components listings however. Actually, more with components *used* than with components listed. I would like to see the reviewers make a greater attempt to use system products more likely to be used with the product under review than is sometimes done. While "state-of-the-art" products, often quite expensive, may provide a greatewr degree of credibility in some cases for the reviewer and/or readers (depending on their faith in the product), I'm not sure that, for example, using a $ 10,000++ pair of speakers, or an extremely expensive amplfier or CD player or turntable with a product under review that does not fall into the same price category does the readers a service. Granted, I've seen some reviewers explicitly attempt to put together systems when reviewing a product that don't exhibit a financial imbalance, but all too often, I find the "associated equipment" list dominated by relatively high price electronics and/or sources - *irrrespective* of the fact that the product being reviewed may not typically be used by people with access to the "high price stuff". Not only does it allow more exposure to similar "high end" gear, it promotes a sense that the user is really in the know and experienced by having the gear at hand; it's all part of the "mystic" that great pains are taken to cultivate. That may be how you use it - but not me. I find revierwers are all too human - and since many lurk on this group, I will leave it at that. I certainly don't agree with all of them, and from some equipment, I can see how some come to a conclusion I might not. It is all part of the "real" product the mags sell. So too is the mindless listing of the music sources, ie. the "kid napper cover of the king macoo on the bando label ba 435x71d", in other words see how clever and in the know about the really cool insider stuff I display? I could not disagree much more than I do - I think if you are familiar with the music being played - it is not some sort of show off - but if you know of similar or the same music - you could perform a similar test for yourself. I am not sure why you have a chip on your shoulder about this - most of the tunes they play - they aren't hard to find if you care to... In many cases, the music being suggested is not particularly rare, collectible, or IMHO, representative of anybody going on an ego trip. To assume psychological motivations not in evidence, is highly speculative in my professional opinion. It would appear more likely that the reviewer is simply using LPs or CDs (redbook or high-rez) from his collection that he finds useful for evaluating various pieces of equipment. I find this no different than the old, frequently given advice, that when people audition any product, they use music sources with which they are familiar. As has been suggusted, a standard set of reference bits of various music and other sound sources used everytime would be so much more meaningful, other then the display of one's "hep" factor. Agreed. Given the number of classical music recordings, for example, that are often cited, arguments re. "hipness" would seem to be malplaced. When was the last time any Beethoven recordings created a stir on Billboard or found their way onto a Napster listing?  Please! Some of the music they use is so un-hip it is remarkable their pants hold up! :-) Seriously, though, I don't think the magazine is some sort of "show off" platform for the reviewer. AT least to me, they seem no cooler (and in many times less cool) than in the start of the article. Take for example T$S - I subscribe, though with the music reviews I have not found much that I like save 1 or 2 CD's - the musical tastes are a bit too ... Old to me. But one example that supports your hypothesis in T$S, at least: And one reviewer actually had the cajones to publish live music they attended for the last year in the "recommended recordings" issue - absolutely zero use to me and only displayed that the reviewer got to see a lot more live music than most people (though to be honest, sounded like they were good ones). Unless one is using gear not competent in the most crude way, the "night and day" revelations should jump right out and bite the "reviewer", unless it is a blind listening alone experience being reported; now we are talking the nbasement provisions for science that really make a difference. On that basis if there is a question the other gear is not competent, a simple test will reveal it so. So..... Your complaint is that they review equipment in a way that is not scientific, and use recordings you aren't familiar with, and claim large differences where you aren't sure there is any difference between equipment that is "competently" designed (define competent)... I would surmise your issue is not with magazine reviews, but the whole concept of hifi. Bruce J. Richman |
|
#422
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Nousaine wrote:
Bromo wrote: Take for example T$S - I subscribe, though with the music reviews I have not found much that I like save 1 or 2 CD's - the musical tastes are a bit too ... Old to me. But one example that supports your hypothesis in T$S, at least: And one reviewer actually had the cajones to publish live music they attended for the last year in the "recommended recordings" issue - absolutely zero use to me and only displayed that the reviewer got to see a lot more live music than most people (though to be honest, sounded like they were good ones). Actually those were only the pop/rock/blues perfformances I attended in the last half of 2003. I realize that wasn't made clear in the copy. But the buried-content was the Hearing Protection message about protecting yourself at these kind of events and a specific recommendation about the amount I suggest would be appropriate for events such as the ones described. It was also a hidden travelog about summer/fall outdoor venues available in the midwest (Michigan, Minnesota) hoping that it would encourage midwesterners to take advantage of some good stuff this summer. The reason I did this is because, like you, I seldom find listings of Recommended Recordings that useful. Note to T$S : *enough* already with the Mahler! ; -- -S. Why don't you just admit that you hate music and leave people alone. -- spiffy |
|
#423
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
(John Atkinson) wrote:
(Nousaine) wrote in message news:rU9xc.14227$HG.12445@attbi_s53... (John Atkinson) wrote: Now you are asking that I limit my analysis to just the issues for which I have been responsible, though it is fair to point out that you made no such qualification when you claimed that "12%" of the products reviewed in Stereophile were cables, nor did you clarify until pressed that you were basing your "12%" figure on just one year's worth of Stereophile issues (2001). I specifically said that I examined one January list of prior years products and found that 12% of them were cables. Yes. I pointed out that you happened to choose a year where more cables had been written about than usual to derive your typical figure. You then cherry-picked another year... No. I mentioned another year's figure in passing to demonstrate that the year you chose was not representative. I made no claim that this other year was typical or representative. and then just added up all the issues back to day one which included 15 years of material which was not on your watch. As I said, if you are going to talk about the overall percentage of reviews in Stereophile that are devoted to cables, the overall statistics seeme a good point to start. As I also said, the correct figure for _all_ reviews is 5%, not the 12% you claimed. You then asked about the statistics for the reviews that had been published during my tenure as editor of Stereophile. This raises the proportion of cables reviews to 6%. Hardly a big difference, I suggest. It looks to me from your reference that all but one or two of the cable reviews occured during your watch. This is what I said. That the first cable review to appear in Stereophile was in 1978, 16 years after the magazine's first issue, and that the highest proportion of cable reviews appeared in the 1990s. So what? As I have offered the statistical analysis of the reviews that were published "during [my] watch," and that difference between that period and the overall period is 6% vs 5%, both significantly less than the 12% you claimed, I fail to understand your point. Sure; but those wishing to downplay the significance of cabling may fail to recognize the omportance of cabling in other ways, such as the Recommended Component List and implied reviews of cabling and it's effect in the context of reviews of other products. IMO in the whole context of your publication cabling may have a higher significance because they get mention in the evaluation of other products. That's just my opinion, of course, but it comes from a perspective of a consumer, a subscriber as well as a contributor to a competitor. Further I think that observers as well as readers have been enlightened by an examination of these issues. IMO cabling carries a strong underpinning of high-end audio and an examination of the content of your magazine points this out. I urge anybody with interest to make up their own mind. I have snipped most of the rest of your response Mr. Nousaine as it involves questions that have already been answered. But there are two points that should be addressed: 1: On the matter of what Larry Klein said at the 1990 AES Conference, I made notes at the time of his words and am confident that those notes reflect what he said. However, when you wrote in the following exchange that there was "a negotiation between Hirsch and a manufacturer": Mr. Klein specifically mentioned Julian Hirsch's reviews, where he would discuss negative comments made by Julian with the manufacturer of the product after they had been sent a preprint. If Larry and the manufacturer could not reach agreement on how the review should be worded, then Larry would cancel publication of the review. I don't recall him saying exactly that. That there was a negotiation between Hirsch and a manufacturer. Please note that Mr. Klein made it clear that Julian was not involved in these discussions, that it was Mr. Klein who would negotiate the final wording of Julian's review with the manufacturer, not Julian himself. Thank you for pointing this out. And 2: You keep trying to pretend you were talking about Stereophile's Recommended Components listing when Dr. Richman's original statement and your original response concerned _reviews_. No pretense. I think that restricting the data discussion to items "reviewed" understates the relative importance of cabling in your publication. This has been made even more clear since I've noticed that approximately 10% of your Recommended Component List is cabling and you've pointed out that those products don't need to have a formal review to be Recommended. Further your accessory list that accompanies all reviews highlights them as well. But on that subject, you then wrote: publishing a Recommended Components List that includes products that might not even exist or have not been evaluated (NR) I don't know which magazine you are referring to here, Mr. Nousaine. Every product recommended in Stereophile's listing exists and has been evaluated. The "NR" to which you refer means that while the product has indeed been evaluated, a review has not been published. You acknowledged in this newgroup that your RCL listed a product (Grado cartridge) that never existed and had never been reviewed did you not? This was not pointed out by me either. as far as I know Hachette magazines don't recommend products they've only "heard about." :-) I don't [know] that Stereo Review did either. And neither does Stereophile, Mr. Nousaine. And whom are you quoting when you put the words "heard about" in quotes? Certainly not me, if that is the impression you wished to leave. Referent prior comment above Look, Mr. Nousaine, if you are serious about making it your mission to criticize magazines that compete with your own, why don't you investigate the fact that some reviewers (not Stereophile's) act as paid consultants for manufacturers whose products they review? Why don't you investigate a magazine (not Stereophile) whose current policy is to sell its cover to an advertiser? Surely these subjects are more worthy of your attention than the the proportion of cables that have been reviewed in Stereophile? John Atkinson Editor. Stereophile I'm guessing that you are bringing up The Audio Critoc and Fourier loudspeaker issue again. That's been covered in the newsgroups in depth. The Grado revelation is new. As far as other instances I know of none offhand. Peter Mitchell consulted and evaluated products but as far as I know he didn't review his own work. Or did you have some other instances that you would like to share? Time and resources permitting I will conduct independent evaluations for manufacturers on contract and and DIY enthusiasts but I do not now nor have I ever consulted in design with a manufacturer. |
|
#424
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
On 6/9/04 12:38 AM, in article jvwxc.6293$0y.1992@attbi_s03, "Nousaine"
wrote: I could listen to a CD through the built in speakers of my iMac - and not liste the equipment and pass judgement. If people didn't see that I had good sound quality reproduction equipment - where is the credibility? But what does listing these accessories in a speaker review ( Richard Gray's Power Company Substation isolation transformer, 1200s, 600s, & Pole Pig power conditioners; Sounds of Silence Vibraplane active isolation platform; Symposium Ultrashelf, Finite Elemente equipment stands' Audioharma Cable Cooker; Walker Precision Isolated Motor Drive, ACS Tube Traps, Shakti Stones & On-Lines; PRG, BAD, Abbfusor panels as accessories; in addition to 7 assorted cables) do for anyone's credibility? You know they aren't listening through something that will be nasty for the sound, like an iMac. What does the the multiple cartridged, 2 digital sources, 2 preamplifiers and 2 different power amplifiers tell us except raise questions about how long each piece was actually used in listening? You have a point there - though with either pieces, you should see that it would give a reasonably accurate representation of the stuff. Though it begs the question of what "acssociated equipment" was actually used, though. I am beginning to agree - perhaps there is an element of "product placement" in the column. I still would prefer to see the list of equipment used in the rig... It appears to me that 1) most of this physical equipment has nothing to do with the sound of the speakers being evaluated 2) much of it may not have even been used for any length of time with the speakers; and 3) much of it detracts from credibility and is intended to keep the mythology. alive. |
|
#426
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Bromo wrote:
Mr. N is of the camp (The "if I can't measure it it doesn't exist" crowd) which says all CD's sound the same, all amps sound the same, as do all other components I am sure -- and 16 ga, zip cord is as good as any other more elaborate cables. He is not this way at all. He basically DOES say that if you cannot hear differences between CD players (I assume you mean this and not CD recordings themselves, which for obvious marketing differences nearly always sound different), amps, wires, etc. with a blind or double-blind protocol and precise level matching then differences do not exist - at least for the person doing the comparison. If enough people do comparisons like that and they all cannot hear differences, then it is sensible to assume that differences are essentially inaudible for everyone, or at least nearly everyone. I suppose some super golden ear might hear differences, but there are few of those in audioland. Most are under ten years of age. And even if an occasional golden ear does perceive a difference when doing some very, very careful critical listening, it is likely that when listening is done for musical values and not to determine hi-fi product quality any differences would be inconsequential. On the other hand, some enthusiasts insist that they can hear differences, and when pressed as to how they know there are differences when they are doing their comparing sighted and probably not level matched, they indicate that it is obvious that a super duper amp (or CD player, or set of wires, or whatever) just HAS to be superior to more mainstream versions for reasons that defy clear explanation. However, this is not a good (or at least practical) way to obtain really good components or avoid spending money on overkill (and overpriced) versions. Why does this matter? Well, the less you spend on components, the more money you have left over to spend on recordings. Incidentally, I have compared 16 AWG lamp cord to some pretty serious high-end speaker wires (Dunlavy LCR Ultra, which costs a grand for twelve feet, as well as Dunlavy Z6, a very special low capacitance wire). Even when comparing those Dunlavy wires to 16-AWG versions 24 feet long I could not hear differences. I used both Dunlavy speakers and others by NHT and Waveform. OK, that works for me and I am only one guy. However, I would at least expect someone who claims that they could hear differences to do so by means of a blind or double-blind protocol. Level matching is no big deal with speaker wires, by the way (at reasonable lengths, at least), and so doing blind comparisons with different versions is easier than comparing amps or comparing CD players. To dramatize the lack of impact with wires, it is best to compare some highly regarded high-end versions with that good-old 16 AWG wire you mentioned. Differences should jump right out at you - or will they? Howard Ferstler |
|
#427
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#428
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
On 6/9/04 9:07 AM, in article 7YDxc.502$eu.262@attbi_s02, "Nousaine"
wrote: Bromo wrote: Take for example T$S - I subscribe, though with the music reviews I have not found much that I like save 1 or 2 CD's - the musical tastes are a bit too ... Old to me. But one example that supports your hypothesis in T$S, at least: And one reviewer actually had the cajones to publish live music they attended for the last year in the "recommended recordings" issue - absolutely zero use to me and only displayed that the reviewer got to see a lot more live music than most people (though to be honest, sounded like they were good ones). Actually those were only the pop/rock/blues perfformances I attended in the last half of 2003. I realize that wasn't made clear in the copy. But the buried-content was the Hearing Protection message about protecting yourself at these kind of events and a specific recommendation about the amount I suggest would be appropriate for events such as the ones described. I looked it up and saw you wrote that bit -- after re-reading it I saw your earplug recommendations! The whole thing makes more sense to me now, though it was a bit of a risky proposition for such a rag... It was also a hidden travelog about summer/fall outdoor venues available in the midwest (Michigan, Minnesota) hoping that it would encourage midwesterners to take advantage of some good stuff this summer. That part I missed - might be good to beat it over my head next time! The reason I did this is because, like you, I seldom find listings of Recommended Recordings that useful. Yeah, that is a problem - tastes vary so much it is hard to narrow down. I quite like HiFi+ reviews - they tend to be along my teastes better, though the very best is the Bop Shop in downtown Rochester - the guys fall all over themselves to help you find something you might like. |
|
#429
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
From: (Nousaine)
Date: 6/8/2004 9:38 PM Pacific Standard Time Message-id: jvwxc.6293$0y.1992@attbi_s03 Bromo wrote: On 6/7/04 6:34 PM, in article , "Nousaine" wrote: I've thought about this. IMO unless the ancillary equipment has a bearing on the performance of the product being tested (and there's no evidence that competent electronic devices and accessories such as cabling) then such a listing reduces the space available for the product actually being reviewed (or others, or other content.) So from an outsiders view such a listing can only serve three possible purposes: 1. It's a subliminal advertisement for the listed accessories. 2. It attempts to raise the status and self-image of the reviewer. and finally 3. It helps keep the wire/amp/tweak mythology going. I think it's a waste of valuable space. Thank goodness you don't edit a magazine! To show the equipment used is a basic premise in any sort of scientific record keeping. Whether you agree with their conclusions, test conditions, etc, I cannot believe you are advising them to stop listing equipment used to "measure" the stuff! I could listen to a CD through the built in speakers of my iMac - and not liste the equipment and pass judgement. If people didn't see that I had good sound quality reproduction equipment - where is the credibility? But what does listing these accessories in a speaker review ( Richard Gray's Power Company Substation isolation transformer, 1200s, 600s, & Pole Pig power conditioners; Sounds of Silence Vibraplane active isolation platform; Symposium Ultrashelf, Finite Elemente equipment stands' Audioharma Cable Cooker; Walker Precision Isolated Motor Drive, ACS Tube Traps, Shakti Stones & On-Lines; PRG, BAD, Abbfusor panels as accessories; in addition to 7 assorted cables) do for anyone's credibility? Plenty for some, nothing for some others, including you. However it seems to me that the listing of room treatment for a speaker review is quite important. by the looks of your post it seems you might not agree. I will keep this in mind should I read a speaker rview by you. What does the the multiple cartridged, 2 digital sources, 2 preamplifiers and 2 different power amplifiers tell us except raise questions about how long each piece was actually used in listening? Why does that matter to you if you think they make no difference and shouldn't even be listed in the first place? If you are not interested in what equipment the reviewer is using why now complain about how long he or she is using it in his or her review? It appears to me that 1) most of this physical equipment has nothing to do with the sound of the speakers being evaluated That is one opinion out of many. The magazine is geared towards a range of audiophiles not towards Tom Nousaine. 2) much of it may not have even been used for any length of time with the speakers; And this is a problem for someone who believes such equipment doesn't matter because? Assuming your assumption is actually correct. and 3) much of it detracts from credibility and is intended to keep the mythology. alive. IYO. So I agree with you. I'm glad competing publications employ this window dressing :-) Nice to know your critiques of Stereophile are impartial :-) |
|
#431
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#432
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Bromo wrote:
On 6/9/04 7:54 PM, in article , "Nousaine" wrote: Thank goodness you don't edit a magazine! To show the equipment used is a basic premise in any sort of scientific record keeping. Whether you agree with their conclusions, test conditions, etc, I cannot believe you are advising them to stop listing equipment used to "measure" the stuff! I never suggested anybody do anything they didn't want to do. I'm merely suggesting that such a listing looks, to me, more like a new form of advertisement ormerchandising tool rather than simply disclosure. That would be "product placement" - and I am sure some of it happens, though, I imagine most of the time it doesn't. Simply put, if the Stereophile magazine would consistanly pan products that were good, and praise bad products, they would quickly ind themselcves with falling subscribership. You mean that Stereophile is courting its stakeholders (subscribers and advertisers)? Of course, they do a good job of that as well. But, so did Jim Baker. Just because your audience believes in mythology and you cater to that doesn't make your behavior any more commendable or immune to description. Let me put this in another light. The June 2004 issue lists nothing but branded specialty products as system/accessory components. Many, if not most, of which were probably supplied as loans by manufacturers. IMO if merchandising/promotional considerations were not primary factors then I'd expect disclosure on which products were owned and purchased by reviewers. Or a statement of disclosure on what policy exists for loans and related system components. I'm not against such listings at all. I just think that people should have a balanced perspective on how and why systems get assembled for reviewers. In the Stereophile case it appears to me that this type of listing is more related to selling products, magazines and advertising and maintaining self-confidence of reviewers than than it is to insuring review system quality. |
|
#433
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
"So..... Your complaint is that they review equipment in a way that is not
scientific, and use recordings you aren't familiar with, and claim large differences where you aren't sure there is any difference between equipment that is "competently" designed (define competent)... I would surmise your issue is not with magazine reviews, but the whole concept of hifi." The concern of science in reviews was yours, to which I responded that same is not to be found in the "audition" "review" typical in hifi rags. I maintain that such rags are selling an image, thus the kneejerk listing of gear, including the cryogenic treated toothpicks used after snacking during listening and the easy insertion of music used to do what is not in the least science but an almost sacred included part of the ritual to evoke the image that is the product being sold. If one knows the music or not is irrelevant, it is only another variable added to a hopelessly jumbled list, including toothpicks, that provides absolutly no basis for comparison unless one has the exact list and listening venue. Given the now established benchmark that amps/wire can not be demonstrated to sound different by listening alone, the night and day reported differences are too part of the "audition" rituall, and on that basis I am very sure the reported perceptions so vital a part don't have an analog in reality.. I have the feeling your surmise that I'm not in tune with the "real" hifi scene assumes in the first place the pedestal placement now given the ritual is any way in the least a way to answer questions about hifi realities. So if hifi is the ritual and then we go from there, you are correct, I reject the unfounded "concept of hifi" without reservation. While I'm on this rant against hifi rags, the use of prices sprinkeled so liberaly around in "auditions" is too part of the ritual and image that is the real product and the implied price as measure of anything a bit of theology by which the ritual and image cann't be sustained. If as I think reality, that performance on a listening paraty is the norm and not related to price, then the ritual and image is trivial and irrelevant and the reason for these rags to exist outside the theology irrelevant as well. I read the online version of some of them for technical information and news of products, and I tolerate the "auditions" in so far as it is required to gleen the above from the noise in the system. |
|
#434
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#435
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Tom Nousaine ) wrote in message
... (John Atkinson) wrote: (Nousaine) wrote in message news:rU9xc.14227$HG.12445@attbi_s53... (John Atkinson) wrote: Now you are asking that I limit my analysis to just the issues for which I have been responsible, though it is fair to point out that you made no such qualification when you claimed that "12%" of the products reviewed in Stereophile were cables, nor did you clarify until pressed that you were basing your "12%" figure on just one year's worth of Stereophile issues (2001). I specifically said that I examined one January list of prior years products and found that 12% of them were cables. Yes. I pointed out that you happened to choose a year where more cables had been written about than usual to derive your typical figure. No response from Tom Nousaine. This is a serious point: that if Mr. Nousaine is presenting statistical information to support his point, his picking and choosing among the data is both misleading and bad science. You then cherry-picked another year... No. I mentioned another year's figure in passing to demonstrate that the year you chose was not representative. I made no claim that this other year was typical or representative. No response from Tom Nousaine. This is another serious point. I made no such comment along these lines, even though he has now repeatedly claimed on this newsgroup that I have done so. Here again is the text of mine to which he is referring, in a message posted on May 28: Looking at the raw data, Stereophile wrote about 19 cables of various kinds in 2001 out of a total of 163 components reviewed, ie, 11.65% Of course, you could have chosen 2002 for your analysis, when Stereophile reviewed just 3 cables out of 143 components reviewed, but that wouldn't have supported your argument, would it Mr. Nousaine? As I said, "data dredging." As I wrote, I made no claim that the incidence of cable reviews published in Stereophile in 2002 was typical. I only instanced that year to draw attention to the fact that Tom Nousaine's choice of 2001 on which to base his analysis was data dredging. and then just added up all the issues back to day one which included 15 years of material which was not on your watch. As I said, if you are going to talk about the overall percentage of reviews in Stereophile that are devoted to cables, the overall statistics seeme a good point to start. As I also said, the correct figure for _all_ reviews is 5%, not the 12% you claimed. No response from Tom Nousaine. In response to my offering the overall statistics for cable reviews, he pretended first that he was referring to just one year, then that he was talking about just the issues that I have edited to derive his 12% figure. Surely a reasonable person might assume that Mr. Nousaine would accept the correction and move on? Perhaps Mr. Nousaine is not as reasonable nor as disinterested a commentator as he might have us believe. You then asked about the statistics for the reviews that had been published during my tenure as editor of Stereophile. This raises the proportion of cables reviews to 6%. Hardly a big difference, I suggest. Again no response from Tom Nousaine. One wonders why he asks so many questions if he is not interested in the answers? It looks to me from your reference that all but one or two of the cable reviews occured during your watch. This is what I said. That the first cable review to appear in Stereophile was in 1978, 16 years after the magazine's first issue, and that the highest proportion of cable reviews appeared in the 1990s. So what? As I have offered the statistical analysis of the reviews that were published "during [my] watch," and that difference between that period and the overall period is 6% vs 5%, both significantly less than the 12% you claimed, I fail to understand your point. Sure; but those wishing to downplay the significance of cabling may fail to recognize the [importance] of cabling in other ways, such as the Recommended Component List and implied reviews of cabling and its effect in the context of reviews of other products. I assume you wish to downplay the "significance of cabling," Mr. Nousaine because you believe that cables, provided they are of the appropriate length, gauge, and construction, don't affect sound quality. But all I am doing in this thread is responding to your specific statement, in message , that: A study of one of the last few years Reviewed Components for the previous years showed 12% were cabling, more than any other single product category except digital components and loudspeakers. Note again that you didn't say one specific year, Mr. Nousaine, nor did you restrict your analysis to the years during which I edited Stereophile. Your actual words were "last few years." IMO in the whole context of your publication cabling may have a higher significance because they get mention in the evaluation of other products. That's your opinion, Mr. Nousaine, and I see no reason to argue with it. As I said, all I am doing is addressing your specific point concerning the proportion of _reviews_ of cables published in Stereophile. As I have shown, to derive your "12%" of reviews that you decribed as referring to a "few years," you chose a specific single year (2001) that was untypical. However you wish to describe this -- "cherry picking," "data dredging" -- it is bad science on your part. IMO cabling carries a strong underpinning of high-end audio and an examination of the content of your magazine points this out. I urge anybody with interest to make up their own mind. As do I. Mr. Klein specifically mentioned Julian Hirsch's reviews, where he would discuss negative comments made by Julian with the manufacturer of the product after they had been sent a preprint. If Larry and the manufacturer could not reach agreement on how the review should be worded, then Larry would cancel publication of the review. I don't recall him saying exactly that. That there was a negotiation between Hirsch and a manufacturer. Please note that Mr. Klein made it clear that Julian was not involved in these discussions, that it was Mr. Klein who would negotiate the final wording of Julian's review with the manufacturer, not Julian himself. Thank you for pointing this out. I have not written otherwise. _You_ were the one that said that Julian was involved in such negotiations, Mr. Nousaine. You keep trying to pretend you were talking about Stereophile's Recommended Components listing when Dr. Richman's original statement and your original response concerned _reviews_. No pretense. I think that restricting the data discussion to items "reviewed" understates the relative importance of cabling in your publication. So why then did _you_ make that restriction, Mr. Nousaine? Here again are your exact words: A study of one of the last few years Reviewed Components for the previous years showed 12% were cabling, more than any other single product category except digital components and loudspeakers. Please note that all I have been attempting to do in this thread is to correct your statement that "12%" of the reviews published in Stereophile were of cables. Yes, you have brought into the discussion Stereophile's "Recommended Components" listings, amplifiers, the ancillary components listed in our reviews, even the late Julian Hirsch and Stereo Review's editing policy, when all you really needed to do was acknowledge that your 12% figure was incorrect, that the actual figure is 5% when all the reviews published by Stereophile are taken into account, or 6% when the reviews are restricted to those published since I became the magazine's editor in May 1986. publishing a Recommended Components List that includes products that might not even exist or have not been evaluated (NR) I don't know which magazine you are referring to here, Mr. Nousaine. Every product recommended in Stereophile's listing exists and has been evaluated. The "NR" to which you refer means that while the product has indeed been evaluated, a review has not been published. You acknowledged in this newgroup that your RCL listed a product (Grado cartridge) that never existed and had never been reviewed did you not? Excuse me? This was an mid-1990s parenthetical mention of a product that was not specifically being recommended. Here is the exact wording that bothered Norman Schwartz. It appeared in the middle of an entry for a Grado cartridge that was being recommended, that had been reviewed: "Will hum if used with older AR decks (an "AR" version is available)." As I wrote in message , this parenthetical information (not a recommendation, please note) was based on information supplied us by Grado during our fact-checking on price and availability. As a matter of policy, we note such information in the listing when it appears relevant, as it did in this case. But as I said, every product that is actually _recommended_ in Stereophile's "Recommended Components" has indeed been evaluated. as far as I know Hachette magazines don't recommend products they've only "heard about." :-) I don't [know] that Stereo Review did either. And neither does Stereophile, Mr. Nousaine. And whom are you quoting when you put the words "heard about" in quotes? Certainly not me, if that is the impression you wished to leave. Referent prior comment above I repeat, when you put "heard about" in quotation marks, this is taken as meaning that someone used those exact words. You are incorrect. No-one has used those words in the context of this discussion. And, of course, that you raise this apparent exception at all is just _another_ example of data dredging on your part, Mr. Nousaine, the quoting of an apparent exception as being typical of the whole. Look, Mr. Nousaine, if you are serious about making it your mission to criticize magazines that compete with your own, why don't you investigate the fact that some reviewers (not Stereophile's) act as paid consultants for manufacturers whose products they review? Time and resources permitting I will conduct independent evaluations for manufacturers on contract and and DIY enthusiasts but I do not now nor have I ever consulted in design with a manufacturer. No-one has said that you do "consult in design," Mr. Nousaine. But if you do consultancy work of _any_ kind for manufacturers whose products you review, shouldn't that fact be made public in your published evaluations? And why isn't it a conflict of interest? As far as other instances I know of none offhand. [The late] Peter Mitchell consulted and evaluated products but as far as I know he didn't review his own work. Yes, Peter's primary career was as a consultant. As he wrote a news column for Stereophile when he was alive, not reviews, I don't see that this was a conflict of interest. Why don't you investigate a magazine (not Stereophile) whose current policy is to sell its cover to an advertiser? I'm guessing that you are bringing up The Audio [Critic] and Fourier loudspeaker issue again. Why would you guess that, Mr. Nousaine? I used the word "current." And The Audio Critic didn't "sell its cover to an advertiser," it published a highly complimentary review of a product made by a company in which the magazine's editor, by his subsequent admission, had a 50% equity holding. John Atkinson Editor, Stereophile |
|
#436
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#437
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
|
|
#438
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
(S888Wheel) wrote:
.. From: (Nousaine) Date: 6/10/2004 6:49 AM Pacific Standard Time Message-id: GFZxc.467$zz.441@attbi_s04 (S888Wheel) wrote: ....snips...... But what does listing these accessories in a speaker review ( Richard Gray's Power Company Substation isolation transformer, 1200s, 600s, & Pole Pig power conditioners; Sounds of Silence Vibraplane active isolation platform; Symposium Ultrashelf, Finite Elemente equipment stands' Audioharma Cable Cooker; Walker Precision Isolated Motor Drive, ACS Tube Traps, Shakti Stones & On-Lines; PRG, BAD, Abbfusor panels as accessories; in addition to 7 assorted cables) do for anyone's credibility? Plenty for some, nothing for some others, including you. However it seems to me that the listing of room treatment for a speaker review is quite important. by the looks of your post it seems you might not agree. I will keep this in mind should I read a speaker rview by you. I have nothing against room treatments but the above says nothing about how/when/why thet were used. The above, is what you wrote. The reviewers have described their set ups in detail in previous issues if memory serves me. The details included placement of room treatments. If you are concerned about how room treatements are being used you might consider an e mail asking how they are used. The fact they are used seems pretty relevant to me. First I'm not worried about anything. My point is that when fancy cables, audio equipment racks. shakti stones etc. are "listed" it makes me question the credibility of the reviewer rather than reinforce it AND it appears to me that listing of those audio-candy items as evaluation "accessories" when there is no acoustical mechanism for sonic improvement why would a rational person think that the iterms that could improve the situation would be effectively employed? My point has nothing at all to do with the effectiveness of room treatments (although some of them rate as snake-oil) as such. This post was assessing whether listing such along with audio-jewelry enhances the credibility of the review. IMO, it does not. To me it 'appears' that because the reviewer also lists audio candy that he may not have effectively employed such devices. Room treatments are audio candy? Why speculate on the use? Why not ask? If it weren't listed however, you wouldn't know to ask would you? The use of room treatments certainly can affect the performance of a speaker. It makes complete sense to me that room treatments be listed in reviews. The *more* we know about the system used for evaluation the better. You seem to be arguing the *less* we know the better for the sake of space. You're askew here. I'm saying that listing snake-oil as 'accessories' obscures the relevance of the real accessories and as used here tends to reduce the credibility of the evaluator and appears to have other reasons behind the screen. If you'll accept that (and I'm sure you don't) then it stands to reason that this space might be better suited to other purposes. Even devoting it straight advertisments (instead of subliminal) might enable price reductions for readers. IOW the listing indicates that mythology may be more important than performance to this individual and that the accessory list is just a merchanding tool and a method of emphasizing self-importance. How on earth is listing room treatments perpetuating any mythology? Listing them along side "Ultrashelf, Finite Elemente equipment stands' Audioharma Cable Cooker; Walker Precision Isolated Motor Drive, ...., Shakti Stones & On-Lines;" just tends to put them into the high-end audio-candy realm. Indeed it diminishes their importance. How can you expect me to take your speaker reviews seriously if you hold such a belief? Room treatments can make a huge difference. Listing them for speaker reviews makes complete sense. Well if I listed my draperies and carpets by brand (which have a large acoustical impact) should that make anybody else feel "better" about my reviews? Who should think that there's something special about using Ultimate brand tri-pod speaker stands for my surround speakers to the proper position and height? Would listing the brand of mdf, drywall screws, glue or the brand name of the blade used to cut the panels of my custom subwoofer make any difference in its performance or the performance of a satellite speaker? IMO the answer is No. And, "listing" such diverts attention from true evaluation of sound quality performance. |
|
#439
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Nousaine wrote:
(S888Wheel) wrote: The above, is what you wrote. The reviewers have described their set ups in detail in previous issues if memory serves me. The details included placement of room treatments. If you are concerned about how room treatements are being used you might consider an e mail asking how they are used. The fact they are used seems pretty relevant to me. First I'm not worried about anything. My point is that when fancy cables, audio equipment racks. shakti stones etc. are "listed" it makes me question the credibility of the reviewer rather than reinforce it AND it appears to me that listing of those audio-candy items as evaluation "accessories" when there is no acoustical mechanism for sonic improvement why would a rational person think that the iterms that could improve the situation would be effectively employed? Speaking of snake oil, doubtless we'll be seeing listing of the State Technology Research's 'Audio Collimator' being touted this month in Art Dudley's column in Stereophile. It's one of those wonderous devices whose inventors admit 'we don't know how it works, but it does!' Better yet, Dudley 'tested' it (I use the term in its most meaningless sense) and finds that it works *sometimes but not *all the time*. Realizing that Dudly does have a sense of humore, and being unable to find an STR website, I was really hoping this was some sort of belated April Fool's hoax, but alas it appears they presented their dubious wares at CES this year. http://www.audiophilia.com/reports/ces2004.htm "3) State Technology Research, Ltd Collimators -- This product is available in 4 versions. One with feet can be placed over or under a component ($475). The same version without feet is placed over a small speaker ($425). Another can be used as a speaker stand ($1650/pair) and the fourth is an image-loc pillar placed in the room ($950). These magnetically powered devices are designed to eliminate diffusion of the signal, time distortion, minimize room acoustic deficiencies and in general improve accuracy of presentation." |
|
#440
|
|||
|
|||
|
Doing an "evaluation" test, or has it already been done?
Tom Nousaine ) wrote in message
... (John Atkinson) wrote: (Nousaine) wrote in message news:rU9xc.14227$HG.12445@attbi_s53... (John Atkinson) wrote: Now you are asking that I limit my analysis to just the issues for which I have been responsible, though it is fair to point out that you made no such qualification when you claimed that "12%" of the products reviewed in Stereophile were cables, nor did you clarify until pressed that you were basing your "12%" figure on just one year's worth of Stereophile issues (2001). I specifically said that I examined one January list of prior years products and found that 12% of them were cables. Yes. I pointed out that you happened to choose a year where more cables had been written about than usual to derive your typical figure. No response from Tom Nousaine. This is a serious point: that if Mr. Nousaine is presenting statistical information to support his point, his picking and choosing among the data is both misleading and bad science. No "science" is needed to examine a consumer magazine. Please. The reviews is Stereophile seem to disregard any of the available data on the sound of wires/amplifiers/digital reproduction, even those conducted by yourself. Indeed, you personally made a claim in a newsgroup ('subjects were able to hear a single electrolytic capacitor in the signal path') based on evidence available for review that did not show that to be the case. You forthwith pointed to individual musical selections that 'seemed' to support that conclusion but you did not report that the overall conclusions were null. you did not report that at original publication that you reported (pre-1990) that the tests 'called for more experimentation' which you never conducted. And you failed to respond to the question ....why were some of the musical selections on which you stand ....had reverse positive results (the subjects incorrectly identified one DUT as the other) but still you still argue "data dredging." You then cherry-picked another year... No. I mentioned another year's figure in passing to demonstrate that the year you chose was not representative. I made no claim that this other year was typical or representative. OK why not just post all the years from 1999 to now? No response from Tom Nousaine. This is another serious point. I made no such comment along these lines, even though he has now repeatedly claimed on this newsgroup that I have done so. Here again is the text of mine to which he is referring, in a message posted on May 28: Looking at the raw data, Stereophile wrote about 19 cables of various kinds in 2001 out of a total of 163 components reviewed, ie, 11.65% Of course, you could have chosen 2002 for your analysis, when Stereophile reviewed just 3 cables out of 143 components reviewed, but that wouldn't have supported your argument, would it Mr. Nousaine? As I said, "data dredging." As I wrote, I made no claim that the incidence of cable reviews published in Stereophile in 2002 was typical. I only instanced that year to draw attention to the fact that Tom Nousaine's choice of 2001 on which to base his analysis was data dredging. and then just added up all the issues back to day one which included 15 years of material which was not on your watch. As I said, if you are going to talk about the overall percentage of reviews in Stereophile that are devoted to cables, the overall statistics seeme a good point to start. As I also said, the correct figure for _all_ reviews is 5%, not the 12% you claimed. No response from Tom Nousaine. In response to my offering the overall statistics for cable reviews, he pretended first that he was referring to just one year, then that he was talking about just the issues that I have edited to derive his 12% figure. Surely a reasonable person might assume that Mr. Nousaine would accept the correction and move on? Perhaps Mr. Nousaine is not as reasonable nor as disinterested a commentator as he might have us believe. Perhaps Mr Atkinson is not as interested in how much cabling is emphasized in his magazine as wanting others to think its not? You then asked about the statistics for the reviews that had been published during my tenure as editor of Stereophile. This raises the proportion of cables reviews to 6%. Hardly a big difference, I suggest. Again no response from Tom Nousaine. One wonders why he asks so many questions if he is not interested in the answers? I am intersted in answers. How many of of those cables wound up on your RCL the following year? How many of the amplifiers reviewed in that year wound up on your RCL? You have steadfastly refused to answer those questions; instead "answering" with vague statements. It looks to me from your reference that all but one or two of the cable reviews occured during your watch. This is what I said. That the first cable review to appear in Stereophile was in 1978, 16 years after the magazine's first issue, and that the highest proportion of cable reviews appeared in the 1990s. So what? As I have offered the statistical analysis of the reviews that were published "during [my] watch," and that difference between that period and the overall period is 6% vs 5%, both significantly less than the 12% you claimed, I fail to understand your point. Sure; but those wishing to downplay the significance of cabling may fail to recognize the [importance] of cabling in other ways, such as the Recommended Component List and implied reviews of cabling and its effect in the context of reviews of other products. I assume you wish to downplay the "significance of cabling," Mr. Nousaine because you believe that cables, provided they are of the appropriate length, gauge, and construction, don't affect sound quality. I don't "believe" that cables are cable ....all the extant evidence shows that this is true. Until you (another interested party such as a cable "manufacturer" used in parens because no after-market high-end audio company draws copper) deliver some real evidence why should the rest of civilization take your word (and your reviewers or advertisers) for it? But all I am doing in this thread is responding to your specific statement, in message , that: A study of one of the last few years Reviewed Components for the previous years showed 12% were cabling, more than any other single product category except digital components and loudspeakers. Note again that you didn't say one specific year, Mr. Nousaine, nor did you restrict your analysis to the years during which I edited Stereophile. Your actual words were "last few years." That is true. I should have restricted my wording to 2001. But, even so, why are you so willing to restrict analysis to "reviews"'(even though it was the original statement) and not intersted in dealing with the issue of how much does cabling influence high-end audio? IMO in the whole context of your publication cabling may have a higher significance because they get mention in the evaluation of other products. That's your opinion, Mr. Nousaine, and I see no reason to argue with it. Of course not. It's a way of life. As I said, all I am doing is addressing your specific point concerning the proportion of _reviews_ of cables published in Stereophile. As I have shown, to derive your "12%" of reviews that you decribed as referring to a "few years," you chose a specific single year (2001) that was untypical. However you wish to describe this -- "cherry picking," "data dredging" -- it is bad science on your part. There is no science involved. The significance of cabling is reflected in your RCL, mentions in reviews and perhaps most loudly in the allied component lists. IMO cabling carries a strong underpinning of high-end audio and an examination of the content of your magazine points this out. I urge anybody with interest to make up their own mind. As do I. Mr. Klein specifically mentioned Julian Hirsch's reviews, where he would discuss negative comments made by Julian with the manufacturer of the product after they had been sent a preprint. If Larry and the manufacturer could not reach agreement on how the review should be worded, then Larry would cancel publication of the review. I don't recall him saying exactly that. That there was a negotiation between Hirsch and a manufacturer. Please note that Mr. Klein made it clear that Julian was not involved in these discussions, that it was Mr. Klein who would negotiate the final wording of Julian's review with the manufacturer, not Julian himself. Thank you for pointing this out. I have not written otherwise. _You_ were the one that said that Julian was involved in such negotiations, Mr. Nousaine. I did not say such..... I said that I had not remembered so. You keep trying to pretend you were talking about Stereophile's Recommended Components listing when Dr. Richman's original statement and your original response concerned _reviews_. No pretense. I think that restricting the data discussion to items "reviewed" understates the relative importance of cabling in your publication. So why then did _you_ make that restriction, Mr. Nousaine? Here again are your exact words: A study of one of the last few years Reviewed Components for the previous years showed 12% were cabling, more than any other single product category except digital components and loudspeakers. Please note that all I have been attempting to do in this thread is to correct your statement that "12%" of the reviews published in Stereophile were of cables. Yes, you have brought into the discussion Stereophile's "Recommended Components" listings, amplifiers, the ancillary components listed in our reviews, even the late Julian Hirsch and Stereo Review's editing policy, when all you really needed to do was acknowledge that your 12% figure was incorrect, that the actual figure is 5% when all the reviews published by Stereophile are taken into account, or 6% when the reviews are restricted to those published since I became the magazine's editor in May 1986. Please I made no restrictions on any given point. My point was that Richman was wrong when he made the statement .... and, of course if you restrict yourself to "reviews" he was right. I have said so. But if you look at the overall imprint of Stereophile it looks like you are more than happy to Recommend that enthusiasts devote at least 10% to cabling. You might argue that the "true" figure is 6% but that is in disregard to the statistics that show that approximatly 10% of the 1993 RCL were cables; and totally disregards that the list advertised for at least a few years that the list had 'More Than 700 Components' when if fact it had fewer than 550. People can look at these statistics and figure out who is data dredging and who is not. Even so ... I don't care. I'm just interested in who is caring for readers (like me) and who is not. And as far as I can see Stereophile is not in a general sense. And that's primarily because you are carrying and occasionally promoting the High-End Myths and Urbn Legends instead of emphasizing acoustical performance. publishing a Recommended Components List that includes products that might not even exist or have not been evaluated (NR) I don't know which magazine you are referring to here, Mr. Nousaine. Every product recommended in Stereophile's listing exists and has been evaluated. The "NR" to which you refer means that while the product has indeed been evaluated, a review has not been published. You acknowledged in this newgroup that your RCL listed a product (Grado cartridge) that never existed and had never been reviewed did you not? Excuse me? This was an mid-1990s parenthetical mention of a product that was not specifically being recommended. Here is the exact wording that bothered Norman Schwartz. It appeared in the middle of an entry for a Grado cartridge that was being recommended, that had been reviewed: "Will hum if used with older AR decks (an "AR" version is available)." OK, fair enough. Why did it remain on the list for more than a single issue? As I wrote in message , this parenthetical information (not a recommendation, please note) was based on information supplied us by Grado during our fact-checking on price and availability. As a matter of policy, we note such information in the listing when it appears relevant, as it did in this case. But as I said, every product that is actually _recommended_ in Stereophile's "Recommended Components" has indeed been evaluated. Sure; but some people will argue that Every Change is important and you would include this "maybe" product without verification is intersting? It surely caused at least one subscriber some problems. According to high-end lore even breathing on a product has to make some difference....I'm wondering why the Grado-maybe would be included one way or another. That's the kind of thing that people would accuse of Stereo Review IMO. But thanks for the illucidation. I withdraw my objection. as far as I know Hachette magazines don't recommend products they've only "heard about." :-) I don't [know] that Stereo Review did either. And neither does Stereophile, Mr. Nousaine. And whom are you quoting when you put the words "heard about" in quotes? Certainly not me, if that is the impression you wished to leave. Referent prior comment above I repeat, when you put "heard about" in quotation marks, this is taken as meaning that someone used those exact words. You are incorrect. No-one has used those words in the context of this discussion. And, of course, that you raise this apparent exception at all is just _another_ example of data dredging on your part, Mr. Nousaine, the quoting of an apparent exception as being typical of the whole. And, of course, you will leave out the capacitor affair as well, where you reported in anewgroup that subjects were able to reliably identify capacitor dialectic by sound alone .... when the overall experiemt was null ....and the selection of data that you thought supported that conclusion was also filled with reverse significance (subjects incorrectly reverse identifying dialectric.) Look, Mr. Nousaine, if you are serious about making it your mission to criticize magazines that compete with your own, why don't you investigate the fact that some reviewers (not Stereophile's) act as paid consultants for manufacturers whose products they review? Asked before. Who are you talking about? Time and resources permitting I will conduct independent evaluations for manufacturers on contract and and DIY enthusiasts but I do not now nor have I ever consulted in design with a manufacturer. No-one has said that you do "consult in design," Mr. Nousaine. But if you do consultancy work of _any_ kind for manufacturers whose products you review, shouldn't that fact be made public in your published evaluations? And why isn't it a conflict of interest? It has been publicly acknowledged. And I've never reviewed a product (or measured one for another evaluator that I've measured independently in the course of a review) but even if that had happened, so what? Measurements are measurements. Why would that be a conflict of interest? As far as other instances I know of none offhand. [The late] Peter Mitchell consulted and evaluated products but as far as I know he didn't review his own work. Yes, Peter's primary career was as a consultant. As he wrote a news column for Stereophile when he was alive, not reviews, I don't see that this was a conflict of interest. As above; why not? Why don't you investigate a magazine (not Stereophile) whose current policy is to sell its cover to an advertiser? I'm guessing that you are bringing up The Audio [Critic] and Fourier loudspeaker issue again. Why would you guess that, Mr. Nousaine? I used the word "current." And The Audio Critic didn't "sell its cover to an advertiser," it published a highly complimentary review of a product made by a company in which the magazine's editor, by his subsequent admission, had a 50% equity holding. John Atkinson Editor, Stereophile OK who is currently consulting on products that they are reviewing? And who is selling the cover? |
| Reply |
| Thread Tools | |
| Display Modes | |
|
|
Similar Threads
|
||||
| Thread | Forum | |||
| ALL amps are equal?? | Car Audio | |||
| Light weight system challenge | Car Audio | |||
| Note to the Idiot | Audio Opinions | |||
| Mechanic blames amplifier for alternator failing?? Help>>>>>>>>>>> | Car Audio | |||
| Southeast Invitational Sound Challenge | Car Audio | |||
Linear Mode
