What is transparent like ML, dynamic like a horn, and disappears like a mini monitor?

[jfm]

And you'll also notice that we almost never say the B-word.

i must have missed it, but what's the B-word?

 

[tonepub]

"Best"

Harry Pearson used to tell me to NEVER use that word, because you paint yourself in the corner, especially when saying that something is the best, period.

It's one thing to say something like "these are the best Martin Logan speakers I've ever heard"

It's quite another to say "the Martin Logan CLX is the world's best speaker"

Remember when Stereophile said the Halcro was "the worlds best amplifier"?

 

[tonepub]

"History repeats itself" I wish!! The biggest problem with "reviews" today is the word. They're really presentations, not reviews. Most magazines, like TONE Audio, do a great job showcasing products, and as such, they provide a service to the reader, since manufacturers' websites are generally inadequate. A useful review of an audio component -- something that describes performance in depth and with appropriately selected other components -- is a thing of the past and not likely to repeat itself. And I think that's not really such a big loss. I just wish the people who write up these presentations would stop calling themselves reviewers, though the hype doesn't surprise me.

Please explain why we do not "review" gear and merely "present" it.

When we get a product in for review, it is put into a reference system where only one component is changed, to evaluate the performance of that component alone.

We always make it a point to use a pair of speakers in for review with at least three or four sets of very different cable to try and find possible synergies.

We always make it a point to use a set of speakers with at least six different amplifiers, tube, solid state and chip amplifiers from 9watts per channel to 400.

We always make it a point to use a set of speakers in at least three different rooms, small, medium and large, some with room treatments, some not, to see how they perform in a real world atmosphere.

We do the same thing with other components.

We also make it a point to have at least one other reviewer if not two or three (depends on the size of the gear in question) listen to the product before the review goes out and often we print a second (and sometimes third) opinion on the gear.

That's the way Harry Pearson did it and that's the way I do it.

I am really tired of your passive aggressive potshots at us whenever you get the chance.

 

[Gordon Gray]

And so am I.

GG

 

[jtwrace]

me 2. I'm thinking that TomDac NEEDS to do something at this point...

 

[Rich]

I have to agree that it is getting old.

 

[jfm]

"Best"

Harry Pearson used to tell me to NEVER use that word, because you paint yourself in the corner, especially when saying that something is the best, period.

It's one thing to say something like "these are the best Martin Logan speakers I've ever heard"

It's quite another to say "the Martin Logan CLX is the world's best speaker"

Remember when Stereophile said the Halcro was "the worlds best amplifier"?

ah, ok, thanks. Valin must not have gotten that memo, then. So, Jeff, did you get to hear the system at Sea Cliff? Impressions?

BTW, I applaud your commentary system, as the practice of multiple commentaries is what set the old TAS apart, and what can set a current mag apart in the age of the Internet. Now that anyone can do reviews, what can a publication offer that you couldn't find in online communiities? I believe that it's the commentary system, placing components in many different scenarios, where they interact with different components. That would be more valuable than reading just one review from anywhere, where the reviewer is really assessing one system. This would help unravel some of the mysteries of synergy. (The old practice of listing listening biases was also essential to this multiple commentary approach.)

 

[tonepub]

Thanks.

I never made it out to seacliff, but HP and I still stay in touch. He's a pretty cool guy and told me a few weeks ago, if he was a younger guy and I could afford him, he'd come to work for me!

We really feel that using a component in a number of different scenarios can uncover a lot, because every one of you has a different situation in your room.

I always try to use a review component in as many different combinations as possible to try and turn things up. Some are more sensitive to cables, others are more complicated to set up, etc.

I couldn't agree with you more on listing your biases, because that can also really shape how a review goes. When I was chatting with Dave Wilson, he said "I can see why you like electrostatics so much, I really enjoy them myself..."

I've mentioned it in more than one review that I really like my ML's, so I'm always looking for that transparency that my Summits offer.

The other thing you really get with us that is hard to get from the random people on the internet is that because I am able to do this full time, I just have access to a lot more gear to listen to all the time. I'm pretty sure that HP, Robert Harley, John Atkinson, Mikey and perhaps a couple others only have that luxury. Almost all the other guys at the print magazines have day jobs, so they don't get to spend as much time listening as they would like to.

 

[nsgarch]

Please explain why we do not "review" gear and merely "present" it. When we get a product in for review, it is put into a reference system where only one component is changed, to evaluate the performance of that component alone.
We always make it a point to use a pair of speakers in for review with at least three or four sets of very different cable to try and find possible synergies. We always make it a point to use a set of speakers with at least six different amplifiers, tube, solid state and chip amplifiers from 9watts per channel to 400. We always make it a point to use a set of speakers in at least three different rooms, small, medium and large, some with room treatments, some not, to see how they perform in a real world atmosphere. We do the same thing with other components. We also make it a point to have at least one other reviewer if not two or three (depends on the size of the gear in question) listen to the product before the review goes out and often we print a second (and sometimes third) opinion on the gear. That's the way Harry Pearson did it and that's the way I do it. I am really tired of your passive aggressive potshots at us whenever you get the chance.

And I'm really tired of being the object of your paranoia. My comments on the state of audio equipment 'reviews' were general, and not directed at any person or publication. The only reason I mentioned TONE was because I thought it represented the best example of product presentation among audio publications. Sorry if you didn't appreciate the complement.

As far as I'm concerned, the fact that the testing of audio equipment (against manufacturer's specs) is no longer perceived as "review-worthy" leaves out one of the essential ingredients of a review! Subjective listening experiences might be credible too, but sadly most people, and most audiophiles, haven't spent the little time it takes to develop a few basic listening skills. Instead they associate 'good sound' with an infinite array of unconsious personal associations.

I like jfm's idea that the internet (I assume he's referring to the many audio forums) provide a level playing field for commentary. And this is where I go when I really want unvarnished information about a piece of equipment, or a particular combination.

It seems the photo world is wrestling with the same issue: the difference between a product preview and a review:
http://www.dpreview.com/news/0603/06030802reviewdefinition.asp

 

[Rich]

The only reason I mentioned TONE was because I thought it represented the best example of product presentation among audio publications. Sorry if you didn't appreciate the complement.

That is certainly one of the most backhanded compliments I have seen in a while. You state that these magazines (who do put a lot of effort into analyzing and reviewing products as part of their primary purpose) are actually only "presenting" them for the public. That would not be considered a compliment by anyone. You don't compliment someone by questioning the veracity underlying one of their primary purposes. The only one you specifically mentioned by name was Tone. Given your history of ragging Jeff and his publication on this site, it is quite clear that your purpose was not to compliment them.

As far as I'm concerned, the fact that the testing of audio equipment (against manufacturer's specs) is no longer perceived as "review-worthy" leaves out one of the essential ingredients of a review!

But that is not what you said, is it? What you said was:

A useful review of an audio component -- something that describes performance in depth and with appropriately selected other components -- is a thing of the past and not likely to repeat itself.

This is exactly what Jeff described that he does, and only after he called you on it did you bring up the issue of testing specs. I am sure being an engineer that the testing of the manufacturers specs may be important to you. I am also sure that it is not that important to most audiophiles. People are more interested in how a component sounds to a reviewer's trained ears than whether the THD is actually .01%.

You still never answered Jeff's question. On what do you base your contention that these magazines are product "presenters" rather than product "reviewers?"

It seems the photo world is wrestling with the same issue: the difference between a product preview and a review:
http://www.dpreview.com/news/0603/06030802reviewdefinition.asp

This is a red herring. If you read Tone, you would see they make quite clear when they are doing a product preview and when they are doing an actual review. The other mags do the same. I have never been confused thinking a preview was an actual review.

Neil, I am not saying that there is not some valid criticism to be a laid at the door of all the magazines regarding the objectivity of their reviews and the actual utility of those reviews to the buying public. But in this instance it just looks like you were using the subject as another opportunity to bash a member of this forum and his publication in a thread that he started. This behavior is getting old to many members of the forum, myself included.

 

[eknuds01]

I am in agreement with Rich. These posts do little more than suck up bandwidth.

 

[Gordon Gray]

I also agree with Rich. This is not about "paranoia" but, apparently to me and others on this forum, about vindictiveness.

GG

 

[jtwrace]

I also agree with Rich. This is not about "paranoia" but, apparently to me and others on this forum, about vindictiveness.

GG

Once again, well said.

 

[tonepub]

Thanks for the support.

As we go further down the path, we will be incorporating some measurements in our mix, as we see them being valid and of use to the outcome of you, the reader applying it to a potential purchase decision.

One thing I've seen with evaluating a lot more phono cartridges and preamps that seems to be useful is channel balance. You could drive yourself nuts moving speakers around, when all that's off is your channel balance.

We have taken a few amplifiers in Dan's (our technical editors) lab and done the basic measurements (distortion, power output, S/N ratio, etc) and other than a few SET amps, we really haven't found anything that exciting to report.

All of the reputable mfrs. use the same suite of test gear and know that if they send their stuff to be measured and it doesn't meet spec they are going to get crucified, so again now that this is a mature industry and "more watts per channel" doesn't sell amplifiers anymore, it's a moot point.

We've probably measured about thirty amplifiers over the years and they've all exceeded their published specs.

I know John Atkinson turns something up now and then, but we've made a corporate decision that it just isn't that important. (at least to us)

If any of you found out that your preamp put out .07% distortion instead of the claimed spec of .05, would you want to get rid of it?

I don't think that measurments are invalid, I just don't know how useful they really are in terms of you our reader making a purchase decision on a piece of gear. Usually even an untrained ear can tell the difference between .1% and .01% distortion, but I've heard plenty of gear that's measured great and sounded pretty lifeless and vice versa.

I'm sure you have too.

The biggest problem here is that the anomalies in equipment pales in comparison to the anomalies in our hearing and room response. While my studio measures quite good, my living room is pretty awful with hardwood floors, area rugs and leather couches. (Not to mention that big glass coffee table)

So I like to listen there as well to see what a speaker will do in a real world environment, and again why it is so important to you to listen to the product in your space, because it just might not work.

However, I do enjoy these kinds of discussions sans the insults. And no, I"m not paranoid, I've had enough readers and pillars of the audio industry tell me they love the magazine and they love our approach.

I try to do my best to be as transparent as possible so again, we can help you find a good system and avoid any potential monkey business. Our industry has too much of it as it is. I am always happy to discuss "why we do what we do".

 

[David Matz]

And I'm really tired of being the object of your paranoia. My comments on the state of audio equipment 'reviews' were general, and not directed at any person or publication. The only reason I mentioned TONE was because I thought it represented the best example of product presentation among audio publications. Sorry if you didn't appreciate the complement.

As far as I'm concerned, the fact that the testing of audio equipment (against manufacturer's specs) is no longer perceived as "review-worthy" leaves out one of the essential ingredients of a review! Subjective listening experiences might be credible too, but sadly most people, and most audiophiles, haven't spent the little time it takes to develop a few basic listening skills. Instead they associate 'good sound' with an infinite array of unconsious personal associations.

I like jfm's idea that the internet (I assume he's referring to the many audio forums) provide a level playing field for commentary. And this is where I go when I really want unvarnished information about a piece of equipment, or a particular combination.

It seems the photo world is wrestling with the same issue: the difference between a product preview and a review:
http://www.dpreview.com/news/0603/06030802reviewdefinition.asp

Neil,

Whether you call it a review, preview, or whatever, it seems like you are having issues with subjective/ experiential information, which by its nature ,is difficult to communicate, difficult to quantify, difficult to rigorously describe in an economical way, etc.

Some guys like yourself like Mac tubes, other guys like Bryston and swear it is the most neutral, while others think its hyper analytical and brittle, and prefer Rogue. There is rarely any agreement. Who is right? The one who auditioned the equipment and made a decision based on his preferences without pressure or desire to please others. How many discussions have we had here about tubes vs. solid state? Has anyone been convinced by our discussions?

Also, can you give one specific example of a review written any magazine that you think is worth much without actually experiencing the product? I certainly can't, and the reason is the nature of this type of information. I need to listen to the equipment myself, and then read as many reviews as I can to confirm my thoughts. If I don't listen for myself, I am falsely assuming that what I am reading applies to me. However, based on my tastes, setup or room, it may not. This post started with discussions of Wilson speakers. Can anyone really know what a Wilson sounds like without hearing it? I doubt it.

Let's accept and recognize the limitations of experiential/ subjective information and move on.

 

[tonepub]

Well put.

Even the best review can only point you in a direction. It's up to you to see if it floats your boat or not....

 

[MiTT]

Yes, very good post David.

I'd be stunned if anyone on this board bought a component sound unheard based upon a review.

I use reviews to pique my curiosity, but only enough to prompt me to investigate listening to something (maybe I'm lazy). There have been times where I've read a good review, then heard the component for myself only to be left wondering what the big deal was. On the other hand, I've also listened to components and been able to draw very similar parrallels to the review that might have prompted me to seek it out.

Most magazines are a pleasant distraction for me, or something to do with my hands when I'm on the toilet. I use my ears to make buying decisions pertaining to my audio equipment.

 

[Joey_V]

I'd be stunned if anyone on this board bought a component sound unheard based upon a review.

Actually, it's tough to audition the components you want to buy in your own system or in a dealer's system that's similar to yours... especially some of these more exotic, eccentric components that aren't found more anywhere. I have to rely on reviews (multiple) that give me some feel of the component... but I rely more on forum feedback.

Then I proceed to look for a good/fair deal on the said component and audition it in my system against the old/soon-to-be replaced component just to make sure I like what I hear and I find that it's worth it.

So I suppose I do buy components sound unheard. Sometimes I'm left no choice and I assume there are more on this board like this.

 

[Dreamer]

I'll agree with Joey. Some of us, who live in areas of th ecountry where we don't have access to a myriad of high-end dealers, and who can't flash a wallet full of Gold Cards when we'd like to audition a new bit of gear at home simply don't have the option of getting a new component home and listening to it in our system.

I bought my Benchmark DAC-1 without hearing it in a system that even vaguely resembled a "high-end" home system. I purchased it based on the reports of others--on this forum, other forums, and other online reviewers. Granted, I DID go to a local dealer in Maryland (a shop that specializes in studio gear, NOT high-end audio, BTW) and listened to the Benchmark on headphones and through some nice little studio monitors (I think they were Tannoys). But I don't think it was a geuinely meaningful audition, considering how radically different the setting and equipment was from ANYTHING I current own. Sure it was a little scary when the time came to swipe my credit card, because to me, $1000 is a LOT of money. But I can tell you that I'm VERY happy with the Benchmark, and either I am VERY lucky that it complements my current system, or I have an uncanny ability to match verbally evaluated gear with what I know my current system sounds like.

Same goes for my Oppo DVD player. I bought it without ever hearing one in my rig. In fact, I'd never heard or seen one at all until I got mine. Was that risky? Sure, a little. But based on forum reviews and the industry buzz, and coupled with Oppo's great support and return policies, I figured it was a pretty sure bet. I'm glad I got my Oppo.

I trust the folks on this forum, and other forums that I belong to, to give meaningful reviews of new gear, and I think that sometimes, when a meaningful audition is simply not possible, a stack of good reviews in conjunction with equipment similar to mine can be VERY helpful.

Truth be told, I've NEVER bought a new piece of gear based on research, reviews, and careful consideration that was a disappointment. I must be REALLY lucky that way.

I have, however, put new pieces of kit (on loan from friends or dealers) into my system to try them out, without doing the research, and been VERY let down. Several amps, which were touted as the cats pajamas, sounded like pure crap in my rig--a few NADs, the smaller Brystons, the new Krell stereo amps--and several CD players also failed to impress me in my rig, but sounded GREAT to other folks in other rigs. And we won't even discuss cables.

To be honest, for me, part of the thrill of this hobby is the unknown aspects of system synergy. I buy a fair amount of gear without hearing it in my rig, and I'm VERY happy with the way my rig sounds, and isn't THAT what it's all about--enjoying the music? I know my rig can sound better (a LOT better), but it's still a very enjoyable system. What I have may not be anywhere near "reference", but compared to 99% of the systems out there, my modest rig is pure heaven! I also know that perfection can NEVER be attained, and although I'm not content with my situation, I CAN be happy with what I have for now. When a REALLY significant improvement becomes available to me, I might jump at it, but as for the endless cycle of incremental "upgrades", grasping in futility toward a fundamentally unachievable goal, I'd rather ENJOY what I have than live in perpetual dissatisfaction. I only dwell on the negative parts of my system when I'm presented with a substantial possibility for improvement.

I mean, how many "high-ticket audiophiles" do you know who spend tens of thousands of dollars a year on their system, but can never honestly say that they are happy with the way their system sounds? Are their ears that much more sensitive than mine? I doubt it. Jet travel, sports cars, BlueTooth headsets and whiney trophy wives cause a LOT more hearing damage in the mid- and upper ranges than the surroundings of my lifestyle (camping, painting, printmaking, and woodworking with hand tools are relatively low-audio-impact activities), and I have nowhere near the exposure to those things that most of the folks in these rarified economic levels have. I'd venture to say that their dissatisfaction with their systems is based more on their personalities than their hearing abilities. They probably also go through houses, cars, wives, and cell phones with equally frenetic abandon--ever on the quest for the "perfect rig", but never truly appreciating what they have.

Auditioning new gear at home is a luxury that a LOT of folks simply can't afford, and to be honest, a LOT of high-end dealers are unwilling to extend unless you have a 24kt pedigree and an equally posh bank account. To look down on an audiophile because they buy new kit without auditioning it at home is more than a little elitist, and smacks of disdain for the "lower classes" based on their financial status.

I'm not saying that I would (or ever have) purchased audio gear based SOLELY on a magazine review. I don't. But I do trust the opinions of other folks on these forums who own similar gear to mine, and I think that making a new equipment decision based on the success and failures of others is a reasonable and logically sound methodology, ESPECIALLY if one can't actually get said gear into one's system for a "test drive" before purchase. This is th eprimary reason I'm on this forum--so I can swap ideas, experiences, and experiments with other folks who own similar gear, and use that information to help me make improvements and inform my purchase decisions.

No, MiTT, I strongly disagree with you. I think a LOT of folks on this forum probably DO buy gear without ever test-driving it in their systems, and probably have a reasonable amount of success building their systems this way.

--Richard

 

[rtagle]

I agree with Dreamer. The weakest link in my system is the source. I ordered the Memory Player by Nova Physics (NP) based on the reviews at Stereo Times, Positive Feedback, Stereo Mojo and others. Considering that NP never advertised this product in these publications added some credibility to the review. The MP is a radically new product and production is somewhat limited. Hence, it's not easily available for audition even at the most high end dealers. But I like the concept of a high end music server with tube output stage so I ordered one and I don't think that Harry Pearson and the rest who gave the reviews could be that wrong.