What is transparent like ML, dynamic like a horn, and disappears like a mini monitor?

[David Matz]

One thing to bear in mind with reviewers of high-end components is that they have no interest in Lo-Fi (Emerson, Soundesign, etc..) or Mid-Fi (Pioneer, Sony, etc...) devices. They simply refuse or do not ask for review samples from these companies.

PLEASE do not misunderstand me, though. We all have or will own middle-of-the-road gear in our lives, and they provide enjoyment that goes beyond their price tag.

But when it comes to strict 2 channel audio devices, we have the right -- the obligation -- to see how all components measure up. And then we make the call on whether to even audition, much less buy, the unit(s) in question.

Even though I am 13 years from interaction with the industry, I feel that some things never change and we should try to get better. Namely, that if a producer loans out a piece of its new equipment, that they should know that it's a crap shoot.... If your component sucks, it will be labeled as so. How else can the consumer be protected ??

Reviewers should call them as they see them, as I did with the Watt/Puppy 7's in another post on this forum. I am a fan of Rotel gear, but if Jeff spends a month or so listening to their latest product then I would like it for him to tell me if it blows ass -- if that is the case. If it rocks with authority, then he NEEDS to tell me this too.

I want and need to trust reviewers, as there is so much money at stake in many cases. We're not buying a six pack of tube socks here.... So cough up the real story, regardless of manufacturer response. It ensures informed readers and a straight-up relationship between builder and consumer.

And stop with the free gear and golf trips in exotic places. This type of influence from manufacturers, which I have accepted and nourished in the past, is akin to bribery....

~VDR

VDR,

I think what you are getting at is that you want this to be a science. This would require a lot of double blind testing. One would even have keep track of his mood to calibrate the response. This does not seem is to work in audio, as it is very experiential. Audio reviewing is more like restaurant reviewing. If the critic does not enjoy Korean or Indian, he is not going to bother with the restaurant in the first place. Likewise, why would an audio reviewer rip up the system he loves to plug in a component he does not care for? The food critic will bother reviewing only the places he likes, and then it will be different grades of "I like it" and "this is a bit different compared to that".

I do think that if the double blind testing did happen, a lot of high end manufacturers would go out of business. Some may still sell their goods based on image and trying to impress friends, but many will question.

 

[eknuds01]

VDR,

I think what you are getting at is that you want this to be a science. This would require a lot of double blind testing. One would even have keep track of his mood to calibrate the response. This does not seem is to work in audio, as it is very experiential. Audio reviewing is more like restaurant reviewing. If the critic does not enjoy Korean or Indian, he is not going to bother with the restaurant in the first place. Likewise, why would an audio reviewer rip up the system he loves to plug in a component he does not care for? The food critic will bother reviewing only the places he likes, and then it will be different grades of "I like it" and "this is a bit different compared to that".

I do think that if the double blind testing did happen, a lot of high end manufacturers would go out of business. Some may still sell their goods based on image and trying to impress friends, but many will question.

Tom Siestema (Washington Post) is a well known restaurant reviewer. He doesn't bother to visit eating establishments if he knows beforehand that he will likely end up writing an unfavorable review.

 

[Rich]

This is the double-edged blade of professional reviews, and why we as audiophiles and consumers can't really trust them for much. Because the magazines can't afford to develop a poor relationship with the manufacturers, and because the manufacturers can't stomach even a lukewarm review of one of their products, we rarely see a poor review or even an accurate comparison between wildly varying products.

If the reviewer isn't going to like the product, he isn't going to review it. When he does review it, he always finds aspects of it to praise, and he deftly softens his comments that may be deemed as critical. He has to be conscious not only of angering the manufacturer, but also of stepping on the toes of the many legions of fans that every brand seems to have. Thus every review raves about the good qualities of a product, glosses over the negatives, and ends up stating that the product is comparable to other products costing several times as much.

As readers looking for critical evaluations of a component we may want to consider auditioning, we are left with almost no useful information. That is one reason why I have decided to let my subscriptions to Stereophile and TAS expire. They rarely have any content worth reading anymore, from my perspective. I get much more useful information on this forum and others.

 

[Joey_V]

This is the double-edged blade of professional reviews, and why we as audiophiles and consumers can't really trust them for much. Because the magazines can't afford to develop a poor relationship with the manufacturers, and because the manufacturers can't stomach even a lukewarm review of one of their products, we rarely see a poor review or even an accurate comparison between wildly varying products.

If the reviewer isn't going to like the product, he isn't going to review it. When he does review it, he always finds aspects of it to praise, and he deftly softens his comments that may be deemed as critical. He has to be conscious not only of angering the manufacturer, but also of stepping on the toes of the many legions of fans that every brand seems to have. Thus every review raves about the good qualities of a product, glosses over the negatives, and ends up stating that the product is comparable to other products costing several times as much.

As readers looking for critical evaluations of a component we may want to consider auditioning, we are left with almost no useful information. That is one reason why I have decided to let my subscriptions to Stereophile and TAS expire. They rarely have any content worth reading anymore, from my perspective. I get much more useful information on this forum and others.

I agree, Rich. I find reviews to be more useless and useless... other than pure entertainment. I, too, find forums much more useful because although some are very opinionated and stubborn (some are fanboys, etc...), you get the real deal with regards to their opinions. No facts, but atleast you get the raw deal.

VDR,

I think what you are getting at is that you want this to be a science. This would require a lot of double blind testing. One would even have keep track of his mood to calibrate the response. This does not seem is to work in audio, as it is very experiential. Audio reviewing is more like restaurant reviewing. If the critic does not enjoy Korean or Indian, he is not going to bother with the restaurant in the first place. Likewise, why would an audio reviewer rip up the system he loves to plug in a component he does not care for? The food critic will bother reviewing only the places he likes, and then it will be different grades of "I like it" and "this is a bit different compared to that".

I do think that if the double blind testing did happen, a lot of high end manufacturers would go out of business. Some may still sell their goods based on image and trying to impress friends, but many will question.

I think the bottom line is not to necessarily make reviews only about products that the reviewer likes, but to include all products good/fair/bad in the published review articles. It's not to say that you can't include double blind tests IF the reviewer/magazine desires so, but to atleast include any and all products to have a published review regardless of the outcome.

I don't think it has anything to do with making reviews akin to science. It's all about making reviews more geared towards the consumer in mind - rather than protecting the manufacturers by only reviewing/publishing the quality products and not the crappy ones for fear of consequences from the manufacturer.

 

[Bernard]

When TAS reviewed the Cambridge 840C CDP they really liked it, and at the end made some comment along the lines of "bears comparison with reference quality CDPs". Well, in the same issue they reviewed the ARC Ref 7 CDP, so it would have been logical to compare the two side-by-side, but they did not. Makes no sense, unless there were other motives.

 

[eknuds01]

This is the double-edged blade of professional reviews, and why we as audiophiles and consumers can't really trust them for much. Because the magazines can't afford to develop a poor relationship with the manufacturers, and because the manufacturers can't stomach even a lukewarm review of one of their products, we rarely see a poor review or even an accurate comparison between wildly varying products.

If the reviewer isn't going to like the product, he isn't going to review it. When he does review it, he always finds aspects of it to praise, and he deftly softens his comments that may be deemed as critical. He has to be conscious not only of angering the manufacturer, but also of stepping on the toes of the many legions of fans that every brand seems to have. Thus every review raves about the good qualities of a product, glosses over the negatives, and ends up stating that the product is comparable to other products costing several times as much.

As readers looking for critical evaluations of a component we may want to consider auditioning, we are left with almost no useful information. That is one reason why I have decided to let my subscriptions to Stereophile and TAS expire. They rarely have any content worth reading anymore, from my perspective. I get much more useful information on this forum and others.

Quite right. I would say that the average consumer doesn't need the magazine's help in reviewing a product. While some reviews (i.e. Consumer Reports) could be helpful, the internet is the way to go. The are more products, choices, and people using this stuff (all things electronic, say) then ever before. A quick google search is all it takes to find an online community dedicated to discussing the product in question.

There's more relevent content on this site than any magazine.

Where are mags helpful? To show me the latest and greatest, product news, PR events (such as RMAF), etc.

Jeff's magazine is a nice compromise - lots of product reviews, music reviews, etc.

Like Rich, I no longer subscribe to the main stream mags any longer. It's always the same thing over and over again. Every advertisement and review is this mega-awesome endoresement. To the reviewers' credit, I believe there is more quality gear available now than ever before. Their job isn't easy. Reviewing hi-end amps, for example, is the 4k price range (for example) can't be an easy task.

Erik

 

[jtwrace]

When TAS reviewed the Cambridge 840C CDP they really liked it, and at the end made some comment along the lines of "bears comparison with reference quality CDPs". Well, in the same issue they reviewed the ARC Ref 7 CDP, so it would have been logical to compare the two side-by-side, but they did not. Makes no sense, unless there were other motives.

I'm sure the $50k full page ad in the back means somthing.

 

[Joey_V]

I think the biggest problem in audio review is the fact that these reviewers are somehow biased towards the manufacturer.

Why?

Because none of these mags or ezines have the capital to buy the equipment for the sole purpose of reviewing it. All (or most) rely on manufacturers to be "kind enough" to send them the product for review ON LOAN. Meaning, if the reviewer gets on the good graces of the manufacturer - the reviewer gets more products for future review.

And who doesn't want more toys??

IF there was a magazine whose purpose was to review and they had a large capital to buy speakers/amps/etc for this sole purpose (though I'm not sure this would work since the audio hobby is such a small and niche group), then you would see no-holds-barred reviews that would run the gamut from lambasting a products to praising it.

If only...

 

[Statman]

I think the biggest problem in audio review is the fact that these reviewers are somehow biased towards the manufacturer.

Why?

Because none of these mags or ezines have the capital to buy the equipment for the sole purpose of reviewing it. All (or most) rely on manufacturers to be "kind enough" to send them the product for review ON LOAN. Meaning, if the reviewer gets on the good graces of the manufacturer - the reviewer gets more products for future review.

And who doesn't want more toys??

IF there was a magazine whose purpose was to review and they had a large capital to buy speakers/amps/etc for this sole purpose (though I'm not sure this would work since the audio hobby is such a small and niche group), then you would see no-holds-barred reviews that would run the gamut from lambasting a products to praising it.

If only...

"History repeats itself"

For those of you like me who have been in this hobby over 20 years will remember that's how The Absolute sound began. Harry Pearson would not accept manufacturer advertisments because he didn't want to sway the results by hand picked or tweaked units for review.

If you read the issues from the late 70's to early eighties there are some pretty bad reviews on equipment. It was always fun to read in the back of the magazine the manufacturer's reply. Later on he changed and took the money for help to keep alfoat.

 

[SundayNiagara]

I thought that's how all 'zines ran - they only publish good reviews. This is why you don't see a bad review. Ever... (or almost ever).

Personally, I'd like to see 'zines look out for the consumer moreso than the manufacturer. They don't review bad products (or products that turn out lukewarm to the reviewer) probably because of the manufacturer backlash... and they send it back without a word in the magazine.

This is how my reviewer friend told me the industry functions.

I personally think that consumers are always left in the dark. The only reviews we read are the good ones, we never hear about the bad ones. And even then, we never even get a good sense of how a well reviewed product ranks against other similarly well reviewed products.

As much as I enjoy reading Stereophile or Absolute Sound, I've found myself (on more than 1 occassion) reading the intro and then the conclusion. I bypass the entire middle section of the article because it is almost the EXACT same from every other review article they've done - there's no variety because each and every review is "great! greater!"

I don't even read reviews to the end of the article nowadays, they always either recommend it heartily or recommend it without reservations.

And how is it that every other product reviewed is better than any other product 3 times its price? And that every other product which is 3x the price of the former product is worth 3x the price of itself once again? And so forth??

This is not against you, Jeff. But I just don't think there's much stock in reviews if the consumer is not looked after.

Joey

PS
It's sad when I can predict the final 3-4 sentences of a review article. Always they run... "This component made me hear new things that I had not heard prior. It is the epitome of design, execution, and technological know-how. I recommend this product as a good buy for anything 3x its price."

Try reading Hi Fi Plus!

 

[Joey_V]

Try reading Hi Fi Plus!

I've browsed it rather quickly... but I didn't subscribe because either it was too costly or I think it was primarily Euro in context.

However, is it good, Sunday?

 

[SundayNiagara]

I've browsed it rather quickly... but I didn't subscribe because either it was too costly or I think it was primarily Euro in context.

However, is it good, Sunday?

Great in depth reviews, like TAS used to do. Highly recommended!

 

[MiTT]

Great in depth reviews, like TAS used to do. Highly recommended!

Agreed, my favorite (hardcopy) audio publication too. Even the quality of the product photography and feel of the paper it's printed on seem a cut above. Love their music reviews too, somewhat short, but very much to the point and I've found a few real gems on their recommendations.

Kudo's to Jeff and Tone Audio as well - also a cut above the rest of the pack as far as the mix of content and drool worth product photos.

 

[tonepub]

While I can't speak for the other pubs, my standpoint from the publishers chair is that a trumped up review doesn't serve my readers or my advertisers.

I've had a couple of mfrs get mad because we wouldn't say that their $4000 CD player doesn't "Destroy" all of the $15k players out there, but for the most part, if we tell you a component is way better than it is, you buy it and are unhappy with it, chances are you will move it on audiogon, get on a board like this and tell everyone said product was awful and that we were idiots for suggesting it in the first place.

That might buy me an ad spot for a year, but it doesn't work long term.

I own my three reference systems. Yes, I got industry pricing, but I'm still out about $125k for all this stuff. The point is, I don't review gear to get more "Toys", we are trying to help you guys make a short list for stuff you might want to investigate. It's my job, it's not my hobby anymore.

And as for the advertising thing, because we don't have to print and ship 85 thousand magazines every month, our ad rates are about 1/8th what Stereophile and TAS charges. It's enough to pay the staff and I but not enough that we feel terribly beholden to anyone else.

We also try to review a pretty equal mix of gear from advertisers and non advertisers, as well as only handing out a very minimal number of awards at the end of the year. There's been no list of 500+ suggested components and there won't ever be.

But honestly, I'm not in the quest for the absolute sound, never was. Everyone has different requirements for a hifi system what might be awesome for person a might not be for person b. We just try to tell you what we've heard, try the gear in a few different rooms, using as many different combinations as we can so that we find the bumps in the road and hopefully save you some time.

If you've read our magazine with any regularity, you will notice that almost all of my writers have had pretty much the same reference components for at least the last year or two. We do not depend on long term loans for our reference systems. And you'll also notice that we almost never say the B-word. Harry Pearson beat that into my head when I worked at TAS.

So in the end, even the best review can only make the lightbulb go off for you, and put a component on your short list. In the end its up to you, we're just trying to help you out.

Why does one person prefer a Lexus over a BMW or a Mercedes or a Camry? Everyone has different criteria. None of those cars are "JUNK". It's the same with hifi. I view most gear as someones personal interpretation of how music sounds to them. We obviously like the way the folks at ML do it, but we all know there are other choices.

Remember the way you felt the first time you heard ML's in a good system? But I bet you have three buddies that will tell you that "panels are junk". Does that make you want to sell your speakers? I don't. It's all personal exploration.

My three goals for this magazine are:

To help you find a hifi system that YOU enjoy

To turn you on to some new music that perhaps you didn't know about

To hopefully give you a couple of laughs by the end of the issue.

If I do that on a consistent basis, I'm good with that.

 

[David Matz]

While I can't speak for the other pubs, my standpoint from the publishers chair is that a trumped up review doesn't serve my readers or my advertisers.

I've had a couple of mfrs get mad because we wouldn't say that their $4000 CD player doesn't "Destroy" all of the $15k players out there, but for the most part, if we tell you a component is way better than it is, you buy it and are unhappy with it, chances are you will move it on audiogon, get on a board like this and tell everyone said product was awful and that we were idiots for suggesting it in the first place.

That might buy me an ad spot for a year, but it doesn't work long term.

I own my three reference systems. Yes, I got industry pricing, but I'm still out about $125k for all this stuff. The point is, I don't review gear to get more "Toys", we are trying to help you guys make a short list for stuff you might want to investigate. It's my job, it's not my hobby anymore.

And as for the advertising thing, because we don't have to print and ship 85 thousand magazines every month, our ad rates are about 1/8th what Stereophile and TAS charges. It's enough to pay the staff and I but not enough that we feel terribly beholden to anyone else.

We also try to review a pretty equal mix of gear from advertisers and non advertisers, as well as only handing out a very minimal number of awards at the end of the year. There's been no list of 500+ suggested components and there won't ever be.

But honestly, I'm not in the quest for the absolute sound, never was. Everyone has different requirements for a hifi system what might be awesome for person a might not be for person b. We just try to tell you what we've heard, try the gear in a few different rooms, using as many different combinations as we can so that we find the bumps in the road and hopefully save you some time.

If you've read our magazine with any regularity, you will notice that almost all of my writers have had pretty much the same reference components for at least the last year or two. We do not depend on long term loans for our reference systems. And you'll also notice that we almost never say the B-word. Harry Pearson beat that into my head when I worked at TAS.

So in the end, even the best review can only make the lightbulb go off for you, and put a component on your short list. In the end its up to you, we're just trying to help you out.

Why does one person prefer a Lexus over a BMW or a Mercedes or a Camry? Everyone has different criteria. None of those cars are "JUNK". It's the same with hifi. I view most gear as someones personal interpretation of how music sounds to them. We obviously like the way the folks at ML do it, but we all know there are other choices.

Remember the way you felt the first time you heard ML's in a good system? But I bet you have three buddies that will tell you that "panels are junk". Does that make you want to sell your speakers? I don't. It's all personal exploration.

My three goals for this magazine are:

To help you find a hifi system that YOU enjoy

To turn you on to some new music that perhaps you didn't know about

To hopefully give you a couple of laughs by the end of the issue.

If I do that on a consistent basis, I'm good with that.

Jeff,

Great points.

Ultimately, audio information is subjective and experiential. Here's an economic perspective on the situation:

There is a famous Supreme Court case in the United States in which a judge ruled that he could not define a concept, but knew it when he saw it. That is also an excellent example of information that is subjective or experiential. Such information is usually difficult to communicate by its nature. It is called subjective or qualitative because it is very difficult/ impossible to quantify, and difficult to rigorously describe in an economical way. Communicating subjective information usually requires sitting down and talking about the situation, explaining it verbally, and allowing the recipient to ask clarifying questions. The level of subjectivity is so high that the information is experiential – in order to really understand the information, you have to experience it for yourself.

With audio reviews, this usually does not happen - an article is written, and maybe a few letters written in with questions and follow-up comments. Usually no one has the same room or identical equipment, so it's even harder to relate to the information. Because tastes are subjective, there is even greater difficulty. I love Jeff's blog idea, but would like to see it turn into a conversation.

 

[tonepub]

Absolutely right on all counts.

That's why whenever we can, we are sending components when it's practical to one, two or even three other staff members homes, because we all have different rooms.

My main room is all treated, dedicated power,etc. but that's not really a "real world" situation. My living room has some dreadful things going on, so I always spend time with a component there too. Fortunately, Marc Phillips lives just up the street, so it's easy to let him live with it for a few weeks and one of my other reviewers is just two hours north on the I-5, so whenever we can, I let him have a listen. Sometimes I'll ship a smaller component to our guy in Southern California (our tech editor) as well.

In the case of something like the Continuum, we had about 3/4 of the staff come up here for a weekend (at my expense) so we could all try and evaluate it.

Because all gear comes here for photos before it goes to the respective reviewer, I always spend a couple of weeks with it to cut down on break in time for the reviewers and so I know what it sounds like.

One of the things that HP used to demand in the old days was that a set of speakers was tried with at least four or five different amplifiers. We always make sure to do that and sometimes we might try more than that in search of what really sounds great.

We always ask a mfr what they suggest as far as cables to see if there are any synergies, or if there are things that really DON'T work so that we can expose the bumps in the road. Almost all of the mfrs demand that you use their gear with the stock power cords, etc to establish the character of a component before using your own stuff. Again with the cables, I'll try what the mfr suggests and then try my own reference to see the difference. Some components are more fussy about wire than others and we always try to see if we can find that out in advance.

Again, we can't cover all the bases, but by the time we go to print, we have spent an awful lot of time with a component.

 

[tonepub]

"History repeats itself"

For those of you like me who have been in this hobby over 20 years will remember that's how The Absolute sound began. Harry Pearson would not accept manufacturer advertisments because he didn't want to sway the results by hand picked or tweaked units for review.

If you read the issues from the late 70's to early eighties there are some pretty bad reviews on equipment. It was always fun to read in the back of the magazine the manufacturer's reply. Later on he changed and took the money for help to keep alfoat.

One thing to keep in mind was that the gear from the 70's and 80's was really in its infancy technically and the production quality varied from brand to brand a lot more than it does today.

Most of the top mfrs. get a lot of their components from the same six vendors, so there just aren't as many variations on the theme as there were back then.

Not to mention some of that stuff did blow up! I had a Krell KSA-250 burst into flames once and a few tube amps blow big capacitors and explode.

These days most of these guys have been doing it long enough, they have figured most of the technical glitches out. There's not quite as much drama to report as in the old days!!!

One of the big reasons HP took on some advertisers was because he couldn't run the whole show himself, he needed help and you can't expect good people to work for free. We pay our writers print magazine rates, but we have good help. If we give the content away for free the money has to come from somewhere!

 

[Statman]

It's is a lot better now!

One thing to keep in mind was that the gear from the 70's and 80's was really in its infancy technically and the production quality varied from brand to brand a lot more than it does today.

Most of the top mfrs. get a lot of their components from the same six vendors, so there just aren't as many variations on the theme as there were back then.

Not to mention some of that stuff did blow up! I had a Krell KSA-250 burst into flames once and a few tube amps blow big capacitors and explode.

These days most of these guys have been doing it long enough, they have figured most of the technical glitches out. There's not quite as much drama to report as in the old days!!!

One of the big reasons HP took on some advertisers was because he couldn't run the whole show himself, he needed help and you can't expect good people to work for free. We pay our writers print magazine rates, but we have good help. If we give the content away for free the money has to come from somewhere!

I can remember sitting and listening to my Sequel II's powered by monoblock ARC Classic 120's and watching the tubes arc and fail a couple of times. To this day they are the only amps that I've had trouble with and sold to a new owner who had zero problems with them!

Yes I still use ARC products.


,

 

[tonepub]

I had a D76 that blew up a couple of times in a similar way, but as we all know, they've come a long way!

Quite a few mfrs have told me they've learned as much about shipping as building their products too! After years of shipping heavy boxes, many of them have gotten more efficient about that, eliminating some field failures as well....

 

[nsgarch]

"History repeats itself" For those of you like me who have been in this hobby over 20 years will remember that's how The Absolute sound began. Harry Pearson would not accept manufacturer advertisments because he didn't want to sway the results by hand picked or tweaked units for review. If you read the issues from the late 70's to early eighties there are some pretty bad reviews on equipment. It was always fun to read in the back of the magazine the manufacturer's reply. Later on he changed and took the money for help to keep alfoat.

"History repeats itself" I wish!! The biggest problem with "reviews" today is the word. They're really presentations, not reviews. Most magazines, like TONE Audio, do a great job showcasing products, and as such, they provide a service to the reader, since manufacturers' websites are generally inadequate. A useful review of an audio component -- something that describes performance in depth and with appropriately selected other components -- is a thing of the past and not likely to repeat itself. And I think that's not really such a big loss. I just wish the people who write up these presentations would stop calling themselves reviewers, though the hype doesn't surprise me.