Whats your view on Magazine Tool Reviews please vote in the poll.

[Rob Keeble]

Hi All

I just recieved my latest copy of a popular woodworking magazine and this is the second month in a row that I am upset by what seems to be semi blatant bias in their reviews. (of course based on I presume, who pays the most for the exposure). Do they really think we are dummies. There are times when they insult our intelligence.

So I thought maybe we could get a poll going and hopefully just hopefully one of these editors (who do not have the time or decency to respond to letters you take the time and effort to send to them) might get to see what the general woodworkers opinion of these magazine led tool reviews are.

Also you may wish to add your ideas as to the process that should be undertaken when performing a tool review.

Just for the record I used to rely on these tests for new tool purchases now I would rather ask my fellow woodworkers on a forum.

rant rant rant

[Frank Fusco]

I voted 'no'. But, have to admit, they are as credible as gun reviews in gun magazines.

[Rennie Heuer]

Magazine reviews are a 'part' of my decision making process. I do not rely on them totally. Other WW's, friends, manufacturer's reputation, and hands on experience (when I can get it) all come into play.

[Jeff Bower]

I voted yes, but I too don't rely on them alone and I make sure they are comparing similar products. Sometimes they saw "mid range" and some of the tools are "high range" and some are "low range" and then some in the middle. A recent lathe review comes to mind. Don't remember who did it, but they included very old models for some companies (even though some had introduced newer models) and brand new models for others.

[Vaughn McMillan]

My vote would be for "Partially". I've seen evidence of bias in the past, and also seen reviews that didn't include the particular brand or model I'm interested in. (I've run into the "missing model" scenario with Consumer Reports quite a bit, too.)

[Rob Keeble]

So what do you think the criteria should be for going about a review. Surely there should be more openess would it not be cool to have them ask their readers what tool they would like to see reviewed. There is no excuse for not soliciting input from the readers today.
What about the use of external evaluation personnel where tests are done in a more scientifically agreed manner. Here I think of University Labs or approval bodies such as UL or something of the like.
What about disclosure of the motivation for the criteria that were selected to be evaluated and why. So we get to understand why a feature is there and why it is important that it work.

I really got the bit by the teeth on this one when I saw a review last month of sanders in two different magazines and how they differed. I wont mention the names but there was a new palm sander that has been launched on the market by a very well known brand that spends significant money on advertising in all the magazines. So up pops this review. Then in the one mag they very obviously slant the criteria to lean towards the motivation for this new tool and in the other the criteria were completely different. The results were different. So how scientific was that and how reliable was that. A good publication should have the courage of its conviction to stand up and provide authentic comment which in the end will aid a product that is deficient in improving in order to secure a place in the market.

[Brent Dowell]

I voted 'Yes', because typically they do a good job of describing what features are desirable, what aren't, etc... So I look at them as being a good guide.

I do keep in mind which of the Mfg are buying full page ads in the magazine however, as I'm sure that plays into to a certain degree.

The only magazine out there where I think you can believe they are objective is consumer reports because they don't take ads.

[Rob Keeble]

Originally Posted by Vaughn McMillan

My vote would be for "Partially". I've seen evidence of bias in the past, and also seen reviews that didn't include the particular brand or model I'm interested in. (I've run into the "missing model" scenario with Consumer Reports quite a bit, too.)

Exactly Vaughn. So what should be included (even if it is done by reference to an online report that was too lengthy to publish) is to demonstrate that every available brand in a particular market place, was solicited to participate. Then one knows that it was up to the manufacturer to decide to participate. If there were fees to participate these should also be disclosed along with whatever other conditions apply. Under these circumstances should a manufacturer decide not to participate the consumer or reader of the report than then draw their own conclusions from this. Why would you not? Unless you had something to hide or felt you were not going to get a fair hearing.

[Frank Pellow]

I could not vote because the poll does not have enough options. My answer is sometimes and it depends both on the magazine and the reviewer. Magazine reviews are one of many types of reasearch that I do before purchasing a tool.

[Bill Satko]

Originally Posted by Rob Keeble

Hi All

I just received my latest copy of a popular woodworking magazine and this is the second month in a row that I am upset by what seems to be semi blatant bias in their reviews. (of course based on I presume, who pays the most for the exposure). Do they really think we are dummies. There are times when they insult our intelligence.

Just for the record I used to rely on these tests for new tool purchases now I would rather ask my fellow woodworkers on a forum.

Rob, I am curious as to what you feel the specific bias was in this instance. I am also curious to which magazine you are referring to. When I first read your post I thought you were talking about Popular Woodworking magazine and was trying to figure out what the problem could be, but then I saw the "a popular woodworking magazine" (my bad for reading too fast). I take the magazine reviews with a grain of salt, but not because I feel they have a bias but more because I do not agree with what value they put on certain tools aspects (I may value sharpness over toughness in a chisel review). Maybe I am naive, but I feel that the magazines that I read (Popular Woodworking, Woodworking, Woodwork and Fine Woodworking) are not intentionally bias (we are only human so there is always some bias at work) in their reviews.

I put less much faith in fellow woodworkers. Few of us have worked on enough brands to have an informed opinion of what is best. We also tend to have more bias. It is hard to spend that money on a tool and be honest about it. There is a lot of justification that we need to do to feel good about the purchase. We also tend to be brand loyal. Just look at the Lie Nielson vs Vertias handplane posts in SMC & Knots.