How much can anyone trust a pro with reviews?

allen levine

Member
Messages
12,340
Location
new york city burbs
I really hate to bring this up, but its something that bothers me all the time when I read reviews on tools and woodworking equipment.
I trust the advice of people here who base their recommendations on nothing more than price and performance. I think its clear Ive basically purchased most of my woodworking equipment on advice from people here or from people in the profession.(people in the profession but people I dont think have any type of financial interest or benefit in any way from reco'ing a brand)
How do I know what the pros in magazines or online are stating is actually true opinion or is an opinion tinted by free goods or actual green dollars compensation.
Woodworking is not immune from the lure of big companies shifting opinions by offering free goods or favor in some way.(Like maybe they wont give a pro free goods, but they would pay for him and the wife to fly to a show and review their products)
Is it possible to get full disclosure on reviews of woodworking tools and equipment?
Hate to bring this subject up.
If this thread is "unpopular" for this board, I understand if its removed without notifying me.
 
I think it's a great question. Kind of wondered about it myself.
Myself, I especially wonder about the pro's review of a new expensive jig or gizmo that to me doesn't measure up to a hand made one. I would expect the Pro to say that its Ok but I could make one out of scrap wood for pennies. Instead they give it glowing reviews. :dunno::dunno::dunno:
 
Allen, I have done reviews for a fairly prominent construction journal, both on tools that I had bought and also for tools that were sent by the manufacturer for evaluation. The editor that I work for has always been scrupulously fair about trying to get an accurate write-up, and I have always tried to be as fair and impartial as I could be.

I once got some tools and accessories sent to me by a major tool manufacturer, and I was so disgusted with some of the items that I told my editor that I wasn't interested in writing about the items.

I get paid for my articles by the magazine. I've never been paid by a tool manufacturer. I understand that this is not always the case, and there are many writers out there that are shills for certain brands.
 
Hard to tell. I knew I used FWW review of Hollow Chisel Mortisers to make my decision on the one I bought. It seemed like a pretty fair review. In this day and age though, it's so easy to use google to find 'real' peoples reviews of products. So I made sure that I did a good check on it before I bought it.


My purchase steps are :
  1. Use one of the in-depth type of comparison reviews to bring me up to speed on what features the machine has, what ones are desirable, and what isn't.
  2. Spend some quality time on Google finding other peoples opinions.
  3. Check out the reviews on Amazon.com, if available. I find though that if an item pretty good, the negative reviews are usually from someone with an axe to grind of some sort.
  4. Hide my credit card for a few days
  5. Think about if I really need/want it, will use it etc...
  6. Do the research again
  7. And then if I still think I need/want it, will probably order the one I originally settled on, which may or may not be the top rated tool depending on price/features.
 
Allen you hit a nerve with me. This is another pet peve of mine. I even wrote to Bob Hunter over at Wood Magazine once about this and sometime back i held a poll on whether woodworkers believe what is said in the tool reviews magazines post.

There are times when to me it is blatant that they have been bought off. The most recent one that got to me was Porter Cable launched a new sander. So they had three ads in a the mag and that seemed sufficient for the Mag to do a review of hand held orbital sanders. Low and behold the new sander scored top marks.

However unbeknown to that magazine a competeing magazine happened to do an objective review of the sander since it had just been released. Guess they did not like it cause their review was completely different and contained facts which led me to my conclusion that the Wood magazine had been "bought off" something which was denied profusely when i challenged them.

The best value i have found is from the Fine Woodworking reviews and their annual buyers guide. In the guide they add in readers views scores. Now i have to say that in all the years i have never had my view of a particular tool surveyed so I dont know where they get the readers score but at least it often differs with their assestment.

I also like the way they disclose how they do the tests.

So i think it ends up being only one source of info to look at and evaluate in the purchase.
 
Check the ownership of the magazine you are interested. Chances are the company also publishes magazines for many other avocations. e.g. airplanes, cars, motorcycles, etc.
They are in the magazine publishing business to make money. Period.
I once had ambitions to be a reviewer for gun and outdoors magazines. I had done a fair amount of writing for magazines of this type for years but never product reviews.
So, I wrote a couple reviews and submitted. Several were rejected without comment. But one editor, uncharacteristically and kindly, took the time to write me a letter explaining that my critical reviews would never get published. He candidly, and honestly, explained critical reviews would lose advertisers. My reviews must praise the wonders of the product. If I wished to write that way he wanted more work from me. That was the end of my gun review writing career. I ain't pure, but I ain't a blatant liar either. Couldn't do it.
Do read those reviews cautiously and critically. Look at the equipment in question and make your own decision. And, of course, seek help here. :D
 
I am certainly not the first to read rave reviews about a tool only to discover that a year down the road it shows its faults. Tool reviews are rarely performed a-la Consumer Reports where an item is tested for a year or 50,000 miles, etc.

I also agree that some tool reviews are just plain silly. This silliness is not restricted to any one source and that same source may give great info most of the time. I don't take anything as the end-all, be-all in my decision making. I factor it all in and read everything I can, talk to folks on the forums and email owners privately if they are willing. I have even gone to others shops and tried out things when the offer was made.

Gather all your info, mix well, allow to ferment and then decide ;-)
 
The best sources are usually ones where the reviews of products provide objective tests with repeatable results. Unlike qualitative data, quantitative data can't be fudged; it's either right or it's wrong - and if you're caught lying, you lose your viewers instantly. As a result, websites like Anandtech will frequently pan their advertiser's products, simply because there's no way to put a positive spin on a product that doesn't measure up as well as its' competitors.

If you're looking at something where the reviews are based far more on opinion than anything else - E.G, which router is easiest to use, etc. - I'd try the old standard of doing a Google search like this:

"Name of product" + "sucks"
(Note parentheses - they're important.)

It usually works for me.
 
I agree wtih a lot said here, but to add to Rob's statement about the Fine Woodworking reader's input, I have gotten roped into a 100+ question survey for them. I think it was for an annual tool review issue. I thought it would be short so I started.. then the went on forever.
 
When I was in the software business, we paid "independent" labs to evaluate our product (often a comparison of our product and the competition's), then we'd write white papers (or hire other "independent" parties to write them), and use that in our marketing materials. The tests were done honestly and factually, but they were often designed to play upon our strengths and against the competition's weaknesses. We had a good product, but we did anything we could to make it look better than the competition.

...

"Name of product" + "sucks"
(Note parentheses - they're important.)

...

I think you meant Note the quotation marks, but I agree with your suggestion. (Unless you're researching dust collectors or shop vacs.) :D With the availability of search engines these days, we can look at a wide variety of opinions (some honest, some paid for) when shopping for stuff.
 
Top