Biased reviewing

I was interested in one of the links coming to my blog that was from the LFTO forum. Mutterings about biased reviewing are nothing new. If you think about it, all reviews are biased. What I like may not be what you like or suitable for you. “Hike your own hike” as a certain John Manning says.

I want to make it clear that apart from two items (a pair of HiTec boots and a Stickpic), I have not received any free gear. I have also only received one piece of discounted gear (a pair of Salomon Quest boots). Everything else has been paid for by my own hard-earned pennies.

You could argue that this introduces bias as, perhaps, I don’t want to admit that I’ve bought a lemon. Also my gear choices are selective rather than covering the market. However, I always endeavour to give an honest opinion in an effort to help others. Sometimes that opinion will change over time as I use a piece of gear and I find limitations or problems.

All I can say is that I am beholden to no manufacturer or retailer as I carry no advertising, neither do I receive any freebies (with the exceptions I’ve mentioned). The other thing I’d like to say is that, in theory, I don’t owe readers anything. Unlike a magazine or OM, I receive no financial rewards for doing this.

I do this for my own pleasure, but also have an altruistic motive in wanting to share my experiences so others can benefit from them. The Ultrahike reviews are a good example of this. This is a product that I purchased because it fitted a niche I wanted to fill. There is very little information elsewhere on it, so I thought an in-depth look at it was worthwhile and would interest others.

This is the way I want to do my blog. Others do it differently and that doesn’t invalidate them or make them worse. One of the good things about blogdom is the variety. Enjoy it. Remember it’s free!

 

Advertisements

28 thoughts on “Biased reviewing”

  1. I have to say this, Robin…

    Do you really care what they say over at the LFTO Forum?

    Most people reading your blog are fully aware of your impartiality and value your excellent revues. Those who don’t, are probably subscribers to the crappy LFTO Forum.

    No offence meant to the LFTO Forum, of course….

    😉

    1. No I don’t really care, but I thought it was an interesting enough topic for a quick post to state my position. Hope the hand is improving. Sad news about Stan.

  2. Great post this: I feel that all of these popular mags are showing products pushed to them by manufacturers and probably all of these publishers are on back hander’s!

    Often I feel a product, typically a jacket, gets a glowing review because the manufacturer has an endless supply because they were never as popular as they’d planned!

    Often I’ve bought it or seen it and it doesn’t match up to my requirements and of no use! Much the same when a product doesn’t get such a great review where as to me it’s functional!

    1. Most gear is OK. There are very few true lemons out there. Choice then becomes a matter of personal preference. Everyone has a different idea of desirable features and martix of weight and function. All I can give you is my expereince and views. When mags do a feature on twenty-odd jackets (or whatever), you’re only ever going to get a superficial views. It helps, but bloggers can usually give a more in depth appraisal based on real world usage. That’s what I try to do.

  3. I’ve not read the LFTO article, it doesn’t render to well on my BB. However …
    I think we need to remember that you’re running a blog. Your giving your opinion on a piece of equipment you bought and how it performed for you.
    This is very different to a magazine publishing a comparitve review of simmilarly positioned products.

    I don’t see a vast difference between what you do and a review posted to a forum.

  4. I used to frequent the LFTO forum but, after a while, I realised that blogs, unlike forums, tend to attract people who want to read (and, more importantly, comment on) the blog and therefore already have an affinity for the writer of the blog. A forum is a free-for-all and, in essence, you’re likely to find all sorts including rude, obnoxious people who feel it is there inalienable right to say whatever they think regardless of how unpleasant. I haven’t been there for some time and I see it hasn’t changed much. Some good people and some not so.

    As far as reviews are concerned – opinions are subjective, surely that is a truism? I happen to pay less attention to your reviews than some others, in fact, as your priorities are different to mine – but I KNOW that so I read the detail but balance the opinion against your priorities. I like the MYOG parts of your blog, the trip reports and what I like to refer to as the miscellany – bits of news, discussions and so on. Read and make your own mind up. It’s far more useful than the basic group tests done by LFTO/Trail/Country Walking. Also, Trail really does not cater as much for the UL/cottage manufacturers as that is not their market. The UL blogs do that.

    Finally, bias. The comments are so richly undeserved, for most blogs in fact – so many of us give poor reviews for stuff we’ve received and we’ve bought. I shelled out £300 for a Vaude Power Lizard and confessed I didn’t want to use it. I am not the first either – MacE? Montane sent me a Prism 2.0 and part of it fell off – probably just bad luck but I reviewed it. You’re not willing to confess on buying a lemon? has anyone READ your blog?? HOW many rucksacks have you had…?? Martin Rye and his Prophet – he specifically bucked the “We Love MLD” trend which was prevalent at the time. Also, we don’t buy stuff we haven’t already got a positive opinion of – reviews are as much about information as What’s the Hydrostatic Head and the material content of the Hilleberg Soulo? Go check out Mark’s blog. Why does the Lightwave Ultrahike have a red carryloop – log onto blogpackinglight. What apps for the iPhone? Andy Howell, Mark Roberts and WhiteSpider1066 will tell you.

    Finally, don’t like it, don’t read it. I applaud the time, trouble and effort you put into your blog. 242,133 times before me, so did someone else.

    1. Some excellent points. None of us have exactly the same priorities. I’m not a tarp man (yet) but I enjoy reading stuff on tarps (viz. Hendrick’s lataest post). It adds to the knowledge base and helps inform my gear choices.

      1. It just makes me salivate with excitement but then I need help. I’m reasonably proud of myself at the moment – instead of shelling out £90 on the Osprey Mutant as I indicated in my Winter/Mountaineering post, I’ve happened upon an old friend who’s willing to do a job for me instead. Post to come soon. ‘Course, it’ll be a bit like a review so you’d better ignore it 😉

  5. As someone who had posted at length in the aforementioned thread on LFTO i’d just like to say that I read and enjoy a lot of blogs including this one.
    My comments made yesterday were in response to Martin’s post of Sat pm. Ultimately blogs are the creation of the blogger. However I stand by my assertion that market research by the likes of Cision is done purely to push the marketing value of blogs to their potential clients in niche markets. Why else would they do it? I don’t think I actually proposed that bloggers were in collusion with big business but are they being manipulated?
    Robin, I was in no way implying that you are receiving anything from anybody, if that is how it sounded then I apologise unreservedly.

    1. No apology needed. I wasn’t offended. I felt it was worth clarifying my position and motivation. I think the Cision survey was complete tosh. It included a blog that was associated with a retailer! It also seemed to be completely random in its selection. Like you say, beware market researchers. I think it showed that they don’t understand what blogging is all about. I appreciate your comments on this blog.

  6. Hi blogpackinglight and Wurz,

    Hope you are well.

    Thanks for your feedback.

    I do understand your criticism for our inclusion of the retailer’s blog, we’re not trying to make judgments about the quality of content, but rather to measure the social web’s verdict as expressed through various metrics (links, comments and so forth). In this specific case, we thought that the comments suggested an engaged, appreciative community. We’ve debated the inclusion or otherwise of “commercial blogs”, but found too many shades of grey – Cision’s Top 10 blog rankings now include any independent “blog” regardless of origin or purpose.

    Always happy to discuss further.

    Kind Regards,

    @SabinaRosander

    1. Thank you for your comments Sabina. Perhaps one metric should be frequency of posts.

      The thing that puzzled me was how a blog with infrequent posts could be that influential. That’s not a judgement on the quality of the blog, just that new posts tend to drive traffic. Depth of content is also a factor, which tends to depend on post frequency.

  7. I think the whole discussion here s superb. My post on Outdoor bloggers was intended to raise such good debate. Wurz agreed with my responses to the points raised there. It got me thinking and hence my point on OB.

    For me Mark said a fine point about testing and reviewing kit for you. Blogs are about the people as much as the content. Maz really nailed a point when he talked about some reviews don’t relate to him based on his kit needs. Again we all have different views and aims. Maz for example is doing light backpacking and Alpine adventures which I have no idea about but enjoy reading how he is going about that. Blogs offer so much information and answers. Post a comment and often you get an answer. Some don’t reply but that is their blog and If they chose not to reply or don’t like you it is their choice.

    It does not matter if it was on LFTO or any other forum it is good to talk about this subject and get it in the open so those who ask can be pointed to answers. My post on OB was to ask are we being subtly manipulated with test kit to do reviews and then who is running the blogs we write if a deadline is imposed for a review for example. And that is a good discussion that we can have.

    1. It’s an interesting debate, Martin. I’m not really looking to be a gear tester as such. My blog is more about my exploration of the gear universe and what suits me. In some ways, the blog might have been a bit unhealthy as I’ve probably bought more gear than I might have otherwise. I want to settle down to a core of gear that I can rely on, rather than constantly chopping and changing. I’m now in two minds as to whether I would accept gear to test. I certainly wouldn’t want the amount of gear that PTC* seems to attract. I’d be happy to have a dialog with manufacturers, in fact I could give them some good ideas, but I wouldn’t want to feel obliged to them.

  8. Robin you will alway get my respect for being honest when reviewing or commenting about gear. You say what you feel, good or bad. In the end it is always about personal experience. A good example is your never ending search for that perfect rucksack! I hope that one day you find it. People will then have to be mindful that although it is perfect for you it may not be perfect for them!

  9. Spot on, Robin.

    I don’t read LFTO much. I occasionally dip into OM and while there’s a lot of useful info there I just can’t stand the pattern there: a thread starts with a good exchange about gear, then the ‘locals’ start their banter, in-jokes, and the like and the thread degenerates into sixth-former backroom chit-chat.

    But there’s a more serious point about the role of bloggers.

    Unless I’m mistaken, there’s a worrying trend in the business model for outdoor gear. A lot of people walk up to their local shop to try out stuff then if they like it they buy it off the net because it costs less.

    Shops realise that and they stock less and less ‘unsafe’ items, they just stock run-of-the-mill stuff that appeals to the uninitiated that don’t even know you can buy the stuff off the net for cheaper.

    So the really interesting stuff (like the Lightwave Ultrahike) is very hard to come by in shops.

    This is where the role of the bloggers becomes crucial.

    In the old days before the net the only option was your local shop. You’d browse, and try on the stuff and see what it could do for you.

    We now rely for the most part on info gained from the net and buy most stuff unseen. Manufacturers are pretty abysmal in the way they give info about their gear. For instance, they don’t give the precise sizes for things like tents (e.g. they tell you how wide a tent is at the front but never how much it tapers towards the end!) and rucsacs (with some exceptions: Osprey do give rather precise sizes for their backpacks). So it’s really a leap in the dark when you order stuff online. And even though the EU distance selling regulations make provisions for no-reasons-given returns (with full refund of even the return postage), few take full advantage of this. And if we all did, online retailers would go up the wall too!

    So with the current trend of limited supply in shops and limited information from manufacturers, there’s really no substitute for relying on well-trusted bloggers for hard facts about gear.

    Yes, there’s always space for opinion and fit in footwear and rucsacs plays a huge role. But the blogger can give you hard facts, like Robin did to me about the sizes of the Ultrahike. On a forum you may get lucky, but most of the time you get a slanging match between the resident bullies.

    Long live the blogger!

    1. Excellent points. I know what you mean about OM! I think you are right about retailers wanting to play safe. The internet has also changed the dynamics of business. The middle ground is increasingly difficult to sustain. You are either a mega brand or you are a specialist. I agree that manufacturers are generally woeful with the information they give. They are also economical with the truth on weights (are you listening Cumulus and Mountain Equipment?). Many bloggers (what me?) are a bit obsessed by gear, but I think it helps to have as many reviews as possible, especially as they are based on real world usage.

      As a general point, I’m always happy to help and give information and views on any piece of gear that I have.

  10. I think Walter, Maz & Wurz have summed it all up nicely. We listen to the bloggers because the manufacturers’ information is generally woeful. But we listen to the bloggers who have the same mindset as our own.

    I for one hardly pay any credence whatsoever to the gear reviews in TGO or any of the outdoor mags. I see them as a complete waste of trees. But this is because I know what I am looking for and most of the kit on test in the mags is not appropriate to my needs. However, I realise that for ‘beginners’ in the backpacking world the magazine gear reviews are at least a good starting point.

  11. I’ve always found your reviews informative and unbiased, as is the case generally in the blogs I read, whether people get test gear or not.

    I’d love to get loads of test gear. I don’t, but even if I did but I’m not going to stop nitpicking or pointing out the annoyances in designs. I think we have a duty to do that, and sometimes it really has an effect and pushes manufacturers in the right direction.

    I do think that some of the ‘first look’ posts tend to be a little adulatory, probably becuase of the shiny new gear effect, and not wanting to have wasted money. But I’m probably as guilty of that as anyone else excited by new gear.

    If anything, I find myself going to far inthe opposite direction. I like to find fault with gear and have reinvented my negative attitude by thinking of myself as a “problems analyist”. Cognitive psychology at work!

    I don’t think any piece of gear is perfect, so I tend to doubt reviews that don’t find a problem with something.

    Alan makes a good point about magazine reviews. They really bear little relation to reality. I don’t understand how Backpacker, for example, can claim to have tested something for 500miles and not have anything critical to say.

    1. As you say, any piece of gear is unlikely to be perfect. I agree with you and Alan that most magazine reviews are quite superficial.

  12. I have come on the tail end of this on- line debate – had a few days away with my wife. My take on it – is that I read blogs not just for gear reviews but to share experiences with like minded bloggers.I don’t mean that we all think the same. – It means that we have a passion for blogging about the outdoors. when I started back walking after a very long lay off. It was the main stream mags that I turned to for advice and still read them. BUT, a year before I started my blog, I found that the blogs ( in the main mentioned in this post) were the places I turned to for news, views and reviews.
    PTC is a blog I like, because Pete keeps me up to date with a lot of gear I would not know about.

    The same is can be said of Robins, Martin’s Maz’s and Hendrik as well as some of the blogs from the States. So what if some people get some gear given to them. I always read around the subject anyway.

    This may come back to a posting the other day on your blog Robin – I wonder how big the specialist backpacking scene is ? Blogs have filled the gap that Trail in particular does not cover and that why they are right for me.
    Mark

  13. I never doubt your impartiality, nor that of other reviewers in our little band of bloggers. I think it’s clear from the writing that these reviews are done by backpackers for backpackers, for mutual benefit and enhancement of collective knowledge and experience.

    In fairness, the forumees are correct to point out the *possibility* of bias (in general I mean), just as we speculate about possible bias in magazines who don’t want to offend their advertisers. These days it’s very hard to spot vested interests, especially on the internet with all manner of webstat-related affiliations, and their influence can be very subtle.

    I wouldn’t say magazine reviews are useless. The star-ratings and best-buy tags certainly are useless, it seems that all of the media have this infantile urge to rank everything and declare ‘winners’, but some of the factual information can be of benefit. It’s a question of sifting out the useful stuff and being aware of any innate preferences of the reviewer. They sometimes help me to narrow down the contenders if nothing else. In particular, I value Chris Townsend’s reviews, especially footwear, but I’m aware that his standpoint is different from mine in some respects, e.g. he generally dislikes membrane-lined footwear but I have no problem with them. I would always trust him to speak his mind on any really negative points he discovers.

    That’s a good point about manipulation of bloggers by outside concerns. The reviews and writings may be completely honest, but it may be that the blogger is being used nevertheless… interesting. Anyway my stance is well known: no connection with commercial concerns whatsoever (despite the regular offers!).

    1. I think it’s worth being aware of the issue and to be alert to the possibility. Like you, I’m happy to accept that the vast majority of bloggers offer honest opinions. There is the odd blog that clearly has a commercial edge, but they are obvious. CT is the only magazine reviewer that I take much notice of.

  14. Robin,

    I always enjoy your posts and most specifically you reviews (and tweaks). Without your thoughts I would possibly not have made the mods I did to my Laser Comp and so used it less than I have. So thanks for that at the very least.

    A quarter of a million hits on your blog is fabulous testimony to your successful formula. If it wasnt good, no-one would read it and make so many comments as they do.

    Lastly, I truly believe that bloggers post real information about products that I pay heed to. I try to get a balanced view on kit and so will make sure I read a few blogs first. I distrust magazine reviews because I dont know if there is a hidden agenda and sometimes find myself disagreeing with the reviewer. With your good self I know I will get honesty

    Many thanks Robin and keep up the good work

  15. Looks like I need to take blogging more seriously. I had thought we were just passing time together as we would round a bothy fire, except in a format which prevents anyone from dominating with a monologue.

    Forums (fora?) seem redundant now I’ve discovered blogging and there isn’t time for both. Bloggers have the opportunity to explain their decisions, which doesn’t always happen in chat rooms. Or does, but gets lost because someone has gone off on a tangent.

    But we are all biased. We can’t help it. That’s one reason why there are so many methodologies in the world of soft systems analysis. Perhaps we should work on coming up with a format for reviews which pushes us closer to objectivity than we would be without it. Oh, wait a minute, that could end up being really boring – dotting all the Is and crossing all the Ts.

    One thought occurs – TGO’s reviewers have listed their all time favourites in the November edition. Is their a blogger who could collate our choices? Limited to four, and one must be from the Big Three. I’ll tell you now that M&S cotton boxers would be one of my choices. And eVent would probably beat GoreTex even though GoreTex changed things so much.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s