Everyday people rely on Google for answers to their most personal, important and most trivial questions. How do we know if the information is reliable? A good place to start is with an information expert. We ask consultant and author Leslie Stebbins to investigate “How can I find reliable hotel reviews?”
Ever wonder how James Bond always seems to know, just in the nick of time, where the evil villain is hiding and how to get to him?
Librarians are kind of like that.
Now hold on, before you scoff and roll your eyes. Librarians are like ninjas: able to pop in and out of information sources, quickly zeroing in on reliable answers to questions.
But we didn’t get that way over night. And, in fact, with more information available at everyone’s fingertips it has arguably become more difficult to locate reliable information today than it was ten years ago. It’s counterintuitive, but with search engine optimization tricks, fake reviews, and questionable 5-star rating systems, finding reliable information remains challenging.
This new blog series was inspired by recent research[1] about how people struggle to find reliable information online, and how we often settle for untrustworthy information without even realising it. It is based in part on strategies I used for my recent book Finding Reliable Information Online: Adventures of an Information Sleuth. The book uses storytelling to illustrate the best (and worst!) ways to locate reliable every-day-life information, such as health information or credible product reviews.
The search for reliable hotel reviews is a perfect example of how millions of people log on to Google, pop in some search terms, and may wind up staying in a place that wasn’t quite what they expected.
My hotel in Italy got five stars, but it was right between a garbage dump and a highway!
In research for my book I spent months determining how best to go about locating reliable travel information, and even I was surprised by what I found. The following is a brief taste, focused on using crowd-sourced travel information.
Can thousands of travelers be wrong?
I had wanted to go to Nova Scotia for years, but knew little about it. I figured I would start the same way most people start their search for information: Use Google, find a good crowd-sourced travel site, and rely on the advice of thousands of travelers. I pop online to see what Google would choose as the best place to start my travel planning:
Google: Nova Scotia Hotels
Not surprisingly, Google brings up its own Google Reviews first, followed by other large players such as TripAdvisor.
Lesson 1: Showing up in the first 5 hits on Google or Bing might be more a reflection of a powerful marketing team than of being reliable and trustworthy
TripAdvisor is a dominant player with revenues of more than one billion dollars a year [2]. Like other sites such as Airbnb they are wildly popular because we feel like we are getting the inside scoop from people who have sampled the product, rather than having to rely on advertisers or journalists.
Lesson 2: Large amounts of user-generated information can make reliability harder to achieve
Though I am sceptical, I decide to search TripAdvisor for a great place to stay in Nova Scotia. The results are impressive. If nothing else, TripAdvisor is a handy niche search engine to almost all the hotels and other lodging information in many parts of the world.
The amount of reviews can be a strength and a weakness, though, as there are too many for an editorial team to check manually. As TripAdvisor states in their Review Moderation guide:
"TripAdvisor has millions of reviews and more than one million hotels, restaurants and attractions; therefore, it would be impractical for us to fact check the details of reviews."
One example of how the review process can be abused is when TripAdvisor got into hot water for the five star reviews and a “Top 100” rating given to a homeless shelter in Glasgow [3]. Pranksters inundated the Bellgrove Hotel with 5 star reviews describing its outstanding service, spa, and pool facilities even though it is actually a shelter housing 150 homeless men!
"Hotel did look a bit shabby on the outside however appearances can de deceitful. Luxurious rooms, spa and leisure facilities a real gem."
The manager of the shelter finally asked TripAdvisor to remove the reviews after confused tourists kept showing up asking for rooms.
Lesson 3: Always take your search to the next level if the information seems "too good to be true"
Of the 192 hotels in Nova Scotia, the Greendale Hotel * comes up as one of the best places to stay. It looks fantastic and everyone online is raving that it is ”excellent” and “life changing.” It seems strange that no one has had a bad experience at the Greendale Hotel.
On TripAdvisor you can email reviewers, so I email 16 reviewers of the Greendale Hotel; only three answer. Each of them indicates that their reviews were completely fair and not coerced, but the third reviewer ends with:
…I've been doing this out of my own interest. However, I've found some of my negative reviews deleted from TripAdvisor many times. I've complained to them. but haven't heard back.
Perhaps this explains why the Hotel has no negative reviews? I decide to dig a little deeper into the research on TripAdvisor and review sites generally.
Trip advisor clearly states in its Review Moderation Guide that:
"As long as a review meets our guidelines, we’re happy to post it whether it’s positive or negative. The majority of our reviews are indeed positive, but we do not influence or change the content of reviews."
Research shows that 79% of consumers trust advice from other consumers as much as they trust information from a travel expert or even a friend[4]. There is also a strong link between how credible a source is viewed and later purchases based on those sources [5]. It is in the interest of a site like TripAdvisor to provide reliable reviews. However, it is important to remember, that people running hotels and guesthouses can have an influence on what reviews are posted, not just travelers.
There are reported stories that indicate that negative reviews are sometimes deleted from TripAdvisor reviews, thus skewing the reviews toward the positive. Howard Orlarsch, writing to “The Ethicist” advice columnist in the New York Times Magazine explains:
I recently spent several days at a relatively expensive hotel. The place was deplorable and unsanitary, with an unresponsive front desk. I gave it a poor write-up on TripAdvisor.com … titling my review “An Overpriced Dung Heap.” The following day, the owner contacted me through the site with an offer to refund almost half the cost of my stay if I would retract my review. I accepted the handsome offer and deleted my post. Who was the most unethical: me (for accepting the bribe), the owner (for offering it) or the site (which enables this chicanery and therefore has untrustworthy reviews)?
The Ethicist concludes that Howard and the hotel are unethical, but not TripAdvisor. They warn people on their site to not engage in deceptive practices. But recently the UK Advertising Standards Authority ordered TripAdvisor to delete all references on their site to the words “honest” or “trustworthy,” including the phrase “reviews you can trust” [6] after hotels alleged that there were more than 27,000 legally defamatory reviews posted [7].
Researchers are looking into travel review sites after discovering hotels were bribing guests to write positive reviews in exchange for room discounts or money [8].“Astroturfers” now advertise online that they will write fake hotel, restaurant, and product reviews for five dollars on sites such as Fiverr.com. Competitors have also been known to write negative reviews [9]. And, while many people are convinced that they can spot fake reviews, researchers at Cornell University found that subjects did poorly in separating fake from real reviews [10]. Other research demonstrates that between 10 and 30 percent of online reviews are fake, and software developed to combat fake reviews only catches the most obvious ones [11].
Lesson 4: Never rely on one source for information for something you care about
If you use TripAdvisor or similar sites, use the following strategies to check the reliability of the information provided:
- Read the reviews; don’t rely on the aggregated star ratings. Research shows there is a 41 percent misalignment between star ratings and reviews [12]. The problem is that a simple star rating system cannot capture all the nuances involved in a review. For low starred reviews the alignment is much higher, people rate the item low and then list negative experiences. But for hotels with five stars, read through the reviews to get an accurate picture, and also keep in mind that some percentage of them are fake.
- Check the number of reviews. Researchers found that people were more apt to focus on the rank of a hotel, such as one that came in first or second, but did not attend to the number of total reviews [13]. It is safer to choose a hotel ranked several places down on the list but has many reviews. The more reviews about a hotel the less effective astroturfers will be.
- Look for trends. Try to spot information that appears multiple times. Also, look for specific information such as someone mentioning that the sheets were not clean or the beds were uncomfortable. Specificity combined with items mentioned by several people can suggest reviews might be credible.
- Ignore the extremists. Extremists are more likely to be either fake reviewers or people who tend to exaggerate. Stick to the reviews that are a mix of positive and negative information.
- Check the date of the review. If the review is more than a year or two old things might have changed.
- Check the track record of the reviewer. See if they tend to be overly negative or positive, consider emailing them to see if they are real people.
Lesson 5: Pick sources who value their reputation and do not write anonymously, who work for you not the travel industry, and who know they will only get repeat customers if they provide reliable information
For the Greendale Hotel I read reviews, checked the number of reviews, and so on. Everything checked out, but I ended up booking somewhere else after double-checking the hotel and several other places I found on TripAdvisor by doing the following:
- Using reliable travel blogs that do not accept sponsorship such as National Geographic Travel.
- Using good old fashioned guidebooks such as Lonely Planet and Fodor’s that are now making a comeback [14] (I accessed them online through my library)
- Relying on expert travel journalists who work for old and new media. (Type your destination + hotel + favorite online newspaper or trusted site.) Journalists follow a code of ethics and have a reputation to uphold.
One final trick if you want to really scope out a hotel before you get there: go on Google Earth and see what it really looks like!
*The name of the hotel in Nova Scotia used in this article has been changed to protect its reputation.