medium bookmark / Raindrop.io |

Netflix officially ditched its classic 1-to-5-star rating system for a binary thumbs-up, thumbs-down rating system. They made the in-product announcement last month with a full-screen overlay that promises ease.

In addition to the new rating system, Netflix has new match percentages (up to 100%) to more accurately predict how much users will like something:

The new rating system received 200% more ratings in A/B tests, according to Netflix VP of Product Todd Yellin.

When it comes to rating movies and shows, stars reflect the preferences that people want to have, rather than how people actually behave. Todd gave the example of users giving 5 stars to a documentary but just 3 stars to an Adam Sandler movie that they watched over and over again. “What you do versus what you say you like are different things,” said Todd.

The change has annoyed some of Netflix’s 100 million subscribers, but this isn’t the first time a major content provider favored thumbs over stars.

In 2009, YouTube made the same shift after realizing that most users only rated videos they loved, with a smaller portion rating videos that they hated. Users rarely gave videos 2-, 3-, or 4-stars as demonstrated in this J-shaped curve:

Source: YouTube

Simpler rating systems might improve user engagement for Netflix and YouTube, but switching to thumbs-up, thumbs-down might not be the right decision for your product.

How Ratings Can Drive User Engagement

Ratings serve different purposes for different products and different users. Before you can answer which rating system will work best for your product, it’s important to come back to the why. Here are some of the things that ratings can help do:

• Ratings help you better understand individual preferences and present a more personalized user experience. This is especially true for products that have sophisticated algorithms for personalization, like Netflix. The more a user rates, the smarter your product becomes in adapting to their taste and making recommendations.

• Aggregate ratings help inform users and guide their decision-making. This works for products that have high or low degrees of personalization (a brand new user on Amazon will still benefit from average ratings even if they don’t receive personalized offers). It’s especially useful for products that require longer buying-decision time.

• The user feedback helps products understand whether or not the content is valuable and improve offerings over time. This can benefit products with and without sophisticated personalization.

Depending on which of these—or other—factors are most important to your product, you may opt for stars, thumbs, or another rating system altogether.

Star ratings help users make complex decisions

While most products want to increase the speed of purchases, there might be an inherently high buying-decision time for certain goods.

A person who likes shoe shopping may want to look at hundreds of different shoes and read dozens of reviews before settling on the perfect pair. Zappos’ 5-star ratings, accompanied by even more detailed “fit survey” percentages, are helpful in guiding the decision.

Stars ratings are often accompanied by qualitative reviews, which further aid deeper considerations. This is true of online retailers like Zappos, Amazon, and Nordstrom, among others.

Qualitative reviews, with stars, increase engagement time but can delay key actions

Increasing complexity with stars and qualitative reviews is a good combination if time per session is an important engagement metric for your product.

Yelp is a site for restaurant reviews. While people can make reservations and book events on the platform, it’s primarily known as a destination for crowd-sourced reviews. One of Yelp’s engagement metrics might be around time on platform, which might be more important than the conversion from looking up a restaurant to eating a meal there.

More complex ratings, alongside 5-star reviews, probably help Yelp make their site more interesting and browse-worthy.

When users have more creativity with their reviews, the reviews can become a point of pride and meaning. Yelp has done an excellent job gamifying their reviews to massively grow their user base.

In contrast, OpenTable has a shorter path to activation. They are in the business of helping people make reservations and may not want to encourage aimless browsing that does not lead to a conversion. They present a mix-bag of stars and a thumbs-up.

This gives enough information for users who do want to browse around but also a shortcut to make a quick decision.

Thumbs-up ratings help products personalize content

One of the reasons that Netflix adopted a binary system was because they used star-ratings differently than other products. The stars were predictions of how much Netflix thought users would like something, not an average of community ratings.

Netflix Director of Product Innovation Cameron Johnson told Business Insider that people didn’t intuitively understand that Netflix’s stars would get more personalized with more ratings.

Other products that are dependent on personalization have long used binary ratings. Pandora curates radio stations based on your listening preferences. To get feedback, they use a simple thumbs-up or thumbs-down. Thumbs-down also skips the song.

Spotify also uses thumbs-up, thumbs-down rating for its radio feature, which provides new music based past behaviors.

Products with sophisticated personalization and curation abilities seem to favor binary rating systems. Simplier rating systems could help products gather larger datasets and may encourage users to make a split-second decision. 

Other rating systems to consider

Stars and thumbs-up, thumbs-down are two popular rating systems, but they’re not the only ones out there. Some products use 10-point ratings (IMBd), 100-point ratings (Rotten Tomatoes), only upvotes/likes (GrowthHackers), and emoji ratings.

While this post can give you some ideas of how other products are using different rating systems, experimenting within your own product will help you find a more definitive answer.

Netflix made their change after extensive testing. As Netflix’s Todd Yellin said, “We are addicted to the methodology of A/B testing.”