The problem with ‘Relevance Score’

Have you ever prepared a Facebook ad campaign and were pretty sure it was going to do well? The target audience was defined, the creatives looked good, and the copy clearly communicated your value proposition.

But shortly after launching it, Facebook’s algorithm determines that your ads’ relevance score is a 2 or a 3 out of 10. Now what?

Frantically, you consider ways to improve your relevance score: Should you adjust the copy? Maybe the creatives are too bold? Or too boring? Perhaps try different placements or another ad type?

Turns out you’re optimizing for the wrong metric

When you’re starting a campaign, you don’t have significant amounts of useful data coming in (e.g. for conversion rates or CPAs). Especially during a campaign’s learning stage, costs, click through rates, and cost per actions should be taken with a grain of salt.

CPAs vary during learning stage

In comparison, the relevance score is determined after an ad receives 500 impressions, which is why many marketers like to use it as a leading indicator for performance.

The problem is that the relevance score is a poor proxy because it doesn’t measure what you think it does.

Even Facebook’s own definition of the metric is misleading:

Facebook's relevance score definition

Here’s the problem according to a source within Facebook: your ad’s relevance score is ranked relative to other advertisers targeting the same audience. 

“Relevance score aggregates various ad quality and relevance factors to give you an idea of how relevant your ads are to the people in your target audience compared to other ads targeting that same audience.”

So you’re not actually finding out how relevant your ad is to your audience.

Instead, your relevance score is a ranking against other companies (not even business competitors), who offer varying products or services, but happen to target the same audience.

Facebook even advises against trying to improve your relevance score:

“We recommend not focusing too much on raising your relevance score. Raising it is not directly tied to improved performance. Instead, we recommend using it to get a sense of your ad’s relevance, and then focusing on improving your targeting and creative. […] we don’t recommend making changes just to raise your relevance score.”

Two simplified scenarios that highlight the problem

Scenario A: Your (amazing) ad is up against another company who is offering free ice cream to the same target audience as you.

While I don’t know your business, I’m going to assume it’s not as attractive as free ice cream…

What will happen is that more people engage with the ice cream ad and convert on the website, indicating to Facebook that the ad was highly relevant.

In turn, because your ad won’t drive the same level of engagement, Facebook deems it less relevant and assigns a lower relevancy score.

Scenario B: Unfortunately, another ad you created is mediocre at best.

Unbeknownst to you, the other companies targeting the same audience have terrible ads that are even less engaging.

The result? Even though your ad isn’t engaging, it receives a high quality score because it still does better than the other companies.

If you then increase your ad spend as a result of this false positive, you’ll end up burning money.

The better alternative – positive and negative feedback

Fortunately, there are two metrics you can look at that provide a much better indicator of relevance: positive and negative feedback.

Positive feedback

Facebook Positive Feedback Description

Negative feedback

Facebook Negative Feedback Description

You see, these two feedback types actually inform you what actions people take regarding your ad.

So what actions directly impact these two scores?

 

Actions impacting positive feedback

Users can influence your ad’s positive feedback in four separate ways:

  1. Clicking the ad’s link/CTA
  2. Time spent on the ad (especially important if it’s a video)
  3. Reacting to the post
  4. Commenting on the ad

I want to clear up two misconceptions regarding reactions and comments.

First off, all types of reactions on a post are considered positive.

Types of Facebook Reactions

An ‘angry’ or ‘sad’ reaction is as equally positive as ‘love’.

Secondly, hiding comments left by people doesn’t have a negative impact.

Responding to comments (even negative ones) might lead to more interactions and therefore more positive feedback.

But even if you simply hide the comments left by others, regardless of their content, you won’t be penalized.

 

Actions impacting negative feedback

The one indicator on Facebook that will result in a higher negative feedback score is when people hide your ad.

Hiding ads on Facebook

Let’s consider why people might hide your ads.

The three main reasons why ads are hidden are:

  1. The user doesn’t consider the ad relevant to them
  2. They keep seeing the same ad (ad fatigue)
  3. They find the ad misleading or otherwise offensive

To prevent people from hiding your ad, make sure to target well, keep the creatives fresh, set a frequency cap, and don’t be offensive or misleading.

 

Where to find your positive and negative feedback

In your Facebook Ads Manager, scroll to the far right and hit the blue plus symbol.

Simply check the “Positive Feedback” and “Negative Feedback” boxes and hit “Apply”.

Adding-metrics-to-Facebook-ads-manager-v2

Positive and Negative Feedback Metrics on Facebook Ads Manager

Translating your feedback score

Unfortunately, the feedback your ad receives can only put it into one of three categories: Low, Medium, or High.

Which of these ads would you pause?

Low, Medium, High Feedback as an alternative to relevance score

As a general rule:

  • When the positive outweighs the negative, keep the ad running (and potentially scale)
  • When the negative outweighs the positive, pause the ad and revise
  • For the cases in between, where positive and negative feedback seem to be balanced, keep a close watch. An ad that has both highly positive and negative feedback might be polarizing, which can potentially work well

Positive-to-negative-feedback-table-v2

For the example campaigns above, the following picture emerges:

Ad evaluation as an alternative to relevance score

If this was the only information I had to go on, I would scale the budget for the first ad, pause ads 4 and 6, and focus on improving the others that I’ve marked as orange.

Key take-aways

  • Facebook’s relevance score doesn’t actually tell you how relevant your ad is
  • Relevance score is a value relative to other companies targeting the same audience and even Facebook recommends to not focus on improving it
  • A better indicator of your ad’s performance is the positive and negative feedback score it receives

Ultimately, positive and negative feedback is only an indicator of ad performance and shouldn’t be the end goal.

Your campaign objectives and business goals need to be the primary lenses through which you evaluate whether or not your advertising efforts are successful.

 

What experience do you have with Facebook’s relevance score? Have you overemphasized its importance in the past? Let me know in the comments!