Get the book

Testing Marketing Claims: Prove It’s True, Then Prove It Sells

Three colleagues at a wooden table examine a plain product sample that one of them holds up, with a closed laptop and coffee mugs nearby.

Written by

hello@inkandsignal.co

in

Testing marketing claims means checking two separate things: whether a claim is true enough to defend, and whether it actually makes people buy. Most claims-testing guides cover only the second, and usually through surveys that measure what people say they like. The stronger approach treats each major claim as a hypothesis. You state a predicted result, a measurable outcome and a kill condition, then let real response decide.

TL;DR:

  • A marketing claim faces two tests: a truth test (can you substantiate it?) and a pull test (does it change behavior?). Run both, in that order.
  • Survey-based claims testing measures stated preference. A live split test measures what people actually do. Use live tests whenever you have the traffic.
  • Write each claim as a hypothesis before launch: "If we lead with X, metric Y will move by Z, because of W." Add the result that would make you drop the claim.
  • Size the test first. At a 3% baseline conversion rate, detecting a 20% relative lift takes roughly 13,900 visitors per variant. Detecting a 10% lift takes roughly 53,000.
  • The most common failure is testing after you've already decided, then peeking at results and stopping the moment your favorite pulls ahead.
INKANDSIGNAL

Want claims that pay for themselves?

The Direct Response Marketing Playbook distills a century of tested advertising into 100 principles, 30 example ads and a 30-day campaign build.

Visit →

Table of Contents

What "Treat Every Major Claim as a Testable Hypothesis" Means, and Why It Holds

Principle 3 of the 100 field principles in Ink & Signal reads, in full: "Treat every major claim as a testable hypothesis." In plain words, a claim in your ad is not a fact you already know. It is a guess about two things: what is true about your product, and what will make a stranger act. Guesses get tested and facts get defended, and most marketers mix the two up.

The principle holds because your opinion of your own claim is the least reliable data you have. You wrote it, you like it, and you know the product too well to read the claim cold. A hypothesis forces you to say in advance what result would prove you wrong, which opinion never does.

Direct response was built on this idea. Claude C. Hopkins, whose Scientific Advertising appeared in 1923, argued that advertising should be judged by traced results. He used keyed coupons so each ad's returns could be counted separately. John Caples carried the same discipline into Tested Advertising Methods. Neither man had analytics dashboards, but both refused to call a claim good until the returns came in.

Testing Marketing Claims Is Two Tests, Not One: Truth and Pull

A person's hands drop coins into one of two identical glass jars, one of which holds more coins than the other.

Every major claim has to pass a truth test before it earns a pull test. The truth test asks whether you can substantiate the claim with evidence you hold right now. The pull test asks whether the claim changes what buyers do. Pages that rank for "claims testing" mostly describe the pull test. They skip the truth test, and the truth test is the one that gets you fined.

Regulators are explicit about the truth test. The FTC's Policy Statement Regarding Advertising Substantiation says advertisers must have a reasonable basis for objective claims before those claims run. If your ad says "tests show" or "clinically proven," you need the specific level of proof that the wording implies. In the UK, the CAP Code administered by the Advertising Standards Authority requires marketers to hold documentary evidence for objective claims before publishing.

Substantiation comes before publication, not after a complaint. A claim you can't prove on launch day is a liability, however well it converts.

A claim that wins a split test and fails a substantiation audit is not a winner. You have to pull it, and the money you spent learning that it converts goes with it.

Examples of Marketing Claims, and Which Test Each One Needs

Marketing claims are any statement in an ad that a buyer could, in principle, check. Here are common types, with examples:

  • Performance claims. "Cuts onboarding time in half." These need hard substantiation and a pull test.
  • Comparative claims. "Twice as fast as the leading brand." These carry the highest legal risk. You need like-for-like testing against the named or implied rival.
  • Numerical claims. "Trusted by 12,000 teams." These are easy to substantiate and easy to get stale. Check the number every quarter.
  • Superiority or "best" claims. "The best CRM for agencies." When a reasonable reader takes these literally, they need evidence. When the claim is obvious puffery, it mostly needs a pull test.
  • Risk-reversal claims. "Results in 30 days or your money back." The truth test here is operational: can you actually honor it at scale?
  • Health, environmental and financial claims. "Eco-friendly," "boosts immunity," "earn 8%." These are the most heavily regulated category almost everywhere. Get specialist review before you test anything.

The method you use should match the question you are trying to answer:

Method What it answers Traffic or cost Blind spot
Substantiation audit Is the claim true and defensible? Low cost, internal Says nothing about whether it sells
Survey or MaxDiff ranking Which claims do people say appeal? Moderate panel cost, no traffic Stated preference often diverges from purchase
Ad-level split test Which claim earns the click? Moderate ad spend Clicks are not revenue
Landing page or email A/B test Which claim drives the conversion? Thousands of visitors per variant Slow on small sites
Test market or geo holdout Does the claim move sales at scale? High Expensive, and slow to read

My pick: run a substantiation audit on every claim, then run a live split test whenever you have the volume. Use surveys to shortlist claims when traffic is thin. Don't use them to crown a winner. People are generous in surveys and stingy at checkout.

How to Test a Claim: The Four Claim Testers, Adapted for Advertising

Four people around a round table each closely inspect the same kind of plain product sample.

The "four claim testers" come from classroom critical-thinking and history teaching, where students learn to evaluate a claim against evidence. That is also what "claim testing in history" usually refers to. Curricula word them differently, but they generally reduce to four questions. Each one translates directly to advertising:

1. What exactly is being claimed?

You need to pin the claim down to something specific enough to be wrong. "Saves time" can't be tested. "Cuts weekly reporting from three hours to 40 minutes" can.

2. What evidence supports it?

List the evidence you actually hold. That means customer data, test results and third-party reports. A testimonial is an anecdote. It can illustrate a claim, but it can't substantiate it.

3. Is the evidence reliable and relevant?

Ask whether the data came from customers like the ones you are advertising to, measured the same way. A result from your three best enterprise accounts doesn't support a claim aimed at solo founders.

4. Are there other explanations, or contrary evidence?

Before you call a test, look for the confound. A seasonal spike, a change in traffic source or a pricing change can all produce a "winning" claim. Hold everything else constant, or you're testing the calendar instead of the claim.

Worked Example: One SaaS Headline, Rewritten as a Hypothesis

Say you run a scheduling tool for physiotherapy clinics. Your landing page headline reads "The easiest way to manage appointments." It's pleasant, impossible to disprove, and asserts nothing anyone could check. Applying principle 3 means replacing it with a claim, then testing that claim.

Truth test first. You pull data from your active clinics and find that clinics using automated SMS reminders saw no-show rates fall noticeably. You only claim the figure your data supports, for the segment it covers. The new headline becomes "Fewer no-shows, without chasing patients by phone." You keep the specific percentage for the subhead, with a footnote stating the sample and the period.

Then write the hypothesis. Use this template, which is the core of any testing marketing claims template:

If we lead with [claim], then [metric] will [move by at least X] versus [control], among [audience], because [reason]. We will drop the claim if [kill condition].

Filled in, it reads: "If we lead with the no-show claim, trial signups from paid search will rise at least 20% against the current headline among visitors on no-show keywords. These visitors already feel the pain. We drop the claim if it shows no lift after the pre-set sample."

The reason clause ties the claim to a market that already wants the outcome, which is the work covered in how to find a hungry market before you write a single ad. A claim aimed at a pain nobody feels will lose every test, however true it is.

Pro Tip: Test claims in the ad before you test them on the landing page. In a search or social ad, the claim is nearly the only variable, so differences show up faster and with less noise. On a landing page, the same claim is diluted by layout, proof and price. Use the ad test to shortlist. Then confirm only the top one or two on the page against actual signups, because the claim that wins the click sometimes loses the sale.

Size the Test Before You Start, or Don't Start

A person's hands sort loose coins into small even stacks on a desk next to a closed wallet.

Most claim tests fail before they launch, because nobody worked out whether the traffic could detect the difference. Detectable lift and sample size trade against each other harshly. You can check any scenario with Evan Miller's sample size calculator, using the standard settings of 5% significance and 80% power.

At a 3% baseline conversion rate, you need roughly 13,900 visitors per variant to reliably detect a 20% relative lift, and roughly 53,000 per variant to detect a 10% lift. Halve the effect, quadruple the traffic.

If your page gets 2,000 visitors a month, a test of two similar headlines will run for over a year. A small business should test bold claims against each other, such as a specific number against a vague benefit, or a speed claim against a cost claim. Those can produce lifts large enough to detect. Testing commas against semicolons is a hobby, not a method.

The Mistake That Breaks the Principle, and a Checklist for This Week

The common mistake is testing after you have already decided. The team loves the new claim, launches a test as a formality, watches the dashboard daily, and stops the moment their favorite pulls ahead. Repeated peeking with a willingness to stop early inflates false positives well beyond the 5% you think you're accepting. The test becomes a ritual that confirms the opinion it was meant to check.

A related failure is never writing down the kill condition. If no result could make you drop the claim, you aren't testing a hypothesis, just dressing up a decision you already made.

Apply principle 3 this week:

  • Audit your top three claims. For each one, write down the evidence you hold today. Cut or soften any claim you can't support.
  • Rewrite one vague benefit as a specific claim. It should be specific enough that it could be wrong.
  • Write the hypothesis in full. Include the metric, the minimum lift, the audience, the reason and the kill condition.
  • Size the test. Run your baseline through a sample size calculator. If the duration is absurd, test a bolder variant instead.
  • Fix the stop date in advance. Commit to the sample size, and don't call it early.
  • Log the result either way. Losing claims are data. A file of what failed is worth more than a swipe file of what you admire.

Test Marketing, Pharmacy Test Claims and Other Things Called "Claims Testing"

Several different practices share the name, which confuses search results.

Test marketing means launching a product or campaign in a limited market before a full rollout. A classic example is a restaurant chain trialing a new menu item in a few cities before taking it national. Hopkins ran campaigns the same way, testing in small towns before spending big. A modern equivalent is running a new offer to 10% of your email list before sending it to the rest.

A test claim in pharmacy is unrelated to advertising. It is a claim submitted to an insurer or benefits manager to check coverage and the patient's cost before a prescription is dispensed.

Claim testing in history is the classroom skill described above: evaluating a historical claim against sources. The four questions transfer neatly to ad copy, which is why marketers keep borrowing them.

Get a Test-Driven System Instead of Another Opinion Meeting

If your claims currently get settled by whoever argues hardest, Inkandsignal's Direct Response Marketing Playbook gives you the alternative in 48 pages for $37. It covers the 100 principles this one belongs to, 30 example ads to borrow claim structures from, and a 30-day, 8-stage campaign build. That build puts claim selection and testing in the right order: market first, truth test second, pull test third. A free principle card deck sample lets you check the style before you buy.

One limit: the book gives you the method, not the traffic. If your site sees a few hundred visitors a month, no playbook will make a subtle headline test conclusive. You'll need to test bolder claims, use ad-level tests, or wait longer. The companion Repla tools help put the method into practice, but they are not a substitute for having enough volume to learn from.

Frequently Asked Questions

Can you give me some examples of marketing claims?

Common marketing claims include performance claims ("cuts reporting time in half"), comparative claims ("twice as fast as the leading brand"), numerical claims ("trusted by 12,000 teams") and risk-reversal claims ("results in 30 days or your money back"). Health, environmental and financial claims are the most heavily regulated types. Every objective claim needs evidence before it runs.

What are the four claim testers?

The four claim testers come from classroom critical thinking and are worded differently across curricula. They generally ask what exactly is being claimed, what evidence supports it, whether that evidence is reliable and relevant, and whether other explanations or contrary evidence exist. Applied to advertising, they make a useful pre-launch audit for any claim.

What is an example of test marketing?

Test marketing means launching a product or campaign in a limited market before full rollout. A restaurant chain trialing a new menu item in a few cities is the classic case. For a small business, sending a new offer to 10% of your email list before the full send works on the same principle.

How do you test a claim?

First, confirm you can substantiate the claim with evidence you already hold. Then write it as a hypothesis with a target metric, a minimum lift and a kill condition. Size the test with a sample size calculator, run it to the pre-set sample without stopping early, and log the result whether it wins or loses.

What is a test claim in pharmacy?

A pharmacy test claim is submitted to an insurer or benefits manager to check whether a prescription is covered and what the patient will pay, before the medication is dispensed. It is unrelated to marketing claims testing despite the shared name.

Is survey-based claims testing worth doing?

Surveys are useful for shortlisting claims when you lack the traffic for live tests, because they need no site visitors. They measure stated preference, which often differs from what people actually buy. Use them to narrow the field, then confirm the finalists with a live split test.

Sources

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

More posts