A/B Testing Ads: How to Run Tests and What to Test

An A/B test is a controlled comparison of two ad versions, where the audience is randomly split into two groups — one sees the original, the other sees the changed version — and the results are compared against a metric chosen in advance. Without a test like this, decisions about which headline or creative is better get made by feel, which isn't reliable even for an experienced specialist. Below: what's worth testing in ads, how to split the audience correctly, how much data a reliable conclusion needs, and how this works in Google Ads and Meta Ads specifically.

01 Short answer

To run an A/B test on an ad correctly, here's the order of operations.

• Pick one element to test — headline, image, or audience — and don't change anything else at the same time.
• Set the hypothesis and the success metric before launch, not after seeing the results.
• Split the audience so the variants don't show to the same people and don't compete against each other in the same auction.
• Let the test gather enough impressions and conversions before drawing a conclusion — that usually means full weekly cycles, not a couple of days.

Below: the detail behind each step, and how these principles apply specifically in Google Ads and Meta Ads.

02 What to test in ads

Almost any element of an ad can be tested, but not every element affects the result equally, so it's worth starting with whichever has the biggest potential impact.

• Headline. Usually the first thing worth testing — it decides whether someone even pays attention to the ad in the first place.
• Image or video. Especially significant on visual platforms like Facebook and Instagram, where the creative often drives the result more than the copy does.
• Call to action (CTA). The wording of the button or closing line can noticeably shift willingness to click or submit a form.
• Audience settings. Different segments by interest, demographics, or lookalike audiences, covered in lookalike audiences on Facebook and Instagram.
• The landing page. Technically not a test of the ad itself, but the ad-plus-page combination often decides the experiment's outcome — the basics of a strong landing page are covered in how to build a high-converting landing page.

03 Splitting the audience correctly

The core rule of an A/B test is that both variants need to compete for a comparable audience at the same time, otherwise the difference in results might come from external factors rather than the ad itself.

• The split needs to be random, not an artificial rule like "variant A this week, variant B the next" — seasonality and outside events will skew the result.
• The variants shouldn't compete for the same audience in the same auction — otherwise the platform starts favouring the better-performing one before the data is statistically meaningful.
• Test one element at a time — if the headline, the image, and the audience all change at once, there's no way to tell which change actually drove the result.
• Both groups need to be roughly the same size, so each variant gets enough impressions for a reliable comparison.

04 How much data and time it takes

There's no universal exact figure, but there are benchmarks that help avoid concluding too early or running a test longer than necessary.

• Wrap up a test on full weeks, not arbitrary days — audience behaviour shifts over the course of a week, and cutting a test off mid-cycle skews the picture.
• The lower the traffic and conversion volume, the longer the test needs to run before the difference between variants stops looking like noise.
• If variant B doesn't show a visible, stable advantage within a reasonable window, that's a valid result too — not every hypothesis is meant to be confirmed, and that's a normal part of the process, not a failed test.
• Drawing a conclusion from one or two days of data, especially on a small budget, is usually premature — early swings often even out as the test continues.

05 A/B testing in Google Ads

In Google Ads, testing is usually organised through a dedicated tool rather than by manually comparing campaigns, because the platform itself can split traffic correctly.

• The Experiments section lets you create a copy of a campaign with changes and run it alongside the original, splitting budget and traffic between the two.
• This approach compares variants honestly, since both run under the same market conditions at the same time instead of in different periods.
• Once the experiment finishes, the results can be applied to the main campaign or dropped if the hypothesis didn't hold up.
• The official description of the tool is in Google Ads help on the Experiments page.

06 A/B testing in Meta Ads

Meta Ads Manager has its own built-in A/B testing tool that automatically prevents audience overlap between variants.

• The A/B Test tool in Ads Manager compares two variants on one variable — creative, audience, placement, or delivery strategy — and splits impressions between them correctly.
• Meta shows a statistical confidence level for the test result automatically, which helps avoid drawing a conclusion too early.
• A test can be built for a new campaign or based on an existing one — both are covered in Meta's help centre article on A/B testing.

07 Common A/B testing mistakes

Most failed tests don't fail because of a bad hypothesis — they fail because a basic rule of running the experiment got broken.

• Ending the test too early. Stopping as soon as one variant pulls ahead is a classic mistake — the gap often narrows or even reverses as more data comes in.
• Testing several elements at once. If the headline, image, and audience all change together, the result can't be tied to any one of the changes with confidence.
• Ignoring seasonality. A test that happened to catch a holiday or a demand spike for only one variant produces a skewed result.
• No success metric chosen in advance. If the winning criterion gets defined after the results are already visible, the test loses its objectivity.

If it's worth building an ongoing ad-testing process instead of a one-off experiment, this work can be handed off — see Grottix's paid search and social ads services.

Before testing headlines, it helps to know the formulas working variants are actually built from — covered in ad copywriting: headline formulas.

08 FAQ