Did the campaign bring guests, or would they have come anyway? How to check one campaign

After a campaign, guests came to the restaurant. Purchases showed up in the report, and you want to repeat the successful campaign. But some of these guests could have come without an invitation. If you give the message credit for all the purchases, you risk paying for the campaign again without knowing what it changes.

To check your next campaign, set aside in advance a random group that will not get the message you are testing. This group should come from the same audience that fits the offer. Then compare the share of guests who came during the same period, out of everyone first put into each group. This comparison can help you estimate the extra visits, if there is enough data and the test rules are followed.

Start with a short talk with your marketer. Agree on what exactly you are testing, for what period and where you will get the visit data. Below is what to write down before sending, so that afterward you can decide whether to repeat the campaign.

What counts as the result of a campaign

An opened message, a purchase with a promo code and an extra visit answer different questions. Opens help you see how people react to the message. A promo code links a purchase to the offer by the tracking rule you chose. But on its own it does not show whether the person would have skipped the visit without the invitation.

Here is a made-up example: you remind guests about your lunch menu. Some of the people who get the message were already planning to come for lunch. You need to find out if guests from the group that got the message came more often than guests from the group without it. To do this, you prepare both groups for the comparison before the campaign.

Keep only one difference between the groups

Before sending, agree on whether you are testing the reminder itself or the offer together with it.

In the lunch example, the menu and purchase terms are the same for both groups. The only difference is whether the reminder is sent. But if you also give one group a special benefit, the comparison is about the whole package: the message plus the benefit. From this result, you cannot separate the value of the text from the value of the offer.

Choose a regular marketing invitation, one for which the person in charge has already checked that the channel is allowed. Booking confirmations and other needed service messages still go to everyone who needs them. Terms you have already promised to a guest also stay in place.

Ask the person in charge to check in advance whether the tools you have let you split the audience and count purchases the same way. They also need to make sure the group without the tested invitation does not get it through another channel. Otherwise the one difference between the groups will be lost.

Split one starting audience

The group without the tested message is called the control group. You pick it at random before sending, from the same audience that fits the offer. Regular guests and guests who have not come for a long time behave differently from the start. If you compare these two segments, you cannot be sure how much of the result comes from the invitation and how much from the difference between them.

Agree with your marketer or analyst on how to split the guests at random, and record who is in each group. Then check how active the groups were before: make sure there is no clear gap in how often and how recently guests visited. With a small guest database, even a random split does not guarantee that the groups will be alike.

A concept picture of the method, not a Loona.ai screen. The guest icons are just symbols: they do not set the group size or the control share.

Discuss the group size and how long to watch the results before the launch. They depend on the size of the audience, how often guests usually visit and the difference you want to notice. There is no standard control share or test length: the campaign you chose may not have enough data for a sure conclusion.

Set the period and count the same way

For a first test, it helps to choose a clear metric: the share of guests who made at least one real visit during the agreed period. Count it the same way in each group: divide the number of these guests by everyone first put into that group.

This way you compare shares, even if the groups are different sizes. Several purchases by one person do not turn into several guests. You can look at repeat visits and purchase amounts separately if needed, without changing the main question of the test along the way.

Before sending, check how a purchase is linked to a specific guest. If some visits are not linked to a guest, you need to know how complete the tracking is in both groups. A single guest profile helps with this task, but having profiles does not yet mean the restaurant sees every visit.

Agree on the start and end of the period, where the transaction data comes from, how you count cancellations and refunds, and when the data will be complete enough to check. Choose the time window before you see the result: otherwise it is easy to stop the comparison at the first nice number.

Keep track of undelivered messages separately. In the main comparison, keep the groups as they were first set up. If after sending you remove from one group the people who did not get the message, you will be comparing people chosen in a different way. It helps to look at delivery as a separate result of how the channel works.

A concept picture of a work note, not a Loona.ai screen. A difference in shares does not prove an effect on its own.

What to write down before your next campaign

Put your agreements into a short test card. It is a simple work document with no guest contacts:

• Which campaign you are testing: only the message, or the message together with a new benefit.
• Who is in the starting audience, how it will be split at random and who will check the groups' activity before sending.
• What each group gets and how you will make sure the tested invitation does not reach the control group through another channel. If the promotion is public and open to both groups, write that down.
• For what period you count guests with at least one real visit. In each group, the share is counted out of all guests first put into it, including those who did not get the message.
• Where you get the visit data, how you link purchases to a guest and how you count cancellations and refunds; who will check that the data is complete and when it will be ready to check.
• What other promotions and messages run at the same time, and how the team will take their effect on the comparison into account.
• Who will collect the result and when you will discuss it. Agree in advance on what will be a reason to repeat the campaign, change it, or decide that the data is not enough.

This way you can check whether the team followed the plan before you discuss the numbers. Loona.ai Analytics covers visits, purchases and frequency, but what the reports show depends on the data that is connected. Agree with the team on how you can split the audience, where visit data comes from and how to compare. The card helps you prepare this work. It does not describe fields or a built-in control group setting in the admin portal.

How to decide after the test

First, go back to the plan. Did the groups get exactly what you planned? Is the visit data complete? Did it turn out that some guests got a different invitation, or that only one group was part of an extra promotion? If things did not go as planned, look at how this limits your conclusion.

Then compare the share of guests with a visit. The difference you see is not yet a proven increase. Ask an analyst to check whether there is enough data and how reliable the conclusion is. With a small number of visits, a clear gap between the groups at the start, or missing data, a sensible result is "not enough data yet". This is more useful than a confident decision to send more campaigns because of a random swing.

If there is no clear difference, this does not prove that the messages are useless either. Save the test conditions and the result, and discuss what data you need to collect or what to change in the next plan. A new metric, picked only because the first one did not show the effect you wanted, will not answer the original question.

Even a proven increase in visits does not yet mean more profit. For a business decision, you need the sending costs, the cost of the benefit you offered and the food cost. Also, a short test may only show guests coming earlier than they would have anyway, not a lasting growth in visits.

Before your next campaign, choose one question, one source of visit data and one comparison period together with the person in charge. If it turns out that some purchases cannot be linked to guests yet, start by checking this data. Then your next campaign will give you a better basis for deciding whether to repeat it.