Personalization Campaigns Report

Created: 21.04.2023

Updated: 09.11.2023

Author: Polina Ananyeva

To access the campaign report screen, you need:

1. Go to the "Campaigns" or "API Campaigns" section depending on the campaign type you are interested in;

2. Select the campaign and click on it;

3. Click on the “chart” icon in the row of the experience you need.

Campaign Report Screen:

On the campaign report screen, you can:

1. Select one of existing experiences;

2. Choose a period for analysis or one of the previous versions;

3. Look into details of current version;

1. Choose the metrics to display on the screen. More about metrics;

1. Select labels or specific audiences to display in the campaign report. You can also use the predefined segment breakdown. More about audience comparison;

   • Predefined segments breakdown: Select User Data in the dropdown and choose one of the available options.

   

   • Breakdown by labels Select Audiences in the dropdown and choose one of the colored labels. Audiences that do not have any labels assigned can be found via search by their names.

   
2. Switch to editing;

3. Export the report in .xls format. In this report you can see:

   1. All table information presented on the screen, including metrics and audiences you've chosen;

   2. Version details similar to paragraph 3;

   3. Information, if outliers are included. More about outliers: Outliers

   Report example:

Data Display

On the campaign report screen, you can review the main campaign metrics in charts that reflect metric changes over the selected time period. The charts display results by variations created in the campaign, for example, Variation 1 versus the Control group.

The available metrics include:

1. Number of purchases

2. Revenue

3. Number of users

4. Number of impressions

5. Number of clicks in the variation

6. Average order value

7. Variation click-through rate

8. Average number of purchases per user

9. Average revenue per user

In addition, you can examine the information with or without outliers by toggling the switch:

More about outliers: Outliers

Metrics

In the selection menu, you can choose to display any number of metrics, which are globally divided into two types:

1. Predefined key metrics:

   a. Revenue

   b. Purchases

   c. CTR - Click-through rate

   d. AOV - average order value

2. Goals defined in the platform

3. Custom metrics, such as the number of clicks in a specific block, click-through to a product card, and others (any event sent to the platform serves as a metric).

The metric set as the main one when creating the campaign is called the Primary metric. It always appears first in the list of metrics and the winning variation is declared according to it.

In addition to the above-mentioned metrics, you can also see indicators such as uplift and probability to be best (ptbb):

1. Uplift - an indicator showing the increase or decrease in terms of current metric comparing the variation to the baseline (control group).

2. PTBB - an indicator of the probability that the variation will continue showing the same increase in terms of the metric in case the test is stopped at the given moment. By default, upon reaching 95% in primary metric, the variation is considered the winner. This metric is calculated using the Bayesian model:

Bayes' theorem

How does the winner announcement look like on the platform:

Audience Comparison

In the campaign report, you can compare the performance of different audiences, e.g., desktop users and mobile users; this often helps to understand the nuances of user behavior. You can select either individual audiences or labels; when choosing a label, all audiences that have this label assigned will be shown in the table.