Experiments

Experiments allow you to determine if changes you make to a page component lead to more visitor conversions to a specific goal. You can experiment with the content a component displays and with the audience a component is targeted to. A running experiment serves visitors one of the variants of a page component, tracks conversions for each variant and optimizes variant serving based on visitor analysis. After the experiment has been running for a while you can complete the experiment. Usually you complete an experiment if one variant is a clear winner, i.e. realizes a significantly higher conversion than the other variants.

Tasks

• Manage Goals
• Create an Experiment
• Monitor an Experiment

Concepts

Experiment

An experiment tests whether a change to a page component's configuration - creating a variant of the page component - leads to better conversion. Multiple variants can be tested in the same experiment. An experiment is either a content experiment or an audience experiment.

Content Experiment

A content experiment tests whether a variation in a page component's configuration (i.e. which content it contains, and how it is presented) leads to better conversion, compared to the current default configuration. A content experiment is similar to a traditional  A/B test, although Hippo allows testing of more than two variants and uses a different, more refined algorithm.

Audience Experiment

An audience experiment tests whether a page component leads to better conversion when it is targeted at a specific audience, compared to when it is not.

Page Component

A page component is a part of a web page containing functionality which can be configured in the CMS Channel Manager.

Variant

A variant is an alternative configuration of a page component. It either has different configuration parameters such as the content it displays (in case of a content experiment), or is targeted at a different (specific) audience (in case of an audience experiment).

Goal

A page goal is defined as a visitor reaching a pre-determined page after viewing the page the experiment runs on. A goal determines when a conversion takes place.

Conversion

A conversion takes place when a visitor reaches a pre-determined goal.

Winner

The winner of an experiment is the variant tested by the experiment that leads to most conversions.

How do experiments work?

• In one experiment:
  • you can have only one goal.
  • you can test from 2 up to 26 variants of a single component.
  • you can test any number of components on the same page.
  • you can't add any more variants or components while the experiment is running.
• On one page:
  • you can run multiple experiments (each with a different goal).
  • you can run an experiment with the same goal as a different experiment on a different page.
• The chance a visitor is served a particular variant:
  • is equal for each variant at the start of an experiment, when no data has been gathered yet.
  • converges towards the 'winning' variant as the experiment progresses and more data on conversions is gathered.