Reporting on experiments

  • Treatment campaigns are considered new campaigns and do not copy metrics from control campaigns.

  • Control and treatment campaigns accrue and retain their metrics separately throughout an experiment, even after promotion or graduation.

  • After promotion, changes from the treatment campaign are copied to the control campaign, but metrics remain associated with their original campaigns.

  • Experiment campaigns and base campaigns can be differentiated in search queries using campaign.experiment_type.

A treatment campaign does not copy previous metrics from the control campaign. It is instead treated as an entirely new campaign. During the experiment, metrics for the control campaign and the treatment campaign accrue separately; each one has its own impressions, clicks, etc. This does not change when either promoting or graduating the experiment. Metrics stay where they are and are never copied over to another campaign.

After promotion, the control campaign keeps all of its past metrics and goes forward with the new changes copied into it. The metrics from the treatment campaign are still associated with the treatment campaign after promotion.

After graduation, the control campaign and treatment campaign continue to exist as separate entities and each one keeps its own metrics for reporting.

You can report on all manifested campaigns in both control and treatment arms the same ways you could report on regular campaigns. You can differentiate a experiment campaign and a base campaign in your search query by selecting campaign.experiment_type, which will be BASE (for the control) or EXPERIMENT (for the treatment) to differentiate the type of campaign.