New research sheds light on whether common approaches for online advertising measurement are as reliable and accurate as the ‘gold standard’ of large-scale, randomized experiments.

The study to be published in the March edition of the INFORMS journal Marketing Science is titled «A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook,» and is authored by Brett Gordon of Northwestern University; Florian Zettelmeyer of Northwestern University and the National Bureau of Economic Research; and Neha Bhargava and Dan Chapsky of Facebook.

«Our findings suggest that commonly used observational approaches that rely on data usually available to advertisers often fail to accurately measure the true effect of advertising,» said Brett Gordon.

Observational approaches are those that encompass a broad class of statistical models that rely on the data «as they are,» generated without explicit manipulation through a randomized experiment.

«We found a significant difference in the ad effectiveness obtained from randomized control trials and those observational methods that are frequently used by advertisers to evaluate their campaigns,» added Zettelmeyer. «Generally, the current and more common methods overestimate ad effectiveness relative to what we found in our randomized tests. Though in some cases, they significantly underestimate effectiveness.»

Measuring the effectiveness of advertising remains an important problem for many firms. A key question is whether an advertising campaign produced incremental outcomes: did more consumers purchase because they saw an ad, or would many of those consumers have purchased even in the absence of the ad? Obtaining an accurate measure of incremental outcomes («conversions») helps an advertiser calculate the return on investment (ROI) of the campaign.

Story Source: Materials provided by Institute for Operations Research and the Management Sciences. Note: Content may be edited for style and length.