Measuring Offline Sales from Online Ads: A Case Study
A US-based specialty retailer tests the impact of online advertising on in-store sales.
- The retailer’s prior attempts to read the impact of online advertising tests had failed because of the noise and variability in daily store sales.
- APT's patented Test & Learn™ technology was employed to accurately measure the impact of online advertising on in-store sales.
- Furthermore, APT determined that the ROI on online advertising improved dramatically in markets with stores in higher income trade areas, allowing the retailer to scale back advertizing in low impact markets.
The Company:
US-based specialty retailer with over 1,000 stores
The Challenge:
The client was trying to understand if online advertising was driving offline sales. The client executed an online advertising test in a subset of markets. The problem was that daily sales varied by over 100% in any given week and by well over 400% seasonally. A change of only 1.5% in daily store sales would drive a terrific ROI but the client was not able to accurately read any change in sales in the test markets given the noise in the data.
The client looked to APT's enterprise Test & Learn™ software to reduce noise and find the true attributable impact.
The Solution:
The retailer used APT's Test & Learn for Ads™ to analyze the impact of Search advertising on in-store sales. A Search campaign was implemented in selected markets, and the change in performance of the stores in those test markets was compared to the change in performance of stores in control markets where no Search advertising campaign was implemented. The Search campaign was focused on key words relating to a specific product line. The retailer was interested in understanding what would happen to sales in its stores overall, to sales of the advertised product set, and to other product categories. This may sound like a simple analytical task. However, measuring small changes in sales at a store is much more complex than measuring changes in click-thrus on a web page. The complication is rooted in the nature of store sales.
Daily sales vary by over 100% in any given week and by well over 400% seasonally. However, a change of only 1.5% in daily store sales can drive terrific returns to Internet advertising. Accurately detecting a 1.5% change in sales in an environment where sales change radically on a daily basis, regardless of the Internet advertising, requires highly specialized protocols of test design, control group selection, and analysis of results. This is known as the “signal-to-noise” challenge.
APT's Test & Learn™ process brings highly specialized techniques to bear on the problem of reducing noise and finding the true attributable impact – the “signal” – of the activity under examination. In the case of this specific test, the signal-to-noise challenge was exacerbated by Hurricane Ike, which brought significant winds and rainfall to the middle of the country during the course of the test, impacting sales at many test and control stores.
APT's Test & Learn for Ads™ applied sophisticated analytical approaches to mitigate not just daily variations in sales but also the impact of the weather anomaly, in order to maximize accurate signal detection.
Stores in test markets with Search ads had a statistically significant lift in sales overall, a result that produced attractive returns to the advertising campaign. Sales of both the advertised product, as well as other, more “impulse buy” type items, increased significantly. The return profile improved dramatically with detailed evaluation of differential performance by store. APT identified certain market attributes that were associated with proportionately greater sales lifts than the overall average, including trade area per capita income.
The Results:
The Test & Learn™ approach provided the retailer with a means of targeting its ongoing search campaigns and investing only in those markets that would perform best. As a result, the retailer was able to scale back advertising costs in low impact markets. while maintaining a large majority of the lift and thus significantly improve ROI.
Next Steps:
