25
1 Metrics Therapy—Details, Dashboards and Diligence By Lauren Freedman President, the e-tailing group Research Sponsor: Personalized Shopping Experiences
1
Metrics Therapy—Details, Dashboards and Diligence By Lauren Freedman
President, the e-tailing group
Research Sponsor:
Personalized Shopping Experiences
2
Table of Contents
I: INTRODUCTION/LETTER FROM THE AUTHOR ...............................................................................3
II: WHAT’S BEING MEASURED AND WHY .........................................................................................5
III: CONVERSION AND AOV DYNAMICS ............................................................................................8
IV: EMERGING METRICS - MOBILE, SOCIAL AND CROSS-CHANNEL CONVERGENCE ......................... 15
V: A MERCHANT’S METRIC PLAYBOOK ......................................................................................... 21
VI: ABOUT THE COMPANIES ......................................................................................................... 25
3
I: INTRODUCTION/LETTER FROM THE AUTHOR
As I began to think about metrics and develop a potential position for this white paper, I quickly realized
that given device and channel proliferation, the dynamics are more vital than ever. Baynote, our white
paper sponsor was wise to select this topic, given its many dimensions and colorful dynamics. While
merchants continue to ramp up their ecommerce and cross-channel investments, according to our Annual
Merchant Survey (completed by 147 merchants in 1Q12), the following question clearly stood out from an
analytics point-of-view: “Please rate your top merchandising and navigational tactics for customer
retention with 5 being the most important to your strategy and 1 being the least important.” Evaluating
analytics to see what works topped the list with 93% of retailers recording a top-2 score (most
important/somewhat important). In fact, 61% rated it as “most important,” sending a clear message about
the mission critical nature of this topic.
Source: e-tailing group 2012 Merchant Survey
To understand how merchants are thinking about metrics today, and futuristically, I developed a set of
questions and reached out to the merchant community. A cross-section of retailers discussed a range of
topics including the metrics most important for running their businesses and whether, for analysis
purposes, senior management focused on the same or other statistics beyond those being used by
business units. Starting with the dynamics of conversion and the average order size, quantitative findings
from our 2012 Merchant Survey are integrated while qualitative color commentary provides a broader
perspective on these key metrics and the behavior shifts being seen among those surveyed.
4
Driving up conversion was part of the conversation with a goal of understanding the important influencers
that are elevating this metric where testing, dynamic merchandising, promotions, assortment and
segmentation are in play and often lead to stronger selling initiatives. Successful efforts require dedicated
individuals, focused on the details, with continual attention to development and dashboards that facilitate
decision-making. The pressure of having to make timely decisions when deadlines loom requires a rigor
and diligence not always exercised.
Most merchants struggle to maintain the right measurements to drive business, including having ample
resources to review statistics in the detail necessary for wise decision-making. Today, as devices and
social dynamics accelerate, metrics are emerging to meet the needs of marketers, though all are not yet
accessible to the degree retailer’s desire. Understanding these dynamics will only become more complex;
the challenge will be to put forth the right marketing mix where both engagement and sales performance
are part of the retailer’s evaluation process. As cross-channel selling plays a more integral role for
retailers and tracking across touch-points becomes even more crucial, this topic was explored in our
interviews as well as via an expanded question set in our Annual Merchant Survey. Taking a holistic view
of the numbers and one’s customers will be imperative as competition remains fierce and consumers
continually demand more in their shopping experiences.
Lauren Freedman
President, the e-tailing group
5
II: WHAT’S BEING MEASURED AND WHY
Interviews began with the question, “What are the top five most important metrics for running your
business and why?” Despite talking with a range of individuals who wore multiple hats across a wide mix
of businesses, many of the same responses were continually in the forefront. The goal of this question
was not to create a comprehensive list of metrics but rather to zero-in on those that really matter and
surface top-of-mind throughout these discussions. After more than 20 interviews, these five key areas
emerged.
Rather than focus on shared standards, we have consciously chosen to reveal a handful of stories that
shed light on how metrics are being used. Profit is the one universal constant across channels, making it
the weightiest metric. As one merchant bluntly stated, “Commerce is about making money so the metrics
we track must therefore assess our performance from that perspective.” Conversion rate may get more
media attention and is still important but when it comes down to it, gross profitability is what counts. This
particularly becomes an issue in low margin categories as a few percentage points can have a major impact
on overall performance. One such story shared relates to understanding the real cost of promotions (Buy
one shirt; get another at 50% off). Not only did this sporting goods retailer discuss margin implications but
reminded others of the operational charges incurred to pick the second item. Depending upon one’s cost
structure, customer discount and low margin financials, such a promotion may not be supported with strong
financials. Average Order Value (AOV) is important but the more you discount, the more you erode gross
Business
Profitability
EBIDTA
Margin (gross/product)
Year/year performance
Net Operating Income
Channel
Traffic
Visitors
Conversion
Revenue
AOV, Items per order
Customer Satisfaction
Performance by trading area
Marketing
Customer acquisition cost
List size
Traffic source
ROI on marketing programs
Sales per email
Engagement
B2C: # of new customers
B2B: Active accounts,
retention, # of new accounts
Merchandising
Inventory and turn
Top sellers
Laggards
Performance by brand
Operational
Load time
6
margin which also leads to the degradation of profitability. Difficult choices must be made when the pursuit
of profitability is put ahead of top-line revenue numbers.
Comp results have always been core to retail measurement and ecommerce is not shielded from such
year-over-year evaluations relative to both plan and budget. Several interviewed noted that senior
management takes particular interest in such comparisons to benchmark internally, against competitors
and the industry at large.
Inventory levels, while a balance sheet item, were another fundamental point of discussion as
overstocks that demand liquidation can have repercussions on the bottom-line. Of course, the cost of
goods sold relative to the inventory, having multiple implications, is an important metric as well. With
inventory numbers come turns and sell-through as these performance dynamics ultimately drive the
merchandising numbers.
Lastly, it was good to see customer satisfaction surfaced where one manufacturer spoke of it being the
lifeblood of their repeat customers. Within another organization these satisfaction scores set off senior
management alarm bells particularly as support is core to their business model.
Understanding Metrics in the Context of One’s Business
Business strategies are often in flux and depending on the direction, there will likely be metrics
implications. Understanding corporate goals and objectives is advised. While one B2B company had long
relied on a loyal, core customer base, they now have shifted to a customer acquisition mode, relying more
on digital advertising instead of print to drive traffic. As a result, initial KPIs are lower, but still within
planned target ranges. Another company is growing from their base and has seen shifts in performance
metrics including a higher AOV. Sacrifices are often made in support of corporate goals like one sporting
goods multi-channel retailer that was willing to accept lower conversion rates to achieve a more balanced
mix of page views. Their evaluation of the number of total page visits/page type shed light on the use of
the store locator and product finder features which do not lead to direct onsite conversion but instead
support overarching business objectives.
Choices are also made on how much traffic to drive when the composition of those traffic patterns can
easily impact conversion rate. Notably, catalogers bring their own metrics to bear where understanding
sales per 1,000 pages circulated assists them as they weigh their dependency on print and its ability to
cost-effectively drive business.
For those retailers without national coverage, an understanding of traffic is essential to see how much
falls within one’s trading zones or beyond. Within one’s trading zones, customers are familiar and can
bring their past shopping to bear. In fact, conversion within these zones is three times stronger than
outside. These retailers seek to understand if their digital platform can support shoppers outside of their
trading area while also informing them of what effect the web has on overall store-based business.
Additionally as category and product purchase behavior ebbs and flows, conversion and profitability
can be impacted. One consumer electronics cataloger lamented the decline of TV sales, once a much
larger portion of their business. They are careful to monitor such shifts, particularly in the all-important 4th
quarter when their natural reaction is to observe what may be wrong and how it can be fixed, trying to
make sense beyond the dashboard.
Marketing programs saw more granularities in the analysis starting with email where sales per email and
email engagement including click-through versus open-rate were standard fare. Traffic source was
explored beyond standards where several retailers evaluated results by category (i.e. backpacks via SEO
7
versus paid search) or even top sellers. A look at decliners has also served retailers as an excellent
starting point prior to examination of competitors and SEO performance along with price dynamics to
quickly make adjustments within the context of one’s business. Another touched on knowing their
brands’ ability to pull in customers and how best to allocate resources through an understanding of the
company as brand-related search terms (brand+Calphalon vs. Calphalon). This has helped shed light on
the real value of the brand name to better know where to spend digital marketing dollars.
Finally, the role of bounce rates as an indicator of overall business was instructive to assess where
people leave the site and more importantly why. Whether jumping to the call center or simply leaving the
site, it can give companies a quick read as to whether they are delivering the right initial visitor experience
or not. Merchants just might want to consider this top-line perspective as the part of their analysis.
8
III: CONVERSION AND AOV DYNAMICS
Conversion continues to get significant attention within business units and among senior management as
this singular number serves as a critical directional indicator. Caution, however, that the measurement
among merchants is varied so an “apples-to-apples” comparison can be hard to achieve.
Despite that challenge, we will still spend significant time on this topic given its impact on top-line sales
and the role it plays for every retailer. Beginning with insights from our Annual Merchant Survey, 52% of
merchants are seeing conversion in the 1-3% range. Relative to 2011, these results indicate that
somewhat lower conversion rates are being seen across-the-board.
Source: e-tailing group 2012 Merchant Survey
When asking over the course of the past year (2011), “How have the following key metrics been tracking,”
60% of participating merchants reported somewhat/significant increases in conversion. We heard similar
stories throughout our interviews as merchants continue to refine the customer experience and drive
more qualified traffic to the site, where positive results are forthcoming.
Source: e-tailing group 2012 Merchant Survey
9
Shifting Priorities Alter Conversion Numbers
As touched on earlier, priorities can have a critical impact on conversion where some may be purposeful
and others more unintentional. These must always be factored into the discussion when reviewing
results. One cataloger who has seen their conversion dip slightly does not view this number as a
business indicator but rather part of a broader customer acquisition strategy. They are revamping
branding to better target their corporate and small office/home office clientele and expect this trend to
continue during trial periods.
From another vantage point, a cross-channel sporting goods company has altered their strategies and
understands that a strong percentage of visitors to the site never intend to buy online yet a unified
corporate goal of supporting stores may impact conversion numbers as it did for them in 4Q11.
Additionally the competitive landscape which involves significant price comparison, fueled by mobile,
often brings incremental traffic but not necessarily more sales.
On the plus side of conversion, brand advertising aided one technology company we interviewed with
stronger numbers resulting across many of their global regions. Seasonal variances were also observed
where more than one merchant mentioned spikes throughout the year experiencing challenging
conversion numbers in Q411.
Promotions and Profitability Goals Impact Conversion Numbers
Merchants have to make choices about the best way to achieve profitability and while conversion may be
flat or take a hit; their bottom-line is bolstered. One retailer attributed their down conversion rates to a mix
of factors but mostly because their customer became so accustomed to couponing that when they were
not available, they didn’t buy. Satisfied with a revised approach, they are now keying in on higher product
margins; sacrificing discounted product and top-line sales. Others who cited flat numbers also
commented that despite no longer offering free shipping, they are happily meeting sales goals with more
profitable results.
Choices in one’s promotional strategies are often made like an apparel retailer that chose richer
promotions rather than “slash sales” to even out the spikes. No matter the methodology, all merchants will
continue to come under scrutiny so explanations may be warranted internally to satisfy inquiring minds.
Mobile Wreaks Havoc on Numbers
A multi-channel apparel company spoke of conversion rate fluctuation where their numbers are “all over
the place.” Emphasizing that 20% of their customers may visit via mobile at least once a week, they have
learned “not to freak out with focus on a daily conversion number” with knowledge that if they strip out
duplication, the downward dynamics will not be as scary. As a seasoned merchant, her biggest challenge
is to educate internally ensuring that senior management grasps the why behind the metrics.
Tablets too are having an impact as these couch consumers tend to browse a great deal but don’t always
finish their purchases via that device despite additional time spent. With these devices growing in
importance, initially retailers hope to better understand shopper behavior and most importantly to secure
the sale in any channel, particularly as many sites are not fully mobile optimized.
The Traffic Factor - Many Ways to Read
Another company expressed disappointment that a 22% growth in traffic still resulted in conversion being
down. A third felt that conversion remaining steady was a positive given traffic growth and a somewhat
higher online spend. Traffic levers can be turned on and off so in this case, it is about goal setting and
monitoring to achieve desired objectives.
10
Assortment Challenges
Sometimes one need not look farther than assortment to find the answer to current conversion rates.
Assortments are always central to convincing shoppers to buy as one candid accessories company
lamented performance woes, “We did this to ourselves with our merchandising assortment.” On a more
positive note, a home merchant added to their assortment last year through an extension of their vendor
drop-ship network, going deeper online and moving beyond stores into brand new categories. This
coupled with increased supporting content, including videos, has proved powerful for conversion.
Focus on the Customer Experience
Other merchants spent significant time focused on the customer experience and their payback ideally
comes in the form of improved conversion rates. Consumer adoption is still a topic of discussion and
essential for one mass merchant who reported that they started with a low baseline but saw conversion
numbers up 11%. With more people adopting online buying they believe their customer is finally ready ---
and proof is in the numbers.
Another improved site experience included representing shopping tools enabling customers to get to what
they want more quickly. Time spent on tweaking search algorithms also yielded results as onsite search
usage remains high on many sites. Operational strides were made including improved load times and
limiting of error messages, all contributing to reduce bounce rates and boost KPIs across the board.
Work on product page pricing and associated messaging came up in one discussion where call-to-action
buttons improved the user interface while also impacting conversion. Supporting content was seen as
making a difference for shoppers where both engagement and subsequent conversion goals were met.
Last but not least, a simplified buying process was suggested by several retailers based on improvements
in the shopping cart experience delivering positive results. Through testing and optimization, one
technology brand reported great strides which were reflected in their conversion numbers.
Conversion Influencers Start with Knowing your Brand and your Customers
Most retailers realize the value of testing and our Merchant Survey participants concurred as 71% saw
testing as somewhat/very important relative to retention.
Seizing on the value of testing is a case of the haves and have-nots. We start with those taking
advantage of testing today who reinforced that without employing it, you cannot not know what really
works. Commentary around testing included the following positive reflections which brought about
improvements and in some instances a material difference in results:
“..used 3rd party software to alter visual experience and navigation including search bar and
product page all leading to increased conversion.”
“…allows us to moderate promotions (i.e. test a $5 vs. $10 offer) without merchant bias as well as
layout, features and design.”
“…doing some testing in email where we learn which things to tweak resulting in better
merchandising.”
“…conducting mostly via email including creative and subject line.”
“…testing and targeting and then tweaking as there are so many opportunities to drive up
conversion.”
“…can't say it's in an absolute true science as there are always variables but value being seen
from taking advantage of testing.”
11
Some of those not in the game know what needs to be fixed but hope to get started and learn from
expected findings while adding more dynamic elements into their shopping experience via
personalization. Others look to shopper pain-points and make necessary changes to impact KPIs.
Those who currently cannot test are anticipating the role it will play in their business with several
intending to add functionality this year. They appreciate its value but simply lack the resources to
leverage the results. For others, operating on an ad hoc basis or lacking a comprehensive action plan,
testing without resources may not be the best course of action.
Dynamic Merchandising Delivers Desired Conversion
From the simplistic to the sophisticated, merchants reported positive feedback regarding the role of
dynamic merchandising. While some have still not taken full advantage of such tools, others are under-
employing as one shared, “We supply canned recommendations of what customers should buy and must
haves including top-10 lists. We hope to be able to elevate recommendations in the future based on
profiles but aren’t there today.” There were several instances where retailers feel that such merchandising
is not a good fit with their brand as customers do not buy the same product over and over again, making it
tough to predict future buying behavior. Likening the challenge to engines that don’t understand the
nuance of their product assortment, one executive believes a right brain/left brain merchandising
approach better serves their needs.
Getting customers to the right product as quickly as you can should have a huge impact on conversion.
While these observations reflect the sentiments of a few companies, my sense of the discussion is that
experiences have shown retailers make more money with dynamic merchandising. A sporting goods
retailer emphasized, “If you don't do dynamic merchandising, your conversion will just be lower.” While
some still merchandise manually, others utilize sophisticated engines and report they simply “set it and
forget it as they let the algorithm take control.” Their only regret is that they need to implement it in more
places to gain even greater value. One specialty apparel retailer finds that their merchandising “cocktails”
inform the site on how to best structure the page but she still faces internal resistance as merchants may
say, “I hate the way it looks" and trust of the black box and wisdom of the crowds has not yet permeated
the entire organization.
Those who revealed strong performance have learned how important such recommendations are to move
their business forward. Conversely, those still operating in manual mode disclosed seeing return when
they put time into it making changes based on analysis resulting in increased conversion, though not
scalable longer-term.
Accelerated Promotions - Merchant’s Choice but Not for Everyone
Accelerated promotions did not see the traction of other tactics already discussed. Utilization of this tactic
was more about go-to-market vision, though one can easily become dependent on watching what’s
moving the needle and taking markdowns to maintain that momentum. Rather than focus on accelerated
promotions as a business model, savvy merchants suggested that managing inventory by looking at a
one week supply, then constructing open-to-buys was more efficient. A number of retailers have taken an
“everyday low price” stance instead, placing an emphasis on customer service and shipping.
Others address promotions on more of an ad hoc basis, employing the tactic when it makes sense. A
smart consumer electronics company stressed, “We can sell more but it doesn't mean we will apply that
method.” A pure play retailer equated accelerated promotions to power-house tactics that as short
duration events can give people the right offer at the right time, zeroing in quickly to achieve desired
conversion results. Another followed that same train-of-thought, viewing them as part of the
merchandising tool kit, “They help to move product quickly, yet we are still cautious that profitability must
12
remain a factor.” Some surveyed retailers observed their customers continuing to gravitate to free
shipping so integrating this perennial favorite into marketing and promotional cadence was deemed most
important. Overall merchants are careful to manage accelerated promotions and one even cited cutting
back on such promotions within their affiliate networks as sales via that marketing channel had increased
from 16% to 25% of the business, causing undesired margin erosion.
There was talk of testing a variety of tactics like Groupon in 203 markets, though strong results were
overshadowed by the high cost of participation, causing one mass merchant to change the way they
manage promotions. This merchant, like all others, must continually weigh the role promotions will play in
their business with a clear understanding of financials and the ever-changing competitive landscape.
New Warrants Attention and Feeds Desire
Many expressed the importance of new product with one gifting merchant equating it to “the oxygen of
our business or our lifeline.” Even those who did not place a huge importance on the tactic acknowledged
that those who buy new product often spend more. Categories will determine how integral new is to one’s
success and even when not top-of-mind it can be seen as a way for all retailers to make valuable
merchandising statements. Comments such as, “new motivates people to buy more” and “when there's a
good offer they buy more often” illustrate that the marriage of these two tactics may manifest itself in
stronger results. “Despite not having a stellar track record, we understand it’s important to have new
product and features on the site,” suggested one merchant while another concurred, “It’s essential to
communicate with the customer about fresh product.” New can also be opportunistic when merchants
are evolving assortments, as one mentioned, regarding electronics. The beauty of new is that when
introducing fresh and compelling products one typically does not have to discount as consumers are often
willing to pay full price. Merchants will always find creative ways to communicate with the customer about
the latest products onsite, by email or even via social and mobile communiqués.
Assortment’s Role in Elevating Conversion
It is apparent that merchants must revisit their approach to assortments as customers look to solve
problems via the quickest means possible, seeking out one-stop experiences, particularly in a commodity-
driven society. Maintaining complete assortments to remain competitive, especially where drop-ship
comes into play, allows specialty stores to compete in both brick-and-mortar and Amazon dominated
worlds. While breadth matters for many, others expressed that their products and brands are unique so it
is about having the “right” product and not necessarily about how broad the mix may be, though of course
both work hand-in-hand.
Commentary around assortment was insightful and often ran tangential to new product. Many considered
it instrumental in terms of influencing conversion. Product extensions, marketplace and drop-ship models
were almost always seen as a value-add and received mention in regard to broadening assortments,
particularly where inventory implications were not a factor. Being able to extend offerings was reported
from B2B office supply players to a niche lifestyle merchant who believes they can be many things to their
customer. Another extends offerings via online-only product where today it represents 5% of their
business driving breadth from key brands. While for some it is simply an expected component, others
emphasized that their stores will always have the products their customers expect. By visualizing one’s
assortment with the right product in play, one mass merchant was able to reap broader sales across the
category. In the end consensus about broad assortments versus a curated vision needs to be reached so
merchants can focus on the right direction for their business.
Personalization/Segmentation - Still a Promise
Optimization via targeting and segmentation is a continuum where many are not even in the game.
Personalized emails typically get the greatest attention given many vendors’ ability to execute on this
13
front. On the whole, retailers acknowledged wanting to become more sophisticated regarding
personalization, but it is seen as difficult to get prioritized because many other baseline fundamentals or
proven tactics take precedent. Results from our Merchant Survey once again extend an understanding of
merchant status relative to personalization.
Source: e-tailing group 2012 Merchant Survey
One apparel multi-channel merchant shared that personalization is core to their strategic plan. Knowing
they need to do a better job, a project is in the works to enable looking at their customers across all
channels. They hope to understand a customer’s path-to-purchase and whether a given email drives
them to the store or even to contact the call center. Once they have this information, their ability to
segment and market accordingly will be vastly improved. Another viewed personalization as the Holy
Grail, aspiring to get there but still struggling with the concept and admitting that they must be more
proactive.
Personalization via segments rather than on a one-to-one basis was seen as the norm since many lack
resources to invest beyond these efforts. Targeting can become challenging when audience segments
are so small that despite a strong performance, the effort expended exceeds the dollars recouped.
Several merchants did mention tying their personalization to testing as a way to maximize both efforts.
Working scenarios ranged from marketing to brides one month before their weddings when they spend
the biggest dollars to a personalized “new to” file welcome series that has initially improved conversion
and will become even more impactful over time. Recently browsed and remarketing also see results that
should make retailers stop and take notice.
General sentiment was that if segmentation is done well; specific to a person's needs and desires, it can
be highly effective. Conversely a gifting merchant was adamant that segmentation makes his business
more challenging and overcomplicates their marketing efforts. The promise of personalization will be
realized when the devotion and a need to further elevate conversion makes it a game-changer for more
companies.
Assessing the Role of AOV
Up-sells and cross-sells that contribute to the bottom-line are still eminent though somewhat challenged,
according to our latest merchant report. Assortments and promotions were most often cited when
rationalizing changes to AOV with diligence in assortment planning achieving desired goals. Pressure on
14
pricing and subsequent price points was often a self-fulfilling prophecy when it came to AOV where many
saw swings throughout 2011. Seasonality was also touched upon as ensuring numbers are met and may
also slightly denigrate final AOV numbers. Free shipping thresholds also force down AOV, particularly
when retailers are matched to more formidable competitors. Sometimes category price drops or a lack of
a hot product made comparing last year’s AOV numbers more difficult as well. One retailer noted the
shifting “do it yourself” mindset severely impacting AOV when their customers gravitated to lower price-
points. On a positive note, personalization tactics that performed aided in driving up AOV and product
recommendations saw value for those pursuing such an approach. Others simply attributed the uptick in
the economy as aiding their efforts.
Source: e-tailing group 2012 Merchant Survey
15
IV: EMERGING METRICS - MOBILE, SOCIAL AND CROSS-CHANNEL
CONVERGENCE
The pace-of-change requires retailers to deliver on a real-time basis where end-of-month numbers do not
provide sufficient lead time to adapt and meet the concerns of connected customers. Additionally as
channels and devices proliferate, increased measurement complexity grows exponentially. Based on
these evolving circumstances, along with retailers’ desire for a more holistic view of their business and
their customers, a brief discussion of mobile, social and cross-channel dynamics plus their supporting
metrics is in order.
For retailers to deliver holistic experiences where web, mobile and stores operate in concert, new screens
need to be optimized to serve shoppers wherever and however they want to be served. This will be a
work in progress over the next few years as technologies evolve and merchants adapt to ever changing
opportunities. While some tracking was specified, the overarching goal of knowing “we have to be
everywhere and that is our #1 priority,” often superseded the right metrics.
Mobile Delivered
Mobile traffic was expected but the actual volume achieved surprised even the staunchest believers.
Mobile was primarily viewed as a bright spot and area of growth. Double and triple-digit visits and gains
were not unusual and those who had optimized their sites for the phone felt they were in a better position
to capitalize on the traffic. Words like “significant business from the smart phone” to “we’re believers in
mobile” were frequently heard. Our e-tailing group Annual Merchant Survey reported that 43% of
merchants saw traffic in excess of 5% and corresponding sales for 35% exceeded 3%, following closely
behind. Beyond sales, the importance of store-based services was noted and tested by one multi-channel
retailer who saw locating this information as paramount to performance.
With such open and browse interest that many sites are seeing on smartphones, sadly not all merchants
are optimized so user experiences may be compromised. Some retailers report shorter engagements on
the smartphone and higher bounce rates. Email readership on the phone in particular has wreaked havoc
on delivery timing. Quick information appears to be a priority as consumers seek to secure store basics
including the phone number and hours of operation by using the phone as a utility versus a shopping
device. Understanding behavior will aid retailers as they further refine mobile strategies. Additionally
some are testing mobile campaigns to gain a better understanding of mobile consumer behavior for future
digital allocation. From a metrics point-of-view, one retailer spoke of a weekly mobile dashboard given its
growing importance but like the majority of those tracking, it was composed of standard demand metrics
including sales, conversion and AOV.
Tablets Take Off
While the smartphone performed it was the tablet that peaked retailer interest. Most saw strong traffic and
high conversion. One retailer even testified to “hockey stick upticks” in their business. Apple’s iPad
dominates the market according to most retailers interviewed so it is logically where their focus lies.
Tablets are following along the lines of more traditional ecommerce experiences with KPIs appearing to
run comparable to web stores though many are not yet distinguishing mobile’s performance from the rest
of their business. Paid search is being tested by some retailers to understand performance levels where
keyword optimization is sought to better determine future spending objectives.
Apps lend themselves to metrics including the number of downloads, app store ratings and the device
being used. Operating systems are often known and devices noted. For the mobile web most are
following the traditional “same metrics/different device” approach (number of orders, AOV, conversion,
16
visitors) instead of looking at mobile performance through a different lens. While rolling into the aggregate
can work for now, long-term this will become problematic.
Performance starts with conversion and here according to our Annual Merchant Survey tablets stand out
with elevated conversion results particularly noteworthy as they are in their infancy.
Source: e-tailing group 2012 Merchant Survey
The frequency of access also means paying greater attention as this customer is attuned to changes and
experiences; often more fully engaged with the brand. It is still early, but merchants know the time will
come where distinct measurement expectations will be required to truly understand mobile performance.
In order to optimize they will want to understand browse patterns as well. Knowing the growth projections
expected from tablets, retailers should ready themselves for future gains and start deciding which metrics
to measure. Issues some faced when outsourcing mobile included reporting often not up to par and
irregular delivery of the numbers. Whether one outsources mobile or develop it in-house, creation of
merchant-driven dashboards is mandatory.
Social Looks to Evolve Beyond Engagement
While mobile warrants the lion’s share of attention, retailers are also looking at ways to measure social
dynamics. On that front success is gauged via more simplistic means where an engagement orientation is
the order of the day. As a result, some retailers perceive that a customer who comes from social needs to
be treated differently than a legacy shopper. Our Merchant Survey revealed less sophisticated
measurement tools with tracking of “followers” topping the list. Click rates and growth numbers are being
observed while sales lag behind, showing limited traction. Such rudimentary tracking is being used to
garner early learning and in some instances helps to justify the investment being made into this new
media.
Generally, skepticism and many contradictions surround social for retailers where learning and waiting
best describes merchant sentiment. Efforts center on testing but most report traffic that does not result in
sales therefore metrics have been less of a focus to date. Some merchants do believe that social may
17
drive branding so understanding engagement has drawn attention, despite being difficult to measure for
most. When a product or brand goes viral the impact is clearly evident but standard measurements are
not employed. Retailers might have tried to compare the results, but wisely view it as a different way of
managing information, more along the lines of a publicity or public relations brand-play.
Retailers have seen customers connecting in search of customer service and some have found the ability
to improve service in a very timely fashion. One cataloger is dedicating several hours a day to Facebook,
extending their strong service across this newest channel for consistency purposes. He feels it is
essential for their brand to be involved from this perspective. Pinterest also was a tool of interest,
particularly for home retailers so monitoring this emerging vehicle is advised.
Source: e-tailing group 2012 Merchant Survey
Tracking for one merchant meant the creation of a dashboard for “sentiment,” mimicking what they had
received from their agency. Seeing some value, she advised that the ability to understand the importance
of influencers should not be minimized.
Interactivity seemed to be of greater interest for retailers to clarify how wide a net was being cast in the
social stratosphere. This moved beyond the number of fans to the levels of conversation and how much
they were promoted and shared. Interest in growing social as a communication tool means monitoring the
conversations and jumping in to bolster activity. One retailer suggested that they have stopped the “fan
count,” skipping the pure number and instead are focused on interactions from comments on posts to
subsequent views along with the overall time spent within social media. More sophisticated players are
attempting to track the cost to acquire a new fan and understand where and when they convert in order to
put necessary social resources in place. At the same time an awareness of how consumers embrace
promotions within the social environment and what resonates can help retailers shape future selling
strategies.
Along with these rudimentary insights, retailers have a goal of learning more about their customers. One
reported a much bigger international presence in Facebook which has impacted other shopping
strategies. They are also looking to ascertain who signs up for their emails from social locations and how
18
they subsequently engage with the company. Customer service feedback is an excellent opportunity to
share learning with operations that should foster changes in how one interacts with customers. Paying
attention to referral sources (blogs, Facebook, Twitter) and understanding from where traffic is derived
along with general onsite sharing tools (share, like) were part of the discussion as well.
Gaining a Cross-Channel View
The ultimate situation for retailers is to have a 360-degree view of the customer with a goal of
comprehending both interactions and transactions across all touch points. For retailers that are multi-
channel our Merchant Survey suggests that there is some measurement with anecdotal feedback topping
the list. Positively, lifetime value is being evaluated by more companies as it is a metric that has long
served the catalog industry well. One cataloger admitted that, like most, they are just starting to look at
lifetime value across different touch points. If they find the same level of sales across the board, they
hope to reduce their catalog spend without impacting business. Contact strategies are under considerable
scrutiny as retailers seek to understand the most cost effective ways to reach their customers and drive
conversion.
Source: e-tailing group 2012 Merchant Survey
Other cross-channel measurement starts with coupon redemption and promo code usage but only one-
third of participating companies were in a position to take advantage and learn from these numbers. The
call center was touched on during interviews as retailers look to follow and understand the evolving path
to purchase.
19
Source: e-tailing group 2012 Merchant Survey
The common-thread in discussions was measuring cross-channel and the subsequent challenges
surrounding superior execution. Several retailers relayed stories of looking at data nightly to see who
shops what store, using this as an indicator of the value of a multi-channel customer. Others monitor a
visitor’s behavior including search, email and sites though lacking sophisticated tools, they have yet to
piece together the meaning of these metrics. It will be imperative to set expectations where retailers look
at their businesses through a slightly different lens. Direct marketers have long track records for applying
data and felt anxious about assuming the accuracy of market contribution including understanding which
marketing channels performed best (search, email, affiliates).
There will be unique approaches inspired from revamped cross-channel thinking as more retailers see
stores as an entry point to direct marketing whereas in the past direct roads typically paved a way to the
web. Online is so much broader, and available to so many more people that retailers now want to
comprehend, "How do you provide great in-store experiences but also educate customers to buy online?”
For one large mass merchant, the most significant development over next two years will be using a single
login which can be carried from one enterprise to another. Ideally, it can be connected via the front and
backend, but worst case scenario a back end workaround will be put in place to seamlessly recognize the
customer.
Cross-Channel Behavior Warrants Cross-Channel Understanding
A holistic viewpoint requires out-of-the-box thinking, expanding and segmenting analytics beyond today’s
tool box. A look at cross-session conversion will be mandatory. Additionally a new approach to cross-
device usage must be implemented as people are using devices differently and the implications need to
be understood. It is also important to monitor the call center relative to the web should customers opt to
contact a merchant via phone.
The path-to-purchase will emerge as one of the most urgent areas warranting review in 2012. Merchants will want to know where shoppers initially click and the path they follow to their last stop. Ideally this will
20
make attribution possible by giving merchants more confidence in these numbers and the contribution they make to one’s business.
While some retailers will find metrics to be a springboard to greater success, for others it will be about
sheer survival. The right resources will be required as one pureplay suggested the need to transform from
an entrepreneurial company to an enterprise class institution as today they have no mid-level
management and senior management wears too many hats. Reorganization is in order; putting in place a
deep bench to foster growth. Investments in people and tools will be the differential. It starts with going
back-to-basics with the metrics in the hands of the business user. Testing should be core to those efforts
where learning fosters future growth.
From this learning, the knowledge to ask tough questions will emerge. Evaluating marketing across
channels and assigning attribution accordingly can inform segmentation strategies that work across the
whole company and help to set company-wide rules for tracking purposes. Data can then be shared
across channels; elements of each can influence where the sum is greater than the parts.
21
V: A MERCHANT’S METRIC PLAYBOOK
Each retailer has a unique approach for securing and sharing metrics within their organization. As I
sought to learn about such challenges and more importantly to glean ideas for improving the analysis
process, it is my belief that these tactics can readily be adapted within any organization. As one
technology manufacturer contributed, “Establishing shared goals across the organization versus a more
siloed mentality is a requirement for effective analysis.” To make sense of these insights and the
anecdotes shared by the retail community, a playbook for embracing metrics was constructed that is sure
to inspire any merchant to revisit their vision.
PLAY 1 - ALIGN GOALS
Success starts with alignment of goals where individuals are often incented in ways that drive personal
goals but fail to achieve more critical corporate objectives. All companies should reevaluate their
compensation structure and set attainable goals that benefit the organization as a whole. One merchant,
while appreciative of the corporate goals, acknowledged that factoring in retail store conversions driven
via the web would better reflect on the web channel.
Goal alignment is essential for all companies but the need becomes more apparent across global
organizations. According to an interviewed manufacturer, the complexity of coordination is but one
challenge considering that each geographic region has different goals and agendas (i.e. one may be in
retention mode while another is ramping up customer acquisition). Additionally, incentive structures may
be such that conflicting goals make it difficult to align under a global umbrella so balance is welcome.
PLAY 2 - RUN COMPANY LIKE IT’S SMALL
Being nimble and thinking small can be beneficial for any size retailer. As one large company proffered,
“We are very transparent and run our company like it is small, passing metrics to senior executives and
direct reports via email including KPIs for Internet and multi-channel performance. From there we can dig
deeper on why any given category is performing up or down.” Heeding this advice, even on a small scale,
should deliver results for all.
PLAY 3 - MEET REGUARLY: THE MONDAY MEETING AND MORE
Organizations must put themselves in a strong position to succeed when it comes to metrics and
analysis. While analysis paralysis was paramount for some interviewed, many more merchants appear to
be tackling metrics head on starting with “Monday meetings.” Sharing data should be addressed on a
broad scale relative to the sophistication of one’s tool. It can be used for general communication or be
more self-serving to inform coworkers of trends, suggest a site improvement or advise of a marketing
change. Another multi-brand business reported that their eight brands and the parent company have
individual needs so within each business unit they have their own processes. Despite their uniqueness,
every Monday morning an internal report is produced and distributed to executives with last week’s
analysis including a corporate rollup to the parent side as well as dissemination via a variety of forums.
I recall my early days in retailing when morning flash reports were core to our culture. It seems that
merchants have adopted a similar course relative to ecommerce and cross-channel metrics where the
“Monday meeting” still has meaning within many organizations. While there are certainly variances in
style it is interesting to observe the inner workings behind the array of approaches shared:
“…15 page recap created monthly and then followed up with quarterly and yearly data”
“…put together 1-2 pages (ecommerce, email, social media) on Monday followed by distribution
to all concerned parties and discussion with a few key players”
22
“…get standardized operations and e/c reports from Omniture and legacy system every Monday
followed by a weekly business and priority review meeting then revisit monthly and quarterly”
“…series of weekly dashboards for each business unit pulled by our web analytics teams;
distribution to a broad internal audience + key traffic partners followed by a weekly manager
review meeting to digest and reset business”
“…weekly dashboard + meeting; review KPIs; dive into analytics where two individuals address
questions; bimonthly business metrics that are channel-specific”
PLAY 4 - STANDARDIZE DEFINITIONS PRIOR TO ANALYSIS
Aligning definitions for all metrics was a topic that surfaced often with methodologies for comparing
performance across channels challenging for some. Unfortunately, internally in many companies people
look at data in different ways. There is no consistent measurement where reports/dashboards are
mandatory for benchmarking relevance. It was highly recommended that visibility to metrics be
differentiated between operating and senior level executives and standardized deliverables, with agreed
upon definitions presented earlier rather than later. One merchant was adamant that an educational
process was most valuable when ecommerce upgrades are in order and KPIs are attached to a goal
focusing companies around these pre-established goals. Ensuring that methodology is in place to
compare one business unit to all others avoids complications down-the-road. All of these efforts rely on a
clear and early understanding to avoid surprises or divergent viewpoints.
On a side note, the biggest frustration regarding metrics for one retailer was Google changing what the
metrics mean, such as the way visitors are calculated, particularly as the investment bankers that control
their purse strings have little patience for poor year-over-year comparisons and an inability to defend the
numbers.
PLAY 5 - FOCUS ON THE ACTION NEEDED BY MOVING BEYOND THE NUMBERS
It was remarked upon routinely that streamlining the level of information being shared with teams and
senior management is essential as most retailers simply have too much information to process on a
weekly basis. As one multi-channel cataloger stated, “We must go beyond the numbers to know what
they mean and more importantly to determine what to do next.” Another merchant offered a salient piece
of advice suggesting, “Retailers should not getting paralyzed by data but instead look for important items
you can actualize.”
Leveraging Data for Action
A forward thinking retailer praised their ability to capture and store data using this old ecommerce
analogy, “We now have the highway paved but the evolution must come from leveraging the data.”
Determining how best to capitalize on all the available data while ensuring the right people are in place
appears to be the winning combination. Today this job often belongs to a web specialist as the nuances
are complex and the learning curve steep. One equated the analyst’s job to “finding a needle in a
haystack especially when it comes to attribution.” Excel was seen as an analyst’s best friend and strong
skills must prevail despite access to a business intelligence team. Per another merchant, “We found that
it can take up to 6-8 weeks of development to scope out tables.” Slicing and dicing is par for the course
but all roads still lead to excel tables, particularly in this company’s numbers-driven culture.
Internal Transparency: Assess Level of Visibility into Metrics across the Organization
Choices must be made with caution when determining the level of visibility into metrics for executives and
divisions. Ironically, one retailer who supplied a Google login, saw limited interest among her team. Being
transparent with available metrics means everyone can interpret and influence based off a level playing
field. Several retailers confirmed that they have a long way to go to reach complete transparency
internally though they believe they are headed in the right direction.
23
PLAY 6 - EDUCATE INTERNALLY
Retailers face a balancing act educating and encouraging management to look beyond the top-line as
running the business may mean justifying additional resources. “While they are asking good questions
today and we want to respond accordingly, we can always go deeper but have to understand there is a
cost in gathering this information.” Several cited the need to add more resources as there are many
areas of the business not being examined, making it easy to forget about important elements. They would
feel much better ensuring all touch-points are under review. Frequently there is pushback so marketers
and merchandisers must continue to reinforce why it is crucial for them to be running the business.
One retailer acknowledged that they have lots of teams that can access data with analysts in finance,
marketing and ecommerce. Like many things in life, knowing who to ask to secure information and get
questions answered is critical. One can find themselves “in the weeds” when initially seeking an answer
to a very simple question and time then quickly spirals out of control. Sometimes, in large organizations,
this also means tying together analysis from different parts of the company, adding further complexity to
the task-at-hand.
PLAY 7 - PROVIDE HOLISTIC VIEW OF INFORMATION
Ultimately the goal must be providing a holistic view of the customer with marketers and merchants
running the business. “Data despite its availability doesn’t always work well together,” lamented one
retailer where a lot of work needs to be done to flow this information together. While individuals may
have strong experience within the organization, rolling data to meet the needs of senior management in a
more holistic fashion often poses problems. Many times business units have their own agendas; goals
can be conflicting and not all the necessary data is put forth so there is a cause and effect with the
choices to be made. One referenced their lean staff making the requisite time to get stuff done a
challenge as their dashboards (internet, whole business) are time consuming to produce but worth the
effort when everything comes together.
PLAY 8 - INVEST IN DATA WAREHOUSES
Several retailers described in detail investments in data warehouses while others have them on their
roadmaps moving forward. The hope is to develop a system that houses all numbers where data totals
roll to more realistic numbers. Reflections included:
“We now have a data warehouse that automates our analysis where previously everything was a
manual pull. This includes 60-70 automated queries via email on a scheduled basis. We are
hoping to create dashboards to better use these new capabilities.”
“With new back-end systems and having replatformed in 2011, metrics have been pulled from
many places and when aggregated never add up to 100% (30% higher!); Our focus is to balance
this out by building a merchandising reporting data warehouse from which transactional details
can be filtered and dashboards created.”
“We are trying to make sense of data that is being derived from multiple sources (Google,
transaction, email) and that can be a challenge.”
PLAY 9 - DIALOGUE AND DISCUSS
Fostering an internal dialogue may be one of the most important moves one can make relative to metrics.
The discussion will play out based on the expertise and attitude of participants. When novices in
ecommerce and cross-channel metrics encounter a dashboard for the first time, dialogue will likely center
around the most basic metrics and their accompanying definitions. Once these are standardized,
encouraging all parties to ask questions to gather greater insights and fuel future planning will surely
24
evolve. An explanation of why a metric is important may be even more significant than the number itself.
When fluctuations are observed team members may even request a little more background on what
happened and why.
One seasoned merchant reflected that the direction of the discussion will depend on who is present.
Some people will project/interpret the results from a defensive position (i.e. my email is great), rather than
seizing the moment to evaluate why performance levels didn’t achieve desired goals. The ability to learn
from one’s mistakes and go further through exploitation of the information; gathering a true look into the
why behind the numbers will best benefit future planning.
The e-tailing group Metrics Checklist
Determine what you want to measure and why with a lens on profitability
Assess if you have the right resources to analyze the findings
Build a playbook that aligns goals and focuses on taking action
Foster a culture that shares metrics and dialogues around findings in order to adapt
Understand the changing role of conversion and the influencers that direct its performance
Follow emerging metrics and evolve dashboards as traffic and revenue escalates
Invest in technology to garner a 360-degree view of the customer
25
VI: ABOUT THE COMPANIES
About the e-tailing group
The e-tailing group is a niche e-commerce consultancy that helps merchants deliver the right customer
experience on their websites and across all of their channels while adeptly assisting technology
companies to create and execute go-to-market strategies that simultaneously educate the retail
community and deliver cost-effective thought leadership and lead generation. For more background about
our research or for additional information on the e-tailing group, inc. please contact Lauren Freedman via
email at [email protected], by phone to 773-975-7280 or visit the e-tailing group website
www.e-tailing.com.
About Baynote
Baynote is a leading provider of personalized customer experience solutions for multi-channel retailers.
Using Baynote’s patented approach, retailers are able to personalize the consumer shopping experience
“in the moment”, displaying compelling offers, content and products that increase engagement,
conversion and average order value. The Baynote Customer Experience Layer rapidly integrates with
existing websites, onsite search, chat or email systems to increase ROI without deep IT involvement or
expensive system upgrades. Based in San Jose, Calif., with offices in the U.K. and Germany, Baynote’s
personalization solutions are trusted by more than 300 of the world’s most well-known brands, including,
Anthropologie, Bluefly, BT, Campbell’s, Dell, Expedia, Intuit, JCrew, Jockey, and Urban Outfitters. For
more information about Baynote, visit http://www.baynote.com.
