Title text

Arthur EtchellsDirector of Product Management

Imaging Resource

Our product looked like this

We added a cool product tour shortly after launch

‘Engagement’ was through the roof!

People loved it!

“Lorem ipsum dolor sit amet, consectetur adipiscing elit, set eiusmod tempor incidunt et labore et dolore magna aliquam.”

– First Name Last Name, Attribution

Or did they?Was ‘engagement’ what we wanted?

• Yes, engagement was great (Optimizely)

• Bounce rates were not (Google Analytics)

• Months pass

• Some internal feedback (that tour is annoying!) …

• Led to further research…

Title Text

• Crazy Egg

• In-page click analysis

Showed us something interesting…

• Bullets only

• Up to five items

• One to four words

• First letter is capitalized

• Periods are not used

Not the engagement

we were looking for

The Question

*non-product tour content

Do users interact more with our content* after viewing a product tour?

The Experiment

Two Variations

Two Metrics

Engagement

Clicks on non-product tour content

Results

Title Text

• Engagement

• Dropped by 20.4%

Title Text

• Clicks on non-product tour links

• Increased by 36.3%

Lessons Learned

Be careful what you wish optimize forBe careful what you wish for

Engagement may not be the best metric to optimize

Make sure product tours are actually doing what you hope they are

There should be a high bar for anything you put between users and your content

Take a holistic approach to optimization effortsSubjective feedback can let you uncover issues earlier

User feedback sessions

In-page feedback prompt

HeatmapsAnalyticsBiz goals: revenue, lead gen

Objective Subjective

Problem identified here

User feedback sessions

In-page feedback prompt

HeatmapsAnalyticsBiz goals: revenue, lead gen

Objective Subjective

Hypothesis generated here

User feedback sessions

In-page feedback prompt

HeatmapsAnalyticsBiz goals: revenue, lead gen

Objective Subjective

Could have identified

the

problem much earlier

here

• Those were the tools we used, but there are lots of options…

• Analytics (Omniture/Google Analytics/Mixpanel)

• Heatmaps (ClickTale/CrazyEgg/HotJar)

• In-page feedback forms (Qualaroo/HotJar)

• User feedback sessions (In-person/UserTesting)

Subjective feedback can allow you to uncover usability problems earlier

What does that look like in practice?

UserTesting/HotJar/Optimizely Discussion…

“Lorem ipsum dolor sit amet, consectetur adipiscing elit, set eiusmod tempor incidunt et labore et dolore magna aliquam.”

– First Name Last Name, Attribution

Questions?