Stories from the PM trenches: How an experimental “delight” feature ended up increasing adoption

The Challenge: A Delightful Feature

Every year, Heap hosts a Ship Week. This is our version of a hackathon, where we build things and ship them to our users in the span of a week. We love Ship Weeks. (You can read more about why in our blog posts on the topic.) They’re often the source of some of our best ideas! 

Each Ship Week has a theme, and this year it was “User Delight.”

I’ve worked on lots of “Delight” features in my career. Previously, I worked on ringtones at PagerDuty, a user favorite to help make going on-call just a little more fun. At Asana, I worked on the famous unicorn and other celebration characters that would zoom across the screen when you completed tasks. 

These “Delight” projects all started as fun features the team built for themselves or as part of hackathons. However, I’ve found these features are major drivers of user adoption and engagement, so you’ll find me championing delight whenever I can.

The Winning Entry: Data Quiz

The most delightful thing our team came up with this year was Heap’s Data Quiz, which was created by Alex, Engineer on our growth team, and Bryan, Designer on our growth team.

What is it? Well, the Data Quiz uses Heap’s autocaptured data to ask users automatically generated questions about their own dataset! How many users visited your site last week? Which segment of users is the largest? And which event happened most frequently yesterday?

The goal is to find a fun way to see how well Heap users know their own product. On a deeper level, it encourages users to dig deeper into the insights Heap surfaces. And, of course, it adds some healthy competition.  

Data Quiz Screenshot

Internal Competition

Before launching externally, we tested it internally. The team at Heap is a pretty competitive bunch, so the internal launch was … exciting. And it led to some changes! For example, Vijay, who leads our Insights team, absolutely crushed everyone else in the first few days, simply by answering a lot of quiz questions.

That couldn’t stand, of course. So we evolved the quiz to set a daily limit on the number of questions. Now to hit the leaderboard, you have to actually know your data. (Vijay still holds the top spot. Respect.)

Because the internal launch was such a success, we polished it up and shipped it to everyone. And it turned out our customers loved it, too! 

The Results

 Right away, we saw tons of users answering quiz questions. With lots of users answering more than one! We’ve also seen it have a big impact on query rates, because it automatically generates queries for users to explore. 

Here’s a usage report:

Screenshot - Data Quiz Usage Report Query

Screenshot - Data Quiz Usage Graph

Why this matters: we also measure Trial Adoption – whether a trial account is using Heap successfully – by seeing whether an account has run six queries in the first 14 days or not. (It’s our version of Facebook’s “7 friends in 10 days.”) It's too early to tell if we've impacted overall trial adoption, but seeing users running more queries is really promising.

Turns out that the quiz is fun, but it’s also a really easy way to get started understanding the data and running queries. Using path comparison to see users who answer quiz questions versus ones who do not, we see the ones that do are more likely querying in the same session.

Data Quiz Path Comparison Chart

 I personally have been enjoying the Data Quiz questions. I’ve learned about our user base and gotten competitive with my teammates. Delightful features can sometimes be hard to prioritize, but they can often be some of the most beloved and impactful features. Plus, they’re a lot of fun.

Special thanks and congrats to the team – Alex Frieder, Bryan Zavestoski and Connie Yuan – who worked on the Data Quiz!