How Splunk Uses Product Analytics To Win
Who Is VictorOps / Splunk?
VictorOps is an incident management solution for IT teams within Splunk’s portfolio that automates getting the right alert to the right person at the right time to reduce alert fatigue, and increase uptime. VictorOps allows customers to provide faster incident response times, empower DevOps teams, and improve the on-call experience.
Splunk, its parent company, helps turn machine data into answers, providing unique insights to unlock innovation, and solve tough business problems. Currently, Splunk has over 4,700 employees, and dozens of offices across the globe.
Improved Product Management
Understanding which products and features to prioritize is crucial, given the many thousands of directions the product team could pursue.
David Godbout, a senior product manager on the VictorOps team at Splunk, turns to Heap when looking for answers.
"So, Heap is the first place I go to start answering questions. It starts giving me a baseline of understanding what our customers are doing, current product usage, and a baseline of exploration when we move forward."
Having a thorough understanding of their customer base helps ensure that the products and features they do invest in will be a good use of their time.
Given their engineers are often on tight deadlines, Davis likes the fact that with Heap they can quickly implement custom tagging without needing to take their engineers away from their work, "You don't have to go tap your engineer when they're building a feature… you can add a tag to this feature so I can analyze after the fact."
PMs Always Have An Answer With Heap
“It really takes a lot of pressure off of me. It means I don’t have to have everything perfect and there might be questions after the fact that I’ll still be able to answer. It’s all in Heap – it’s got my back”
PMs have a difficult job – they often take a lot of heat, and they’re expected to always have an answer. For many PMs, these are sometimes difficult questions to answer in meetings and saying ‘I don’t know’ has become a common phrase among PMs. Even if there’s a general hypothesis around product investments, PMs are still expected to know the baseline usage and what impact the work had on that baseline. This is true whether or not they’ve done the engineering work beforehand to write tracking code in the product.
The entire VictorOps team can turn to Heap to retroactively to see past data they otherwise may have missed. Instead of worrying about previous insights being lost or deleted, Davis knows that with Autocapture, product data will be there when needed.
"Retroactivity allows me to always have an answer. Sometimes I'll get asked a question that I might not have had a perfect instrumentation to capture the impact that I was looking for, and I'm able to answer that question regardless of whether or not I've done the work beforehand."
Product managers at Splunk no longer have to manage situations where they may be caught flat-footed. The can move quickly and ship improvements to the product without ever having to say ‘I don’t know’.
“If I had to try to guess all the different questions I’ll have in the future, I’d have my engineering team build a tag for every single event they possibly could. This detracts resources from building our product.”
The Scientific Method Of Product Development
The product team at Splunk subscribes to a philosophy known as The Scientific Method of Product Development. The general philosophy is to build product in a hypothesis-driven approach, and then backing any results (positive or negative) with data.
VictorOps’s product team is ruthless in their quest to understand how customers are using the product, which features are most valuable, how to improve conversions, and segmenting cohorts based on retention. They take the product analytics baselines from Heap, and then build hypotheses of why certain metrics are the way the are.
From those hypotheses, PMs are able to ideate everything from new features, complete overhauls of experiences, or small improvements to make their customers more successful. At this stage, it’s all about prioritization and investing engineering resources in the most leveraged way. PMs at Splunk believe that’s not possible without a baseline – a clear understanding of how customers are using VictorOps today.
After new things have shipped, the product team is able to use Heap to understand the impact their efforts are having – either positive or negative. This hypothesis and results-driven approach is what enables VictorOps to iterate faster than their competitors.
Delivering An Exceptional Customer Experience
A significant portion of delivering a great customer experience is the result of truly understanding customer needs. Splunk invests heavily in product analytics to understand which features are actually used and how they can continually improve their current feature set.
Given VictorOps works with hundreds of businesses in virtually every industry imaginable, the ability to deliver a customized user experience to their customers is a must. VictorOp’s customer success team uses Heap’s product analytics to see account usage and feature engagement.
Drew Abernethy, Customer Success Manager at Splunk, uses Heap to help better understand his customer's wants, needs, and desires.
Using Heap, Drew and his team are able to determine with specificity whether or not their decisions are making a direct impact for their customers.
"From a customer success point of view, that insight into behaviors and sort of where people are giving their attention and their time in your product, is super valuable because you can make different hypothesis and of things you think you know, but then you bring in Heap, and you can actually get to the bottom of, "Oh, this is actually how it's playing out."
Extensive product usage data provided by Heap allows the VictorOps team to see patterns and trends from their customer's behavior that their users may not necessarily mention in casual conversation. Additionally, Heap allows the team to be agile and deliver product and feature requests faster than before.
The customer success team at VictorOps has found Heap’s product analytics powerful and approachable. Not only does Heap provide the product usage insights needed to improve the overall customer experience, the customer success team loves its simplicity, "The biggest thing for me is just the ease of use is just amazing," says Drew.
Building A Best-in-class Product
Splunk leverages Heap’s product analytics to optimize everything from their product investments and prioritization to delivering an experience that customers get value from.
Given the complexity of building software and tools that thousands of businesses around the world rely on, Splunk continues to innovate not only in their technology solutions but how they approach creating value for their customers and clients.
With the ability to ship products and features faster, avoid making strategic decisions based on educated guesses, and ensuring customer success through product analytics, VictorOps continues to grow at a rapid pace while setting the standard for IT incident response management.