Redfin saves 2-4 weeks of work per month with Heap

Redfin’s mission is to redefine real estate in the customer’s favor—and part of fulfilling that mission includes building software for Redfin agents to help them deliver a great customer experience. Heap has become a significant tool for Redfin to measure and improve performance at every step of the buying and selling journey, so that clients and agents alike can enjoy seamless interactions and technology that’s intuitive and easy to use.

We asked Nick Smith, Senior Product Manager at Redfin, what he likes best about Heap.

INTERVIEW:

What’s your job like and how does Heap fit into it?

I focus on building internal tools for the Redfin agents in the field. Our agents close more deals per year than the typical real estate agent, because we’ve taken our technology beyond initial search to make the entire process—the home tour, the listing debut, escrow—faster, easier and worry-free. I was one of several product managers who pitched Heap’s benefits to our execs. The goals were saving engineering time, putting product and design in the driver’s seat, making decisions faster, and not having to hire additional analysts to help us do the necessary homework before redesigning a feature and after rolling a feature out. Now Heap is helping us to make better, faster product decisions through the whole end-to-end relationship.

What was life like before Heap?

Basically, we had limited data around how different agents or support staff used our old tools. We could define events in our system, but it cost engineering time and required perfect foresight. And then you usually needed an analyst’s help, because doing anything other than just impressions or clicks required gnarly SQL skills which not all product managers have. And there’s also the age-old problem of as soon as you pull all that SQL data, it’s hard to share, and out of date as soon as it’s pulled.

How does Heap improve your workflows?

We have a homegrown CRM solution built through our agents’ own hard work. It’s pretty robust, but like any 10 year old platform, it has a large surface area that gets difficult to instrument and manage. We’re always refining it and making the tools easier to use. We use Heap to see what’s being adopted, and whether users like new features. When we redesign pages, rather than asking agents how they use it, we can test hypotheses based on their behavior and move ahead faster. For example, since agents are always on the go, we prioritize quick, time-sensitive tasks and design them for mobile first. When we see an agent doing something on desktop that they should be able to do quickly on mobile, we know there’s an opportunity to streamline the experience. Heap helps us understand it, or to ask the questions that help diagnose the problem.Also before Heap, we just could not do any type of funnel analysis or segmentation. We have five, six, seven different roles using our tools daily. Some things are designed for agents, some are designed for managers. And we couldn’t really split that out or know who was who.

So you do lots of segmentation then?

Yes. We like to see how new agents do things, compared to Redfin veterans. And we’ll create segments for alphas and betas and release it to a subsection of a market. For example, if something is successful for 10 people in Atlanta, then we’ll do a full beta in Atlanta, and if things look good, extend it to everyone across the US and in Canada. Segmentation lets us see exactly how the alpha people are using the new thing, compared to how everyone else uses the old thing. And it helps us be really methodical about identifying problems early, before we just roll it out to everyone hoping that it’s going to work. Heap let us get a lot more in tune with setting good quantitative and adoption-type and engagement-type benchmarks for new features.

Any rollouts where measuring had a critical impact?

We created a new page range for deals in flight which featured an overdue task indicator. People were clicking on it…but they weren’t actually completing all the tasks assigned to them. Part of the reason was they were clicking through to a different page. Heap helped us realize we should scroll the tasks into view, versus taking agents to their task list. And that one change TRIPLED the completion rate for that flow, versus agents getting partway through and bailing before they completed the task.

Any documentable wins you can share?

Having accurate drive time estimates between homes is essential. In towns like Phoenix, it’s such a huge area, you need to make sure you’ve accounted for this. In our old algorithm, coordinators were doing drive time adjustments manually, which took a lot of time. And so we updated that algorithm to need fewer coordinators to book the same volume of tours. Using Heap, we were able to tell people weren’t manually calculating drive time as often—the algorithm update actually saved time! Just based on that, we were able to adjust future headcount plans.A different example: we looked at features we assumed agents liked and found out “Oh, no, only 5% of people use that.” So that saved us design resources, testing, and building new components to our React library. It’s less exciting than creating a new feature, but sunsetting features no one uses has a huge impact and reduces product bloat too.

So it sounds like Heap really helps you succeed with your goals.

We even set rollout goals for Heap! For our first year, we wanted to reduce time to insight—how long it takes product managers to answer any questions they have. We wanted to make sure that we could trust Heap as a source of truth. One fear was that people would just go in, define events all over the place, and make an unwieldy mess. Data governance was really critical. The benchmark we used was that if we hire a new PM, they should be able to go and build their own reports and immediately make sense of how we’ve named things and where things live. It was a success.Heap has really put the product folk in the driver’s seat, and also gives the benefit of hindsight. You forget to instrument something, doesn’t really matter. You can come back and do it later. That’s a powerful plus.The goal we actually struggled with was consistent adoption by product managers in Redfin. Our goal was 70% of folks using it on a monthly basis, and we saw it vary between 44 and 69%. That’s partly awareness where people didn’t know what questions they could ask, partly bandwidth, and partly developing new habits.

How much engineering time would you say you’ve saved?

Heap probably saves us 2-4 days for every single feature we roll out, and we’re shipping between five and ten features a month. And that’s probably just in people hours. According to our Eng team, “Without Heap, every item we want to track must be manually coded up. Though this isn’t technically complex, it does take time, and is incomplete, since we pare down the data to what’s worth the cost to add. As a result, we make worse decisions on feature usage because of missing data, and the cost of pulling numbers. Heap gathers every piece of data automatically, and at no extra cost.”I was working with our Analytics team to build a usage dashboard for a page we were designing. They estimated it would take two to three weeks of work. During our Heap trial, I instrumented those inquiries myself, and it took me two hours.

What big things are you building that Heap will probably help with?

We’re focusing on improving our mobile app for agents. To have the ability to go in and instrument everything will let us help our agents to be a lot more efficient and responsive to customers while they’re on the go.

If a friend asked: “Should I use Heap?” what would you say, and why?

You get peace of mind and power at the same time. It automatically captures everything, then you can go in and really explore how people are using the product, rather than just look for a very specific thing. That’s just really powerful to have at your fingertips. Makes your job easier, frees up engineering and analysts’ time, and helps you be a better product manager.