Data-driven decision-making

My first startup job (at HOTorNOT) was essentially a "startup school." Though the site seemed frivolous on the surface, the team behind it was small, dedicated and fiercely data-driven–using homegrown tools—before being data-driven became a "lean" mantra.

These days, anyone can collect data simply by dropping in tracking code from Mixpanel, Heap, or even, a meta-analytics service. But all this data collection is pointless unless you can use it to make decisions.

Collect correctly

The first requirement for data-driven decision-making is good data. Collect it all and make sure it's correct, that you're measuring what you intend to measure. It's easy to make mistakes here—whether it's a database query that's slightly off, or missing a segment in Google Analytics.

If you and your colleagues don't have faith in your data, then you won't be able to make decisions with it.

Ask good questions

After you've got good data, you should ask good questions. The more specific your question, the better—not "how many users have signed up", but "Which channels have high-value users come from in the last 6 months?" 

This question is slightly more complex than it sounds—you have to define what "high-value" means (and different people in your company might have different opinions)—but it's certainly answerable, and will give you more actionable analysis than just asking about "all users."

Analyze and Act

Next, analyze your data to answer your question. There are plenty of tools available for this—segmentation, funnel analysis, cohort analysis, plain old number-crunching in Excel. I won't go into detail here. But you should be able to answer the question you've asked if you've collected correct data.

The last step can be the hardest—acting on your analysis. If you're like most product designers, you'll often find data that disprove strongly-held opinions about your product. You may have to convince your team that your analysis is correct.

There will always be situations when you don't have enough time to rigorously collect and analyze data, or when you're creating something entirely new--these are times to use your intuition and make predictions based on previous experiments. But if you have the time, making an effort to drive your decision-making based on data will pay off.