Feedback loops are an essential component in product development because these loops provide regular opportunities to learn and adjust the product to have better fit. Qualitative feedback loops enable you to better understand your customers. Prototyping is another powerful feedback loop in product development. As effective as these are, both qualitative feedback and prototyping are poorly suited to give feedback at scale—they are best done in one on one scenarios or small groups and often times in-person.
However, one of the benefits to building products in the age of the internet is that we can do many things at scale that were previously not possible. This includes automated, remote, and statistically significant product feedback loops. This type of product feedback loop depends on having well-implemented data gathering and analytics.
What are analytics?
Product analytics are the computational analyses of data about a product. This can be passive data that is not explicitly under your control, like number of downloads or support requests. But for the duration of this article we will be looking at data that is generated from your product that you have explicitly decided to capture.
Data Events
Data events are the basic building block for most product analytics. A data event is any meaningful action or event that you would like to track or factor into some kind of analysis. Events are composed of a name, like created-account, and a date and time of occurrence. Most event tracking tools will also let you add custom attributes to events to capture other relevant state of the product when the event occurred.
Choosing what data events to capture takes a little bit of foresight (and experience) to know what you might want to track. For this reason, it is often best to start with a small number of events and iteratively add more as you learn about what you want to know. Web analytics tools usually automatically track every URL loaded as a page view event; event tracking for native apps will most likely not have any built-in events.
Segmentation, funnels, and cohorts are three important analytics concepts. You will probably get a better idea of the events you should capture in your product as we discuss these concepts.
Segmentation
Splitting event data into groups based on some particular characteristic is known as segmentation. There are two main types of segmentation: event and customer.
Event segmentation lets you understand different aspects of a particular event. For example, if you have a “save note” event with a custom attribute of “note length” that counts the number of words, you could break down the event into buckets of length to understand if your customers are typing long or short notes. You can leverage this type of analysis to inform future UI improvements or feature decisions.
Customer segmentation lets you look at how different groups of customers use your product. For example, you could look at how customers from different regions trigger an event or use a feature (which could be an aggregation of multiple events).
Funnels
Some action paths through a product are important to track and optimize for the highest completion rate possible, like signing up for an account or making a purchase. You can simply track the occurrence of the final action with a single data event, but you need to be able to connect multiple events together to analyze the entire path. This kind of analysis is referred to as funnel analysis.
Funnels are useful to quickly see where customers are dropping off in a path. For example, are they having issues on the checkout page? You can also use funnels across product releases to ensure a change has not decreased conversions on critical product paths.
Cohorts
Cohorts are groups of customers that have a common characteristic within a specified time period, usually day, week, or month. Preceding concepts, like segmentation or simple data events, can be used to define cohorts, but a common cohort characteristic is the date a person started using a product.
Cohort analysis is helpful in tracking two important product metrics at a high-level: engagement and retention. Pairing a useful cohort characteristic, like start date, with one of your key performance indicator metrics is a powerful way to make sure each new cohort group is staying engaged. Similarly, cohorts provide a quick way to see how long customers are sticking around with your product. Here is one way of visualizing a cohort analysis:
It’s important to note that cohort analysis does not show why engagement or retention is worsening or improving, but it will give a heads up that something warrants further investigation.
What‘s next?
Now that you understand the basics of analytics, you should choose one of the tools out there—like Google Analytics, Mixpanel, or Localytics—and implement it in your product. You should also begin to regularly ask yourself, “What do I want to know? What data do I need to capture to know this? What analysis will I do with the data to give me an answer?”
Have you implemented analytics in your own products? Leave a comment adding to what we’ve discussed here.