Guest post by Anubhav, our fab PMM intern
Most early stage startups fall victim to featuritis; where shipping new features seems like the solution to all problems. But the more sustainable approach is to analyse current usage, find the blocks and funnel drops; and improve from there. During my internship at Kaapi; I set up a repeatable checklist of analysis to do every month, with some great insights. And here are the results. Hope you find it useful!
The first step was to make a list of business questions. We didn’t want this to be an exercise just for the sake of it. Without this step, you can twist any dataset to satisfy the narrative you want:
- Does response rate decrease over time? Ex. - Overall in the system, do responses decrease every week? So if team1 signed up on 1st July, then what was their response rate on 1st week? 2nd week? 3rd week? 4th week?
- What makes a team pay? Is the paid team engaging more with the features or have a specific scheduler configuration
- What kind of teams are using Kaapi?
- Is the response rate of a particular question type higher than others? e.g. private vs public? morale vs alignment category? Paragraph vs radio?
Once we decided what we wanted to look for, the collection and analysis became easier and more organised. This is what we learned about our users and how we incorporated the learnings in our product and messaging. This analysis was done on ~ 700 users and ~ 4000 questions so it is statistically relevant.
Our earliest users are teams that were forced into remote transition because of Covid-19
Our assumption was that remote and distributed teams would find the most benefit from our features. But when we looked at the geo distribution, we found out that most teams were in the same time zones. They were in the same workspace when they started using Kaapi before Covid and have continued using it after going remote. It is a strong indicator that teams want less meetings - be it in person or over Zoom.
Response rate to questions go down over time
There was a general trend of response rate going down but with sudden spikes in the middle and then a downward trend again at the end. This was the case for the paid teams as well and definitely a huge concern. We were not sure if that’s because employees get bored of the same questions again & again or if they are not finding enough value out of the exercise.
This is where we combined our research from the user feedback interviews and launched ice-breaker questions. We are already seeing a positive response on this and would be keeping a close eye on how teams engage with it.
We need to rethink our ideal customer persona
When we were analysing paying teams over non-paying, we evaluated metrics such as:
- Do they get a higher response rate over non-paying teams?
- Are they using daily/weekly standups?
- Do they use paragraph questions instead of radio questions?
- How many employees do they have?
- Who is the manager?
There was no differentiation in most of the metrics except one. 71% of the managers on the paying teams were founders and CXOs. This was a great validation for us but also an indicator of whom we should reach out to for converting trial users to paying users. We are already working on a new set of landing pages focused on the CXO persona.
Our FTU (first time user) onboarding is fab. Let's double down on it
The median time taken to activate Kaapi for a new team is 2 minute 58 seconds! This was very heartening to see. Because usual employee engagement tools and pulse survey tools have a long process of importing excel sheets, creating employee accounts etc before you can use them. We hope to recreate this magic in our web app too!\
Employees don't mind responding to paragraph questions
After looking at various configurations used by team managers in the questions they were asking their team, we didn’t find any particular discrepancy or any key insight. The response rates for all the permutations of frequency and type of questions were in a similar range. We always used to think that simple radio type questions will get more response rate, but that was not true at all. Managers also love such responses, because these are deep insights vs Yes/No stats
We are glad that we took time out to do this. It helped us with feature prioritisation, customer segmentation and validation of our product. We are already at work changing the definition of our marketing personas, and also landing pages based on this data. And for a bootstrapped company on a mission to improve your team productivity, it motivates us to continue hustling and hustling hard.
.png){kind=small}
