Per product NPS survey #143
Conversation
|
@mariusandra @benjackwhite @neilkakkar FYI, since your teams are responsible for these products |
There was a problem hiding this comment.
Having discussed with Annika today, following a marketing strategy meeting last week, I'd like to suggest that we change the way we're running this survey.
Tweak the targeting
The survey has currently only had 50 views in 20+ days, so it's clearly not being shown to enough people. We need at least 100 respondents to confidently start tracking NPS.
Part of the issue is that the survey is incorrectly configured and is not targeting users who signed up 14 days ago: it's targeting users who joined before 2023-11-01.
I'd suggest we:
- remove the 'last 14 days' targeting, as we can't easily target this correctly
- lower the limit to users who have seen another survey to exclude users who've seen a survey in the last 14 days
- increase the rollout from the current 20% to 50% of users
Remove the free sticker
Offering an incentive to users only works if we tell them in advance that it exists. I don't think adding a step to do this is beneficial vs the greater simplicity, so we should just remove this step entirely.
Add a 'role_at_organization' question
Discussed in the recent marketing session was the idea that one of the key challenges for us is that we're a devtool which needs to balance the needs of non-technical users.
One of the most valuable insights we can therefore have is how roles vary in usage and response across products. Is Surveys, for example, used mainly by non-technical marketing users? What do they think about it? This information will help us support users or correct direction.
We should add a question which aligns with the role_at_organization property, asking users if they are engineering, founder, product, sales, marketing, or other.
Rollout to product analytics
Part of the issue may be that feature flags are not one of our most popular products. We should improve this test by rolling it out to product analytics, and potentially to the other products as well.
shouldn't we know this already based on the info they provide at signup? (I know not everyone answers this question, but we can maybe draw conclusions from those who do)
I really think we don't want to ask people for NPS scores if they've just joined. Is there some way we can still do this delay? |
If someone knows a way to correlate this, sure. My main interest is: What roles are using which products, and how do we perform for them? I don't mind how we get that info.
I agree 100%, I just couldn't see a way to implement this delay. At the moment the survey is only showing to users who signed up before 2023-11-01, which also isn't ideal because we're not getting current info. |
Yes, because we can't do < 14 days properly, I need to regularly update this date manually every couple of weeks. Which is not ideal, but I don't think there is a better solution right now. This is European format, so we are only slightly out of date. I will set myself a calendar reminder so that I won't forget.
I just had a look how we could get this. If you want to see this on the survey results page, here is how: https://posthog.slack.com/archives/C0351B1DMUY/p1702399749511089 |
Done. I've rolled this out according to your proposal, and also updated the signup date. I will have a look to see if this makes a dent in the response rate.
Done. I replaced it with a auto-disappearing "Thank you for your feedback!" message.
Going to do this this week. I will also set this up for session replay. |
I'd like to set up a permanent NPS survey per product, so that we can track NPS over time. Please let me know if you have any feedback by the end of the week.