A well designed and orchestrated beta program can provide invaluable insights into how well your new feature or product solves user problems. Betas don’t happen on their own, of course, and pitfalls abound. In bad cases, betas are a cop out for buggy code; in the worst cases, you end up launching the wrong product for the market.
Here at Wizeline, we’ve developed some rules of the road that help us stay focused on the true intent of running a beta. Here are a few of those tips, in no particular order.
1. Define a hypothesis
What are you looking to achieve or prove through the beta? Your hypothesis should define this objective — and it should align with both your product goals and the goals of the broader organization.
An example hypothesis: giving users the ability to filter results by city will enable them to easily run a new type of market report, thus increasing usage of our analytics suite — a stated business goal for next quarter.
2. Identify your beta users
When identifying potential beta users, it’s common to think in terms of titles or personas — such as “digital marketing managers,” or “mobile-first moms.” This is a good place to start, but don’t forget to consider user behavior when identifying beta candidates.
Who’s a good candidate for testing out the latest version of your market reports? Probably someone who’s currently a heavy user of those reports. Other good questions to ask: Which customers have asked for this product or feature? Who has exhibited similar behavior with the existing product or features? It helps to have a centralized, up-to-date record containing this information.
What we’ve found: users who have either explicitly requested the beta’s functionality — or who have been heavy users of related features — are far more likely to be active beta participants than a randomly selected group.
3. Run Agile
Simply defining releases does not mean you’re not being Agile! You must connect the user feedback you’re capturing, assess it, and incorporate it into your actual development work. At Wizeline, we do this on a regular cadence by defining releases that represent sprints. We then incorporate feedback on a weekly basis, ensuring that we’re taking account of all the latest feedback at consistent intervals.
4. Involve your go-to-market team
Your Marketing, Customer Success and Sales teams can help identify potential beta users and help get your beta up and running, faster.
- Marketing can provide information on customer need based on the current market; they’ll likely also have some valuable data on a customer’s history, from original marketing source to current state
- Customer Success has insights on user’s previous behavior with the product, which provides information on their needs, indicates why they are a good fit for testing, and helps predict future interaction with the beta
- Sales should have a list of prospects who have requested your functionality previously, and can give insight into the potential revenue tied to building features or products associated with certain customers. Involve your go-to-market team before starting a beta to ensure you engage the right customers in the right way.
5. Embrace negative feedback
While tough to hear, negative feedback is arguably more valuable than good feedback during a beta — and it will make the product or feature better overall. This is especially true if you’re able to dig in and understand why your beta doesn’t meet the customer’s expectations or fulfill their use case.
Related: Users want to know their voice is being heard, and quickly addressing negative feedback can help users continue to engage with your beta and providing even more feedback for the best product for the market.
6. Iterate until you reach a determined endpoint
Before beginning the beta, identify your “finish line.” This should be based in your hypothesis and quantified as a metric. Was your objective to have users interact with a new feature in a specific manner? An example endpoint could be: At least 45% of beta users created 10 new city-based segments in the analytics suite by the predetermined end date.
7. Report on progress
Share the good stuff with your team as well as with customers! Quantify and communicate the end results to all involved in the beta, and then solicit qualitative feedback in the form of a survey or user research interviews. This feedback will help improve the next version.
At Wizeline, we follow these practices to stay Agile. With feedback coming in directly through the Wizeline platform, our PMs and engineers are informed, continuously focused, and able to deliver according to a clearly defined finish line.