This post is the second in a series devoted to the development of Helperbot, Alley’s Slack application for remote Scrum teams. You can also read about where the idea came from in the first post in the series and read about MVP for Marketing – Realizing Helperbot’s Publicity in our third post in the series.
Minimum Viable Product, or MVP, is an agile concept with roots in Lean Startup, Lean Manufacturing, and Customer Development methodologies. It is generally defined as a product release with the most basic set of features to be usable by early adopters so they may provide feedback for future iterations. The idea behind releasing an MVP is that by continuously iterating on working versions, responding to feedback early and often, and allowing product developers’ assumptions to be challenged, engineers and stakeholders will be saved from costly, time-consuming, and ultimately unnecessary work in the long run.
Releasing MVPs in Sprint increments is a core practice of Alley’s development teams, so determining a basic feature set to bring Helperbot to market quickly was a natural starting point for us. As we mentioned in our first post in this series, our internal Alleybot is crammed with features, some of which are perhaps marketable (quoting your colleagues’ Slack snark), others we have already open-sourced (code review queuing), and others… well, not so much. For example, Swanson me!
To determine an MVP for Helperbot, we had a discussion about our goals for the product. We realized a key area of expertise that Alley has to share — besides strategy, user experience, design, and software engineering — is our role in pioneering Scrum for remote, distributed teams. Tooling is only a small part of fostering remote Scrum — a drop in the bucket really — but it is something we can easily share with the rest of the world. This guided our decision to make Alleybot’s Agile estimation and rate function the minimum viable product for Helperbot.
Feasible, Viable, and Desirable
While workshopping the idea of productizing Helperbot, the book Testing Business Ideas by David Bland and Alex Osterwalder became a key guidepost for us. Billed as a “field guide for rapid experimentation,” the book defined for us three different types of business hypotheses and how to test them.
The first was the Feasibility hypothesis. In other words, Can we do this? Can Alley manage, scale, and access key resources (technology, branding, IP) to bring Helperbot to market? Given that our MVP feature set already existed in the form of Alleybot, the engineering overhead, while not insignificant, was far more feasible than building something entirely new from scratch. Engaging the help of our in-house design and marketing teams for the logo and branding further established the feasibility of taking this project on. We determined that productizing Helperbot passed the feasibility test.
Next up came the Viability hypothesis: Should we do this? Can bringing Helperbot to market generate more revenue than cost? This was a more difficult question to answer. As an agency, our revenue comes from our work from clients, and work not spent on billable hours represents opportunity costs. So the question was, how much time and capacity were we willing to devote developer resources toward an internal project without an established stream of revenue?
This question brought us to the next and final question, the Desirability hypothesis. Do they want this? Is there actually a market for our value proposition, i.e. a Slack tool enabling Agile estimation for remote Scrum teams? If yes, is the market significant enough for our product to be viable?
This is where we had to face and perhaps challenge our assumptions. While Alleybot’s Agile estimation and rate features are super useful, arguably critical, to Alley’s own Scrum practice, would it be so for a significant number of other users? It became clear that the majority of our product experimentation and market research needed to be conducted with the goal of proving, or disproving, the Desirability hypothesis. To accomplish this, our first step was to deliver an MVP to a selection of non-Alley alpha users and let them take Helperbot out for a spin.
Alpha User Testing
To find alpha users, Alley looked to its own network of peers and former colleagues. It didn’t take too long for us to secure the participation of two SaaS companies, one consulting agency, and a startup — each of which employs remote Scrum to one degree or another. We gave each of the four alpha testing groups an early version of the Slack application and asked that they use Helperbot over the course of a few sprints, using the estimation and rates features for Sprint planning and Backlog Refinement. We established five criteria to determine desirability:
- We believe that Helperbot will empower teams to asynchronously estimate and rate user stories.
- We believe that Helperbot’s integration with Slack and Jira will be highly valuable to teams.
- We believe that Helperbot will reduce meeting time teams need for Scrum activities like estimating and rating.
- We believe that Helperbot will improve team efficiency.
- We believe that Helperbot will support the Scrum practices of teams.
To test for the above, we surveyed our alpha users to rate the degree to which Helperbot supported their teams in each of these distinct areas. Our criteria for success was measured as whether more than 70% of users realized a somewhat or significantly improved ability to function as a Scrum Team, and fewer than 10% of users realized any diminished ability to function as a Scrum Team.
In addition to survey testing, we also conducted a series of user feedback interviews that were more qualitative. Our interview below can be adapted to any product idea, so feel free to repurpose these questions!
Interview Questions
- Tell us a bit about what your experience with Helperbot has been so far.
- What features have you found most valuable when using Helperbot? (e.g. Estimate, Rate, Async, Jira integration, using Slack to run estimation games)
- What has Helperbot’s overall impact been on your team?
- Have you experienced any frustrations using Helperbot?
- What would make Helperbot more useful to your team?
- How useful are the following for your team?
- Asynchronous rating and estimation
- Slack integration
- Jira integration
- Has Helperbot helped your team improve upon any of the following?
- Meeting time
- Team efficiency
- Scrum practices
- Has anything unexpected or surprising come with using Helperbot?
- How much of a Slack feature feeling did this have? Add-on?
The results were encouraging and, at times, even surprising! Below are some highlights of the feedback we received from our alpha testers.
- Users reported past efforts to adopt similar products that ultimately failed because they were too far removed from the users’ established practices, tooling, and workflows.
- A key advantage of Helperbot was that it “worked where they live.” (i.e. in the remote office space of Slack.)
- Users reported that Helperbot helped their teams to establish consistent Scrum practices, such as using numbers in the Fibonacci Sequence for user story pointing and fostering team discussion when story votes are out of range by more than two degrees.
- Helperbot helped one user group — new practitioners of Scrum — to get their teams to adopt user story estimation as a general practice much faster.
- Helperbot shortened meeting times and helped to eliminate “dead air” in Sprint planning and Backlog Refinement.
- One surprising finding was that Helperbot’s Jira integration option was less of a key selling point than we had anticipated (based upon the self-reporting of our alpha users).
- Another surprising finding was that the ability to estimate user stories asynchronously was less important to our alpha users, who were almost uniformly in the practice of running user story estimation in real-time.
- Finally, one alpha user group cited a use case for the Rate feature that Alley had never considered! In this case, hiring teams used Rate to determine whether interview candidates would advance to the next round of hiring via a fist-of-five vote.
At the conclusion of our alpha testing phase, we decided that Helperbot had passed the Desirability test. Huzzah! After several more rounds of development revisions – responding to bug reports, feature requests, copy changes, UX/UI tweaks, and design feedback, we finally brought our product to market via the Slack app Directory. In our next and final post in this series, we will discuss the marketing of Helperbot and our present efforts to see wider adoption and usage of our friendly Scrum assistant. Stay tuned!