A Professional Crash Course on Beta Testing


Introduction

Before launching a product, feature, functionality, or even improvement for all customers in the base, you can perform a beta test so that a narrow group of users can test the news.

And unlike other tests (Alpha, or usability test) we perform this type of test without the presence of a moderator, outside a controlled environment.

We use Beta testing typically by developers to identify issues and bugs in an application, but it can also (and should) be used by designers, to understand how the experience of use is being.

Why take a Beta test?

Throughout our research we have contacted our clients a few times: interviews to understand their pains and their expectations, co-creation dynamics, and also usability tests of the hypotheses of solutions designed.

Even so, nothing replaces the use of the resource on a day-to-day basis, over several days, and in its actual context.

Therefore, if we fail to perform this follow-up, we run the risk of not realizing barriers that can arise only in the second moment of use of the resource.

Beta testing: how developers test applications on Android and iOS ...

And how do we do?

In the blog, we have shared how we perform Beta tests. We will discuss about the experiences that you will face in beta testing.

User selection

The first step is to set the number of people to participate in the Beta test. We determine this number considering that there is a funnel of people who will: receive the invitation by email or via the platform, read, accept, use the resource, and, finally, to talk to us.

The number of invited users varies from project to project, as well as the impact of this participation on resource-related decisions. To make this selection, we use our Ideas Portal, our Patrons (partner customers), about whom we talk more in this article, and other interested users. It is important to select various and extreme profiles in these cases to avoid biases in the search result.

Disclosure

We send the invitation for our customers to know a feature or functionality in a communication channel available within our platform. After that, from their first positive return concerning participating in the tests, we continue with the contacts.

What is Beta Testing? Definition, Features & Advantages

Usage tracking

When planning a Beta test, it is interesting to map several usage scenarios, that is, several possible ways to get to a feature or functionality and use it. Having a macro view of what the experience of different user profiles is critical for the future.

Search time

Previously, we’ve set a minimum number of people who need to be using the feature to start our analysis. When that number is reached, we start with the polls. We usually do it in two steps, in light of our Account Product Design Process:

  1. Guerrilla calls: We see the list of users who are using and contact you by phone.
  2. Sending online search: After calling, we send an email search to users who couldn’t answer our calls.

What’s important to know?

In the survey, we try to answer the following questions: Customer motivation to use the feature or functionality: Whether what we design meet the needs of their company and if there is any difficulty in the flow of configuration of this feature or functionality and how much customer satisfaction is with the release.

With these answers presented to the team, we were able to plan the next steps:

  • Have we solved the client’s pain?
  • Do we need to make any changes?
  • Can this change be made in an upcoming release, or does it need to be made now?
  • Is the feature ready to be released to the entire base?

The results of this approach have been increasingly positive. By talking to customers after launch, we approach their reality. And with that, you can see the true paths they take to act.

If the results are positive, we make the launch and dissemination to the entire base. If not, we adjust what is needed before launching.

Conclusion

At the end of all research, we leave with ideas of new opportunities for the product. Another point we noticed is that after this practice, the team feels much safer to release a feature or functionality to the entire base.

Besides, showing the testimonials of customers, we present the value we are delivering and motivate the team to continue improving.