Approaching quality process with experience-based and exploratory testing
Imagine a standard situation – a product went through the current development iteration and has been delivered to the end-user. The product delivers a great experience to the end-user, which means the quality assurance (QA) work has paid off. But with each finished product cycle, the testing scope increases and with that, risks of usability and proper functioning issues increase as well. We want this risk to be as low as possible, right?
The common QA approach includes:
- Following a defined set of test cases
- Planning the test run
- Getting the results
- Repeating the process
While this is a completely valid and functional way of testing, at one-point situations that do not fall into the executed test sets can start to be neglected. This can lead to the wrong outcome of functions, degraded performance, or even crashes. To achieve the desired quality, creativity and experimentation must be included to complement the existing testing activities. This is where experience-based and exploratory testing approaches can help.
Experience-based testing uses experience, knowledge, and the intuition of the QA specialist. Testers often catch themselves thinking:
- I have an idea of what could go wrong
- In my previous testing routines, this feature posed the biggest risk of defects
- What if this action causes issues in a certain situation?
The answer to these questions can only be found by investigating, examining, and exploring. Exploratory testing needs to be approached with a plan for where to head while leaving the choice of different roads and approaches open along the way. A common misunderstanding is that exploratory testing consists of executing simple actions, registering defects, returning to the usual routine, and not going through the process again. However, it’s a technique that needs to be practiced regularly and frequently to adapt to project needs.
Why do we advocate for exploratory testing? We recently worked on a project where we were responsible for providing the client with an application suite for purchasing and managing the stock of office supplies. Experience-based and exploratory testing strongly complemented the test cases and was a major point of success for the quality process. The results were:
- All new features were delivered on time
- All critical and major defects were detected during the testing phase
- Registered and fixed critical issues from simple translation errors to incomprehensible user flows that would have caused severe inconveniences for the end-user
This approach allowed the testing team to define comprehensive test scopes to cover and check scenarios that otherwise would have been missed if only the predefined test case set was used. To apply this to your own projects, the first one must understand what experience-based and exploratory testing consist of.
Experience-based testing techniques include:
- Checklists – an overview of areas that are going to be tested within a specified scope (function group, a page, etc.) with no steps defined
- Error-guessing – identifying areas containing potential issues
Both techniques depend on the tester approaching the problem using their experience acquired during the previous test and their instinct on where this should be applied. Exploratory sessions, on the other hand, consist of timeboxed sessions – a set amount of time spent checking the selected areas.
Here are four tips for exploratory testing:
- Start with rechecking the current process with the team. Identify which stage in development would be a starting point for trying out experience-based and exploratory approaches
- Agree on the conditions for the first session:
- Scope – decide what to focus on first, define specific areas or functions e.g., account management page, payment refunds, user sign-up process, etc.
- Setting – what test environment will be used, what is the timebox amount (30 mins, 2 hours, etc.)
- Frequency – how often (once a week, once every iteration, etc.)
- Reporting – where do we document the results, how do we report the issues found, how do we prioritize the issues
- After the first session is complete, review the results, and discuss what went well, and what went wrong. All feedback here is important.
- Based on the acquired feedback, update the process – increase/decrease the amount of experience-based and exploratory sessions done, change the amount of time used for exploratory sessions
This procedure will yield positive results throughout each iteration, due to the amount of frequent feedback that is acquired. From the team’s perspective, the benefits include:
- QA team has the opportunity to not only test the product from different points of view and catch the pesky hidden bugs, but it will also allow each member to express their unique way of testing and creativity
- Programmers will get an insight into common problems that can be avoided in the future
- Product owners will grow their confidence in the product
You might also ask, but what about test automation, is this applicable to me? The answer is yes – lessons learned during experience-based and exploratory sessions often bring additional scenarios to light that should be automated or provide details on what should be improved in existing scenarios.
The best part is that previously hidden bugs start to reveal themselves, allowing you to further identify actions for the quality process, such as:
- Additional test cases to be added for regular execution
- Additional testing types like performance testing, usability testing, etc.
- Identifying missing technical necessities e.g., tools for test data management, tools for test case management, etc.
Applying experience-based and exploratory testing as an addition to your existing testing activities can yield positive results as it is unique to each project and product. The main takeaway is to start small with frequent adaptations to gather feedback. Experimentation is highly encouraged and will lead to improved quality.
Additional Insights
The future of full stack engineering talent
The art of feedback
Don’t confuse tools with technologies
