A few years ago, I got a job at a startup. It was the first technology company in my professional experience. However, I was surprised, because they did everything manually. Even replies to each new application user were sent manually, although it was possible to set up an auto-reply. “At this stage, it is better to do everything manually”, the CEO repeated.
He turned out to be right! Manual testing helped to detect critical bugs in time. Communication with each new client took time and seemed tedious. However, when the project scaled up, all the efforts paid off. Clients were happy to leave feedback, and also brought referrals. All these are thanks to manual work.
Every year, new tools for test automation enter the market. However, professionals know that manual testing is indispensable. In this article, together with Zebrunner’s QA team we consider reasons why it is impossible to test software without a manual testing approach.
Manual testing means testing software through research, experiments, simulation, observation, and other methods manually. The QA team doesn’t use test automation software. Manual testing makes sure that the application operates without errors in functionality. You also check if your software meets specification requirements and acceptance criteria.
The manual testing approach allows the engineer to interact directly with the product and simulate the end user’s actions. It provides an opportunity to explore the UX aspects of the product. Thus, the specialist is convinced that software works as expected in different scenarios of interaction.
Let’s figure out what are the most common reasons why you can’t do without manual testing.
UX testing helps you find ways to improve the user experience. When you do this kind of testing, you become a user for a while. You use your application the way an end-user would use it. This approach allows you to identify defects, as well as to come up with ideas on how to make your application better.
Zebrunner QA engineers say:
“If we consider UX testing, then manual testing has no alternative. You need to check how convenient it is for a user to work with the application. Only a human can do this. Let's say you use the mobile app of an online store to make a purchase. QA engineer checks how useful the key buttons are located, and whether the app interface is convenient. He also checks whether the application interface is intuitive and performs all the necessary functions to complete the purchase”.
A high-quality and user-friendly interface is essential for product success. It can only be fully tested manually. The engineer pays attention to everything that surrounds the test scenario. The specialist easily notices incorrect layouts, problems when switching between application screens, and more.
Zebrunner’s QA team comments:
“With UI testing, you check on how the application's interface looks like. For example, elements, whether they match the design in terms of colors and layouts. This type of testing is also more convenient to carry out manually. Yes, we have tools for UI testing. We used automation for UI verification on one project. We compare the design and content for web pages for old and new versions of the application. However, such testing was not ideal. There were many inaccuracies. Due to a delay in opening the page, the tool will indicate the presence of bugs, although there were no bugs. The tool for UI testing requires manual QA to verify results. The engineer checks everything again manually, which takes more time in practice”.
This is a very common situation when the budget doesn’t provide additional costs. Therefore, it is better to apply only manual testing. Hiring a manual tester is cheaper than an automation engineer. Test automation is a complex and lengthy process. It requires restructuring the entire software development process. In addition, you need time to write automated scripts and retrain the QA team to work with automation.
Manual testing saves the project budget. A manual tester can start working on a project with minimal experience and learn as they go. For manual testing, it is convenient to use the test case management tool. It helps to systematize the testing process. Such software allows you to speed up the creation of test cases and their management, run tests, and track results.
Any tester has done exploratory testing. It happens when the QA team hasn’t made a test plan and decided which tests to prioritize. This type of testing involves developing and executing tests at the same time.
The purpose of exploratory testing is to test the various capabilities of the software. An experienced manual tester can start exploratory testing of a product almost immediately. A minimal set of instructions or documentation will suffice for him. He will be able to detect the first results and defects already on the first day of work.
Our QA engineers say:
“Exploratory testing is done to explore the product when we don't have requirements. This often happens when a tester comes to a project where there has never been QA or requirements before. The tester begins to explore the application in practice. He learns how the application works and can determine test cases during testing. It is impossible to automate such testing”.
Let’s say your product is an app that helps a traveler find, book, and pay for accommodation. With exploratory testing, you understand from the user's perspective how well the application performs. So you check your key functions to ensure that it has good logic and works properly.
Ad hoc testing is also called intuitive testing. It is based on the improvisation of the tester. The tester checks the main features of the product under different conditions.
QA engineers explain:
“QA performs ad hoc testing after finishing the basic testing scenarios to find non-trivial bugs that the user may encounter. To conduct ad hoc testing, QA needs to know the product requirements very well and be very experienced. This type of testing requires creativity. The tester analyzes the application and performs actions that could theoretically lead to a bug”.
Zebrunner’s team says:
“To start test automation, we need to prepare test scenarios. They contain steps (actions) and expected behavior. Then an automation engineer writes code for these scenarios and reviews them. It takes time. If we have two weeks to develop and test the application for release, then during this period the automation engineer may not have time to automate all test cases in time. This is a blocking factor for the release.
Now let's turn to manual testing. While the developer is writing a feature, the tester can start testing it in parallel. He can find bugs and report them, the developer will fix them. As a result, since the development and testing process was going on simultaneously, the release will not be delayed, and the application will go to the market on time”.
Manual testing is real-time testing. You can immediately test small changes in functionality and provide feedback to the development team. For manual tests, you don’t need to spend time setting up tools and writing code for scripts. Manual checks are often carried out after the introduction of new functionality. If you need a quick spot-check of the new option functionality, manual testing is ideal. The best way to reduce testing time is to use the all-in-one test automation platform. It’s suitable for both manual and automation QA teams. This solution speeds up and facilitates all QA processes, including planning, executing tests, and analyzing results.
If you have a small and simple product, then it makes no sense to implement automation. It is cheaper and more correct to test the functionality manually for short projects lasting less than a year. The entry threshold for manual testing is much lower than for automated testing.
Zebrunner’s QA engineers share an example:
“Take a startup for which is important fast feature delivery to the market. Manual QA can verify and report product quality faster than automation QA. Another example is when some products frequently change requirements. They can roll out some features and update them before the release. For such projects, autotests are irrelevant. Automation engineers need to update existing and create new test cases and test the app in time frames. It's a time-consuming process”.
At the very beginning of the project, the team tests the documentation and requirements. Once the main QA development has begun, engineers must manually test key ways of using the app. Further, with each new feature, this scenario is repeated: for the first time, everything is checked manually.
The application doesn’t immediately become stable during the development phase. Almost always, changes are often made to the various features of the application. It requires changes in automation scripts. In this case, it makes no sense to spend resources on writing autotests. Until the application reaches a stable version, it is more efficient to test manually. After making changes to existing functionality, you check the correctness of its work and the work of related components. You can manually test almost any application, while it makes sense to automate only a stable system.
The safety factor is critical for some applications. It's not just testing software for vulnerabilities. It is also about possible data leakage during testing. For example, most automated testing tools are available through the cloud. For some products, this option is unacceptable due to the risks (though not very high) of data leakage. Manual testing is a defense against this kind of risk. The application is tested by the company's staff, and the application data is stored on the company's servers and isn’t uploaded to the cloud.
Manual testers put themselves in the place of end-users through testing. They analyze the application according to the needs of the target audience. This approach helps to improve the user experience. Automation has no such advantage when all software is tested in a normal way.
The manual tester looks not only at what is described in his test scenarios, but also checks for additional ones that are not in the test plan. He relies on his experience and intuition.
While automation works according to fixed scenarios, manual testing allows you to dive into the details. So you are more likely to find minor flaws. Autotest skips even the most obvious bug if it doesn’t detect errors of this type. With manual testing, this probability is minimized.
Some test scenarios are difficult or almost impossible to automate.
Firstly, automated tests can give incorrect results. Autotest may pass successfully, but in real conditions, the application will work with bugs.
Secondly, when testing a security system, there are scenarios where a modal window is shown to the user with which he must interact. However, no other application sees this window. Such a case can only be properly tested manually.
Thirdly, some test scenarios require physical actions from engineers. For example, when you need to switch connectors, physically move equipment, change the state of its temperature, humidity sensors, etc.
Zebrunner’s QA team notes:
“It is difficult to automate complicated installation and configuration scenarios with device connection. So if you want to test the setup and configuration of mobile device farms and connect mobile devices, do this manually (Zebrunner Device Farm could be the example here). Another illustration for non-automated scenarios is a sign-up process with various types of authentication (fingerprint, face recognition, captchas). Also, don’t automate the testing of gestures on mobile devices, check it manually. The same can be said about accessibility testing with voiceover screen readers. Non-automated scenarios also include integration with fitness applications and verification functionality that requires physical actions”.
It is possible to automate the testing of IoT devices. However, this is a time-consuming process. You can also turn to companies that specialize in automated testing of fitness trackers, smart refrigerators, snickers, etc. In addition to time, this will also require significant costs.
It’s easier to test IoT devices through manual testing. You check their functionality from the point of view of the end user and find critical errors. Moreover, you start IoT testing immediately. There is no need to spend time writing scripts for autotests.
Zebrunner’s QA engineers comment:
“We need different types of tests for smart device testing: configuration, integration, functional testing, and usability testing. Such devices need to be tested both manually and through automation. In simple terms, to test smart sneakers, you need to run. To test a UA heart rate device you need to put it on and perform activities that affect your pulse. You can automate only some components of such products. Manual testing is indispensable here”.
