QA insights from Fortune 500 companies: Agile, QA management & processes optimization

Small company developers often complain that they have to test their code. QA engineers should do this work, they believe. And they are wrong! Everything depends on the company's approach to the development and testing processes. In particular, Facebook, Google, and Amazon follow the practice when a developer tests his code and code written by colleagues. What other approaches to quality assurance do Fortune 500 companies use in their processes? We find this out in our article.

Test automation can simplify work even in a small company. What can we say about the world’s largest technology corporations. Let's see how they approached automation implementation.

• Amazon loves automation and strives to automate everything that could be automated. The more processes are automated, the higher the speed of work on the project. Because of this, the development and deployment processes are well-tuned. So, the company does not need to put in a lot of effort for testing. The software is designed in such a way that it is easy to deploy, and in case of failure, it is also easy to roll back.

• Wayfair is a home shopping marketplace with offices in the US, Canada, and Germany. The company operates in the competitive online retail market. Wayfair automates only those tests where it makes sense from an investment standpoint. The automated test results are analyzed in terms of metrics that are a priority for the company. As noted in the company's blog, performance-related metrics are essential. QA teams then create dashboards with graphs that illustrate the state of the product. This approach allows all employees to be involved in product improvement.

• Test automation enables repetitive tests to be performed with greater accuracy. This is Google's approach. Thus, the release becomes closer. The human mind is too valuable and creative to be wasted on something a computer can do, James Whittaker pointed out in the book "How Google Tests Software".

The world's tech giants use canary testing, dogfooding, and testing in the toilet (yeah, sounds funny and a little scary). Of course, Fortune 500 companies that use Agile methodology as a key approach for testing have a more flexible and iterative testing process. Let's take a look at this in more detail.

• Wayfair takes advantage of the geographical aspect in testing. For instance, firstly they release a new version of the product with certain functionality in Europe and analyze the results. If the reaction of the audience is positive, then this version is made available in America. Also, Wayfair uses an A/B testing method. One updated version is made available to one part of the customers, another version is made available to others. Then the company measures the performance difference.

• Computer and software manufacturer Dell helps users solve device problems themselves. They run hardware tests through the Dell Support website. Such an approach saves the user time and solves the problem remotely. Online testing allows customers to detect viruses, spyware and malware, malfunctions of device components, and other problems.

• Amazon makes the software easy to deploy. Simplicity allows them to make a quick rollback in case of failure. So Amazon takes a canary approach. Canary testing means you test your software only on a part of your customers. This way you can identify bugs, fix them, and only then provide the app to everyone. This strategy allows the company to make a huge number of deployments and easily roll back those that fail canary testing. Failures do not have a significant impact on the entire user base.

The canary method comes from the time when miners lowered a caged canary into a new tunnel. If the canary survives, it means that there is no admixture of gas in the air and it will be safe for people to work there. If the canary was dying, then it is impossible to enter this tunnel, as it is deadly. 

• Corporations often apply the Dogfooding approach in testing. Dogfooding means using your product for work or personal purposes. This approach allows you to identify possible problems in the product. You see how your software works, and you test it from the point of view of a real user. Microsoft employees use Microsoft products for their work, and Apple staff use MacBooks. The Facebook React feature appeared as a result of dogfooding on Facebook (Meta).

• Software quality is something without which the product cannot survive in the competitive environment. Google understood this long before testing became mandatory. In 2007 (it seems that this is already a prehistoric era!) in its blog about testing, Google announced the “Testing on the toilet” initiative. QA activists posted information about approaches to testing in the office's toilets. Ridiculous, but practical.

Turning to a more serious approach, Google has a four-step process for testing search engine changes. Here are its components:

1. Testing by Google employees.
2. Testing on the crowd-testing platform. 
3. Dogfooding.
4. Beta testing.

One more Google approach to QA is that the company believes in a pyramidal testing model. So, maximum resources are invested in the unit and integration testing, and E2E testing is used less. This allows for frequent releases and minimizes testing failures.

No one but real users will be able to test the application better. This has long been understood in leading technology giants’ development and testing offices. Companies implement beta testing programs to find all the possible bugs that users may encounter.

• Apple has a beta testing program in which any user of the company's products can take part.  The Apple Beta Testing Program allows everyone to get involved in the development of Apple software.  So any customer can test pre-release versions.

• Google allows anyone to try out new features before they become available to everyone. The early testing program involves working with Google applications that are not yet released. The beta testing program helps you work with experimental versions of released apps.

• Microsoft has launched its community of independent product testers. In September 2014, the company announced the new Windows 10 operating system and invited everyone to test this system. This happened a year and a half before the operating system went on sale. This is how the Windows Insider Program was created. Windows Insider is a community with loyal customers who enjoy being able to influence the software they use.

Fortune 500 tech companies strive to have their tools for successful automation.

• Wayfair uses many different applications, both external and internal. Over a hundred teams support all these tools. In particular, Wayfair works with solutions for continuous code quality control. This allows them to eliminate all possible vulnerabilities that can spoil the end-user experience.

• Apple owns its quality assurance tools, many of which the company has acquired. In particular, the TestFlight service is owned by Apple. This is the main way to beta test Xamarin.iOS applications.

• Expedia Group is a global travel company. Its products are used by travelers around the world to book accommodation, tickets, and cars. The test pipeline's correct implementation improves the experience for development and QA teams, they believe. It allows a company to speed up the release of products. The key to this is using the right tools and solutions. The company has integrated a static JavaScript code analysis tool ESLint into its testing pipeline. This helps to find all the errors before the release, and save time for development teams.

• Facebook also uses many external and internal automation solutions. Among the tools that the company used are PHPUnit (backend unit testing) and Watir (full product testing).

• Amazon ensures every development stage is easy to control. It is important for the company that developers have confidence in the code they write. Amazon has created and uses internal tools to achieve this.

Testing software must simplify the work on the product and save the team's resources. A high-quality toolkit allows you to achieve the highest possible software quality. Zebrunner testing platform helps you understand how this works in practice.

Developers perform some type of testing, while QA engineers are responsible for product quality at a higher level. See how developers and testers are treated at companies that top the Fortune 500 list.

• Testing at Amazon is usually done by the developers themselves. There are seven developers for one QA engineer in the company. QA employees are focused on test automation infrastructure development and higher-level functional testing. Amazon's developers conduct integration, cross-browser, regression, and other types of testing.  QA engineers are responsible for the automation system and thus they form the basis for development and testing.

• Microsoft has a ratio of testers to developers is two to three. Moreover, their salaries are about the same. Microsoft calls QA engineers Software Development Engineers in Testing (SDET). They are critical to the company. Microsoft earns a significant part of its revenue from software that is installed on customer computers. Such products are much more difficult to update. The company faces big losses if bugs in new functionality are found after the release. The goal of the QA team is to find all the bugs in advance.

• Google also provides the same salaries for testers and developers. Search engine updates are tested very carefully. The team behind the search engine maintains a sophisticated testing framework. Everyone needs to make sure that the product meets the highest quality standards. At the same time, the teams that work on other Alphabet products are more relaxed about testing. Often the developer who is responsible for the product tests it himself.

• Facebook has built a product development system in such a way that developers test their own code and the code of colleagues.