Last Call for Quality

A look at why late testing fails, how to avoid this, and other common software testing anti-patterns.

Testing Too Late

One of the most common software testing anti-patterns is testing too late. In traditional quality assurance models, software is handed over from developers to testers at the end of the release cycle. This leads to QA as gatekeepers and/or a bottleneck, which is not ideal. Additionally, the later a bug is discovered, the more expensive it is to fix. By shifting testing left, bugs can be found earlier, making them less costly to fix.

When I talk about “shift-left,” this means moving things earlier in the process. Take a typical lifecycle process of design → code → test → ship → maintain. Shifting left means moving testing to the left.

Ideally, testing should not occur in isolation but as an integral part of each step.

---

You can read more about the differences between quality assistance and quality assurance in this article from 2023.

Quality Assistance, Not Assurance

---

The Pitfalls of Late Software Testing

Testing is, ultimately, a feedback loop for developers, indicating whether their code performs as expected. When it doesn’t, this means the software has bugs. Viewed this way, testing too late means the feedback cycle is too long.

Why does testing happen too late? It’s often the culmination of multiple software testing anti-patterns and development practices, such as over-reliance on manual testing, separate development and test/QA teams, and a lack of quality ownership throughout the development process.

Consequences

• Higher costs of fixing issues found late in the cycle.

• Due to the higher cost, not everything found will be fixed, lowering the quality of the product.

• Missed deadlines, as major bugs can delay software releases.

• Stress on the team and potential burnout.

• Impact on customer experience, satisfaction, and trust: persistent quality issues can damage the brand’s reputation and erode customer trust.

Alternatives and Best Practices

When we consider testing as a feedback loop, our goal should be to make that loop as short and meaningful as possible. We can do this by shifting testing left (earlier) and, therefore, shortening the feedback loops.

With shift-left testing, testing starts as early as possible in the software development lifecycle. There are many ways¹ to add testing earlier, such as:

• Requirements testing

• Design reviews

• Code reviews

• QA kick-offs

• Test Driven Development (TDD)

• Early test planning

• Unit testing

• Paired testing

• Continuous Integration and Continuous Deployment (CI/CD)

• Automated testing

• Risk analysis and risk-based testing

• Increased communication and collaboration between testers and developers

Other Common Software Testing Anti-Patterns

Now that we’ve covered the pitfalls of testing too late, let’s look at some other testing anti-patterns.

Over-Reliance on Manual Testing

I want to start by stating that manual testing is a great tool and can be essential for a robust quality strategy. However, because it can be time-consuming and human-resource-intensive, we need to be intentional when we choose to use it.

Often, manual testing is done by a separate team after the code is “done.” Whether manual or automated, waiting and throwing it over the wall at the end of the release cycle like this is one of the primary drivers causing testing to happen too late.

Corrective actions:

• Look at your code/test coverage metrics and compare them to the testing pyramid² and testing trophy³. Where can you add more testing that is not manual? Where can you add more testing earlier in the lifecycle?

Ignoring Non-Functional Requirements

Overlooking non-functional aspects of the product, such as usability, security, and performance, leads to software that may meet requirements but doesn’t satisfy user expectations.

Corrective actions:

• Include non-functional requirements explicitly in your product specs.

• Add non-functional testing categories to your test planning.

Poor Test Data and Test Data Management

Poor test data and test data management can lead to the release of defective software, which can affect user experience and satisfaction.

Poor test data can lead to incorrect test results, causing defects to be missed (false negatives) or non-existent issues to be reported (false positives).

On the management side, inadequate test data handling can expose sensitive or confidential information. Using production data in testing environments can inadvertently create security vulnerabilities.

Corrective actions:

• Create, maintain, and secure test data that accurately represents real-world scenarios. There are many tools⁴ to choose from that can help with test data management.

Lack of Test Environment and Configuration Management

A lack of proper test environment and/or configuration management can cause the "works on my machine" syndrome. This often leads to missed bugs because the pre-production environment does not match the production environment.

Corrective actions:

• Tools such as docker images, Puppet, Chef, or Ansible can ensure pre-production environments match the production environment.

Conversation Starters

• What strategies have you found effective in implementing shift-left testing in your development process?

• What have you found to be the most effective ways to shorten feedback loops in software development and testing?

• What methods have you used to prevent testing from becoming a bottleneck in your release cycle?

Like this article? Click the ♥️ button to let me know!

Or leave a comment 💬 and share your thoughts.

xo,

Brie

PS. If you’d like to support my writing and my work on QUALITY BOSS, you can show appreciation by leaving me a tip through Ko-fi.

---

Coaching/Mentoring: Want to work with me? Book a call here to discuss getting started.

My areas of expertise and interest are leadership development, conquering impostor syndrome, values exploration, goal setting, and creating habits & systems. And, of course, Quality Engineering. 🐞

1. I’ll cover some of these in more detail in future articles.

2. Martin Fowler on the Test Pyramid.

3. Symflower on the Testing Trophy.

4. The QA Lead on Test Data Management Tools.