How do you catch unforeseen errors in the system as fast as possible, for example, if your API stops working? Testing will reduce the number of issues you will encounter; however, the most complex ones are the ones that cannot be tested. Be prepared and avoid the illusion of better quality with more automated tests - sometimes you have to invest in different testing strategies by improving the monitoring setup (which actually is tooling for testing in production). In order to give you some good tips, we wrote an article about how to combine monitoring and testing.
When delivering high quality software, traditional testing is not enough. You need balanced testing with some possible edge cases that are monitored instead of tested. Why is writing the tests not enough?
You can write unit tests for complex logic, integration tests for your API, maybe even some UI tests, and from time to time, testing manually is fine as well. However, software systems are complex and in some cases the resources are better spent on detecting errors instead of pre-testing.
Consider a case where your external partner is suddenly changing their API which you have used for integration. Your requests start to fail. If they didn’t notify you ahead of time (which happens), you cannot really test for that. Or if the user has a faulty social security number not matching the standard - your software might handle this case fine, but if it doesn’t, then at least detect the problem. You can’t avoid all of the problems, but if something happens you can let the user know. For example, notification can be sent through an email that you are aware of the problem and it will be fixed in 30 minutes. The user will be impressed by this kind of personal communication because it’s not common.
Test what is reasonable to test since the unthinkable will happen anyway - and when it does, you want to catch it before the user reports come in. How to be prepared:
Make sure you have all means possible to debug different types of issues. With every issue that is tricky to debug, analyze the hacks needed to find the root cause and think about what could be improved in the future. If you wished you would have had a specific type of monitoring graph, now is the time to implement one.
Implement a release process that allows you to fix production problems quickly with confidence. Use feature flags to turn new functionality on and off, on demand. If you can get your mean time to recovery fast enough, the service desk will not get hammered and customer love continues to grow.
In conclusion, testing will reduce the number of issues you will encounter; however, the most complex ones are the ones that cannot be tested. Be prepared and avoid the illusion of better quality with more automated tests - sometimes you have to invest in different testing strategies by improving the monitoring setup (which actually is tooling for testing in production).