Up and down the stack

I am starting to become more and more convinced that to spot issues with a piece of software, you need to go take a wider view of what consitutes ‘the application’. In a typical Java server side application, ‘the application’ does not include only the code running on the JVM.

For instance, looking at a database query log can be very instructive. It is amazing how much the number of generated queries can be reduced in some cases by some simple changes.

But it does not stop at the database. You need to inspect the whole environment the application is deployed in. The web server configuration can be a source of unnoticed bugs, and if logging is not setup correctly you will be left in the dark.

Focus on unit testing is great, but as long as you limit yourself to testing your development configuration, the deployed application could still not function as required after all tests pass.

This raises the very hairy question of configuration management making whole environments reproducible and testable. The issue is further complicated when new programs must be deployed together legacy systems, where the configuration might be undocumented. You are then stuck with an unkown environment that you cannot modify for fear of breaking the legacy application.