The Art of Triage

To many, troubleshooting seems to be a gift that either you have or you don't. For instance, my father is a mechanic. When he owned his own repair shop he would hire young guys who would spend hours troubleshooting a problem, but then within minutes from my old man getting involved he could quickly diagnose the cause of a problem. The timing belt, carburetor issues, whatever it was, he was quick to pinpoint. Inevitably once the problem was found there was the “oh, of course” from the Junior grease monkey.

I was too clumsy to be a mechanic, so my dad fired me and forced me into Computers. However, I didn’t forget what I had learned about troubleshooting.

First, troubleshooting is not something you are born with. It is a skill that is harnessed based on 3 common factors:
1) What you know
2) What you don’t know
3) What you are learning

When you piece these 3 factors together you create the framework for discovery. By adding at negative and positive approach will then lead you down a path of what good troubleshooters simply call the process of elimination.

Do you know what is working? Do you know what is not working?
What don’t you know is working? What don’t you know is failing?
What have I proved with this step? What have I disproved with this step?

So when it comes to troubleshooting complex systems, the same principle applies. You just need to analyze them in layers. Here are the layers that VIGILANT has documented as the logical points to eliminate.

Infrastructure: Hardware, Networking, Operating Systems
Application: 3rd party application services
System Interfaces: Connectivity between dependant systems
Business logic: Business rules that cause transactions to operate differently
Business Process: The way the end-user is executing the transaction
Business Service: Dependency on data or other elements for success

For really complex issues, take each of these tiers and apply the 3 principles of discovery to them and you fill find the problem is not as much as a black-hole as you thought it was.