I want to explain something about where causal analysis comes from in my own background, because it explains why I think it can be learned.
I did not start out as a causal analyst. I was a Royal Navy engineer — infrastructure engineering, communications systems, the technical architecture that keeps a vessel operational under conditions where it is not supposed to fail. The analytical framework I learned there was not labelled “causal analysis.” It was called fault isolation. When something failed, you traced the failure systematically. You did not guess. You did not accept the first plausible cause. You worked through the dependency chain until you found the condition whose removal would have prevented the failure.
Later, I spent significant years at companies that built this methodology into commercial software — RiverSoft for network fault isolation, SMARTS for telecoms root cause analysis, Voyence for military and enterprise network configuration management. The products were sophisticated. The underlying reasoning was teachable. We trained field engineers, sales engineers, and technical pre-sales people to apply it. None of them were data scientists. Most were not engineers in the academic sense. They were practitioners who learned a structured method and applied it repeatedly until it became instinctive.
The same principle applies to causal analysis as a governance skill. It can be learned. It is not an innate talent. It is a method.
What you are actually learning
Causal analysis as a business skill is not about mastering formal Bayesian networks or building computational causal models. Those are specialist tools. The transferable skill is something more fundamental: the habit of asking what caused this, rather than what describes this, and following the causal chain to the structural condition rather than stopping at the proximate event.
The distinction is not subtle once you see it. Most business analysis describes. It identifies what happened, characterises the pattern, and suggests what the pattern implies. This is useful. It is not causal.
Causal analysis identifies the directed relationship: X caused Y, not merely X and Y occurred together. It applies the counterfactual test: if X had been different, would Y have occurred? If no, X is causal. If yes despite X being different, X is correlated but not causal.
For governance, the skill becomes: when an incident is presented to the board, before accepting the description of what happened, ask what structural condition made it possible. Not “what happened” but “what would have had to be in place for this not to have happened?” That is the counterfactual governance question, and asking it consistently is the whole of the skill at its most practical.
The five whys — and where it falls short
The five whys methodology is the most widely used approximation of causal analysis in business, and it is a reasonable starting point. The idea is simple: when a problem occurs, ask why five times. Each answer becomes the subject of the next why. By the fifth why, you have typically reached a root condition rather than a proximate symptom.
An example: a critical AI system produces an incorrect output that reaches a customer. Why? Because the output was not reviewed before reaching the customer. Why? Because the review stage was bypassed in the automated pipeline. Why? Because the pipeline was configured to bypass review when the system confidence score exceeded a threshold. Why? Because the threshold was set based on validation data that did not include the edge case type that produced the incorrect output. Why? Because the validation scope was defined before the deployment context was fully specified, and was never updated when the context changed.
That fifth answer — the validation scope was not updated when the deployment context changed — is a structural governance gap. It is not the incident. It is the condition that made the incident possible.
The five whys is a useful discipline for linear failure chains. Its limitation is that most significant business failures are not linear. They are confluent: multiple conditions that are individually insufficient but jointly sufficient to produce the failure. The five whys, applied to a confluent failure, will identify one causal thread and miss the others.
For AI governance failures specifically, the confluent structure is almost universal. A model deployed outside its validated range (condition 1), reviewed by a human who was not trained on the model’s specific failure modes (condition 2), in a context where no aggregate monitoring existed (condition 3) — none of these conditions alone would necessarily have produced the failure. All three together did. The five whys reaches condition 1 and stops. Causal analysis identifies all three.
The counterfactual discipline
The discipline I would recommend to any executive who wants to develop causal reasoning as a governance skill is this: after every significant incident or decision, apply the counterfactual test.
For incidents: what would have had to be different for this not to have occurred? Work backwards. Each answer to that question is a potential causal factor. Test it by asking: if only this were different and everything else remained the same, would the outcome have been different? If yes, it is causal. If no, it is correlated or coincident.
For decisions: before approval, ask what conditions would have to be true for this decision to produce the outcome it projects. Then ask which of those conditions are verifiable rather than assumed. Then ask what the governance mechanism is for verifying the conditions that are currently assumed.
This is not a complex analytical procedure. It is a consistent habit of question-asking. The habit, applied over time, changes the quality of information requested before governance decisions are made. Boards that apply the counterfactual discipline consistently stop accepting descriptions of governance structures and start requiring demonstrations of governance function.
Military and telecoms RCA translated to boardrooms
The root cause analysis methodology I learned in naval engineering and later applied in commercial software has three features that translate directly to boardroom governance.
Systematic elimination. In network fault isolation, you do not guess at causes. You eliminate candidates systematically, starting from the observable symptom and working backwards through the dependency chain. You do not stop when you find a plausible cause. You stop when you have confirmed that no upstream condition was also necessary for the failure to occur. In boardroom terms: do not stop the causal analysis at the first reasonable explanation. Ask whether the first explanation is complete or whether it is the symptom layer of a deeper structural condition.
Dependency mapping. In network analysis, every component has dependencies — other components whose function it depends on to function correctly. A failure in a dependent component can manifest as a failure in the dependent one. The dependency map tells you where to look. In governance terms: when an AI system fails, the dependency map is the governance architecture — the oversight mechanisms, review stages, escalation paths, and accountability structures that the system’s safe operation depends on. The failure may be in the system. It may be in a dependency. The dependency map is the governance framework.
Reproduce and verify. In fault isolation engineering, a fix is not confirmed until the failure can be reproduced and shown not to occur with the fix in place. In governance terms: a remediation plan is not confirmed effective until the specific conditions that produced the failure are tested against the remediation and confirmed not to produce the same outcome. Most governance remediations are approved as plans. Very few are verified as functions.
A practical development programme for executives
If I were designing a six-month development programme for an executive who wanted to develop causal analysis as a governance skill — not as a technical specialisation, but as a board-level reasoning discipline — it would look approximately like this.
Months 1-2: Apply the five whys to real incidents. Take three incidents from the organisation’s history and apply five-why analysis without stopping at the first plausible cause. Write the full chain. Note where the analysis reaches a structural condition rather than an operational one. The structural condition is almost always in the governance layer.
Months 3-4: Apply counterfactual testing to pending decisions. For the next three significant decisions that come to the board, apply the counterfactual test before approving. What would have to be true for this to produce the projected outcome? Which of those conditions are verified rather than assumed? What is the mechanism for verifying the unverified ones?
Months 5-6: Apply the dependency map to one AI system. Take one AI deployment currently in operation. Draw the dependency map: the governance structures — review mechanisms, escalation paths, data quality controls, human oversight stages — that the system’s safe operation depends on. Test each dependency: is it functioning, or is its existence documented but its function untested?
At the end of six months, the executive will not be a causal analysis technician. They will be a governor who asks different questions — questions that are harder to answer with a governance description and require a governance demonstration.
That is the whole of the skill as applied to boards. The questions change. The governance changes.
The Board AI Governance Framework incorporates the causal analysis questions described here into a structured board governance tool — decision-making frameworks, counterfactual review questions, and dependency audit structures designed for directors governing AI systems.
For boards seeking advisory support on developing causal governance capabilities, contact Steven directly.
