Troubles with quantified risk

  1. Security return-on-investment does not guarantee investment.
  2. Assuming a breach, or proving the absence of one?
  3. Multiple definitions of the word: “Risk.”
  4. Competing beliefs for the meaning of probability.
  5. Systems are too complex to predict quantitatively.

See? If we spend money on security, we save money!”

That lure of quantifying risk is supposed to be appealing to leadership.

  • The lack of obvious mitigations (“Driving without a seatbelt is a risk”)
  • An amount of property at stake (“We have too much risk in volatile investments”)
  • General expressions of fear. (“This feels pretty risky…”)

The three nouns risk, safety, security, and the two adjectives safe and secure have widespread use in different senses. Their polysemy will make any attempt to define them in a single unified manner extremely difficult. *

This may seem to be an obvious point. Still, a probabilistic model still has the burden of being communicated. Enough professionals in security / risk are unfamiliar with specific probabilistic language, or, opposed to the thought of being an exercise in prediction altogether.

Summarizing lessons

I still view probabilistic risk methods as a fascinating area to improve information security, but difficulties with collaborative quant will hold it back. Here’s a summary of lessons learned:

  • Organizational influence is not solved by quantifying risk.
  • Quantification will force you to re-examine your assume breach mentality.
  • Quantified risk language clashes with everyday risk language.
  • We prefer frequentist approaches. Never exclude subjective ones.
  • Design systems to be resilient against consequences with unknown causes.

Other:

I’ve written about a few other areas of criticism. Instead of repeating, I’ll link them here.

  • “You can’t predict adversaries.” — (1)
  • If it’s not falsifiable, it’s not scientific.” — (1, 2, 3)
  • Why can’t I measure performance with risk reduction?” — (1)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store