Risk quantification can be confusing and derailing to groups and decision makers.

The following points are areas of pain when working with quantitative models with others. These areas of friction cause bad experiences, and bad experiences change our approaches in the future.

We’ll talk about the following topics:

  1. Security return-on-investment does not guarantee investment.
  2. Assuming a breach, or proving the absence of one?
  3. Multiple definitions of the word: “Risk.”
  4. Competing beliefs for the meaning of probability.
  5. Systems are too complex to predict quantitatively.

Let’s walk through them. Hopefully, you’ll still be excited (as I am) about the remaining usefulness of…

This is a method I’ve used to help frame and model cybersecurity risks over the past few years. It helps organize a lot of complexity when dealing with a large organization.

This method uses forecasts, scenarios, multiplication and addition. As all risk modeling goes, this has more to do with the journey of collecting risk knowledge in a rigorous way. We’ll discuss it’s weaknesses at the end.

1. Create a scenario.

This walkthrough will use the following scenario:

We disclose ≥1 incidents with ≥$10M damages in 2021.

What future outcome are you looking to investigate? Be confident about your definitions. This example mentions damages

Risk measurement quickly raises questions about management…. but not about risk management. Rather, managing the performance of people who manage complex risks.

My writing on risk measurement often gets attention from management roles. The management audience desires methods to manage the performance of defenders with risk based measurement. I decided to thoughtfully write out my views on that.

This will only partially read like a cyber security essay. Most management and measurement concepts are not specific to our sector — however, the characteristic of risk makes performance management even trickier. Let’s explore why!

We’ll start with the premise that organizations…

My hope is that the cyber security community will develop as a risk science.

Science starts with correctable claims. Progress towards more useful knowledge come from continuous corrections.

However, a risk hypothesis may represent future events that have never previously happened, might not ever happen, or may not be observed when it does happen. How can we pursue science with an unstable vantage point?

A risk hypothesis conflicts with common notions of science when compared to our expectations of the more mature sciences. …

A topic described as subjective is often considered non-scientific. Risk, being a subjective topic, must certainly be one that science has a hand in. Can we pursue a science of risk with these limitations?

An exploration of subjectivity and risk in a context of science forces us to confront the scientific method itself and how it relates to risk. The entire practice of cyber security shows its roots in the scientific method. It also shows how the scientific method is, itself, limited by the subjectivity of its operators.

With this essay, we’ll tie these concepts together towards a more comfortable…

How do we approximate the amount of resource we allocate for security? In this essay, we’ll cover some principles before the quant.

Let’s start simple:

  1. What risks does the organization face?
  2. How might the security team reduce these risks?
  3. Were the associated costs worth the reduced risks?

This approach might sound familiar and reasonable. It’s flawed. We’re now cursed with an organization using a perspective on risk from the late 1800’s that will fund a fledgling security organization.

This essay hopes to provide tools to inspect this gap we’ve created with this mindset.

Three sources inspired this essay: Imposing Risk

Insider threats are often discussed as a broad category. This essay explores the malicious flavor of insider threat and ignores the “human error” or “accident” categorizations.

Each scenario is supported with in-the-wild observations. I’ve done my best to select resources that describe the threat’s TTPs or motives as best as possible.

I’ve expressed these risks as scenarios to help focus a group discussion. Narrow mitigations are easier to brainstorm with clear scenario-based requirements. Lastly, historical evidence is easier to come by when structured this way, as we’ve done below.

Impact is also expressed well as a scenario. However, you (the…

Checking in on our forecasts from “The Big Hack”

An extraordinary claim was published by Bloomberg in October of 2018:

The attack by Chinese spies reached almost 30 U.S. companies, including Amazon and Apple, by compromising America’s technology supply chain, according to extensive interviews with government and corporate sources.

I immediately organized a panel of 22 security professionals to elicit their beliefs about potential future events presented by this journalism. This resulted in a panel estimated belief of 44.82% (YES) that certain events would take place by January 1st 2020. If these events took place, it would largely confirm the article’s claims.

That time window has passed! Happy New…

Applying the types of work from The Phoenix Project to security engineering

The types of work model demonstrated in The Phoenix Project is useful in surfacing and inspecting the work patterns of a security organization. Toil and other frustrating areas of organizational friction are teased out well with this classification method.

An interpretation of these types of work as Security Work puts a security organization under scrutiny as if it were manufacturing workflow. Bottlenecks and imbalances can be discussed where toil is generated. I didn’t expect myself to draw this much usefulness from a fiction novel. But, it works great, and people sometimes know about these types of work from the book.

Driving an awareness project with a risk measurement ethos.

Let’s walk through the steps a fictional security manager takes to pursue a typical security awareness project. We will observe them as they target a risk for mitigation and subject it to risk measurement rather than launching into go-to awareness approaches arbitrarily.

This approach invites a wide range of quantitative and rigorous methods to better decompose a decision to mitigate a risk with the chosen methods.

This person found a risk they want to mitigate.

A security manager is hit with a bolt of insight. Their attention has been drawn to a potential gap at their current company:

Employees aren’t engaging with our security team.

The security team isn’t…

Ryan McGeehan

Writing about risk, security, and startups.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store