This is a method I’ve used to help frame and model cybersecurity risks over the past few years. It helps organize a lot of complexity when dealing with a large organization.
This method uses forecasts, scenarios, multiplication and addition. As all risk modeling goes, this has more to do with the journey of collecting risk knowledge in a rigorous way. We’ll discuss it’s weaknesses at the end.
This walkthrough will use the following scenario:
We disclose ≥1 incidents with ≥$10M damages in 2021.
What future outcome are you looking to investigate? Be confident about your definitions. This example mentions damages and an incident: What damages count? What is an incident, specifically? Disclose where, or to who? …
Risk measurement quickly raises questions about management…. but not about risk management. Rather, managing the performance of people who manage complex risks.
My writing on risk measurement often gets attention from management roles. The management audience desires methods to manage the performance of defenders with risk based measurement. I decided to thoughtfully write out my views on that.
This will only partially read like a cyber security essay. Most management and measurement concepts are not specific to our sector — however, the characteristic of risk makes performance management even trickier. Let’s explore why!
We’ll start with the premise that organizations want to perform well. Some function within an organization must hire, maintain, develop, and lead a team towards a mission or goal. This leads to questions like…
My hope is that the cyber security community will develop as a risk science.
Science starts with correctable claims. Progress towards more useful knowledge come from continuous corrections.
However, a risk hypothesis may represent future events that have never previously happened, might not ever happen, or may not be observed when it does happen. How can we pursue science with an unstable vantage point?
A risk hypothesis conflicts with common notions of science when compared to our expectations of the more mature sciences. …
A topic described as subjective is often considered non-scientific. Risk, being a subjective topic, must certainly be one that science has a hand in. Can we pursue a science of risk with these limitations?
An exploration of subjectivity and risk in a context of science forces us to confront the scientific method itself and it’s relationship to risk. The entire practice of cyber security shows its roots in the scientific method. It also shows how the scientific method is, itself, limited by the subjectivity of its operators.
With this essay, we’ll tie these concepts together towards a more comfortable view of our role in the scientific method as security engineers. …
How do we approximate the amount of resource we allocate for security? In this essay, we’ll cover some principles before the quant.
Let’s start simple:
This approach might sound familiar and reasonable. It’s flawed. We’re now cursed with an organization using a perspective on risk from the late 1800’s that will fund a fledgling security organization.
This essay hopes to provide tools to inspect this gap we’ve created with this mindset.
Insider threats are often discussed as a broad category. This essay explores the malicious flavor of insider threat and ignores the “human error” or “accident” categorizations.
Each scenario is supported with in-the-wild observations. I’ve done my best to select resources that describe the threat’s TTPs or motives as best as possible.
I’ve expressed these risks as scenarios to help focus a group discussion. Narrow mitigations are easier to brainstorm with clear scenario-based requirements. Lastly, historical evidence is easier to come by when structured this way, as we’ve done below.
Impact is also expressed well as a scenario. However, you (the reader) will likely have their own impacts that concern them based on what they protect. …
An extraordinary claim was published by Bloomberg in October of 2018:
The attack by Chinese spies reached almost 30 U.S. companies, including Amazon and Apple, by compromising America’s technology supply chain, according to extensive interviews with government and corporate sources.
I immediately organized a panel of 22 security professionals to elicit their beliefs about potential future events presented by this journalism. This resulted in a panel estimated belief of
44.82% (YES) that certain events would take place by January 1st 2020. If these events took place, it would largely confirm the article’s claims.
That time window has passed! Happy New Year. …
The types of work model demonstrated in The Phoenix Project is useful in surfacing and inspecting the work patterns of a security organization. Toil and other frustrating areas of organizational friction are teased out well with this classification method.
An interpretation of these types of work as Security Work puts a security organization under scrutiny as if it were manufacturing workflow. Bottlenecks and imbalances can be discussed where toil is generated. I didn’t expect myself to draw this much usefulness from a fiction novel. But, it works great, and people sometimes know about these types of work from the book.
This model has a narrow purpose. …
Let’s walk through the steps a fictional security manager takes to pursue a typical security awareness project. We will observe them as they target a risk for mitigation and subject it to risk measurement rather than launching into go-to awareness approaches arbitrarily.
This approach invites a wide range of quantitative and rigorous methods to better decompose a decision to mitigate a risk with the chosen methods.
A security manager is hit with a bolt of insight. Their attention has been drawn to a potential gap at their current company:
Employees aren’t engaging with our security team.
The security team isn’t looped into discussions, receiving escalations, or asked to consult on tasks as frequently as the manager would expect. …
I’ve been helping a few security engineering organizations in the Bay Area experiment with quantifiable risk modeling approaches that use clear language. We’re doing this to subject security teams to better measurement beyond (or in addition to) compliance, checklists, grades, color coding, or maturity models.
It’s difficult to unify broad security work with disparate disciplines under a single quantitative key performance indicator (KPI) that addresses rarely occurring and high impact cybersecurity risks.
We will discuss the potential of probabilistic, risk aware KPIs seeing experimentation at a few large tech companies. First, some background:
The purpose of KPIs is to point a group towards a single direction. They will do this in addition to any mission statements and objectives and perhaps incentivize efforts that see positive gains in that measurement. …