Valuation of non-monetary penalties
Let’s price out a non-monetary cyber security impact.
You might not be hit with a fine after a regulatory body investigates your cyber security issue. Instead, they may impose rules you must follow.
What is the monetary impact of a non-monetary penalty?
Let’s take a stab at figuring that out that. This is a useful exercise to approximate a narrow area of the overall stakes of a security or privacy organization has to consider.
We’ll decompose the Snapchat FTC settlement from 2014 and approximate its overall impact ($7.4M–$54M). We’ll follow that analysis to further approximate the “willingness to pay” value for the point in time impact ($7M).
Note! Snap’s name was Snapchat at the time and will be written as such.
Our toolkit includes…
We’ll be relying on several economic concepts. Willingness to Pay, Indifference Price, Contingent Valuation, and the Reservation Price. They’re similar in concept but have different bodies of research to draw from. They ask:
What is the maximum price you would pay for X?
This area of impact estimation was popularized when it was used in the Exxon Valdez Oil Spill.
There’s lots of academic scrutiny over this implementation of contingent valuation in natural disasters. The rabbit hole goes deep until you begin questioning the concept of “value” itself. This method will have problems, but every model does.
It’s important to remember that this is an approximation method. The only bar we need to pass is one of usefulness. That’s up to you!
Let’s decompose the overall impact of the settlement.
The impact from this FTC settlement comes in several forms: Regulations, legal costs, reputation, and revenue.
Regulations: The consent order dictates rules it imposed on Snapchat. The details are in the order, but summarized here:
- Build a robust privacy program with ongoing improvements.
- Regularly report privacy assessments to the FTC for 20 years.
- Make privacy related statements to/from customers available for 5 years.
- Report on all corporate transitions (M&A) which may impact compliance.
- Respond within 10 days whenever the FTC requests anything.
Reputation: There was a negative press cycle from the settlement. It was the first publicized knowledge that the FTC was even investigating Snapchat.
- In May 2014, there was a large media cycle when the settlement occurred.
- In December 2014 there was a minor media cycle when the settlement finalized.
- There’s also a blog post in response to the settlement which doesn’t appear to be discussed much anymore but was cited in the press.
- There is organized data on the FTC’s website about the investigation.
Legal Costs: It is likely that internal and external legal resources were used for the non-public, pre-complaint investigation leading to the settlement.
There doesn’t seem to be evidence that any discovery against Snapchat occurred. The settlement may have occurred to avoid the costs associated with wholesale litigation by the FTC.
Revenue: The press around the FTC regulation event was sharp and quick. The larger influence to user engagement likely came from the media cycles leading to the FTC investigation, not the settlement itself. The media cycles preempting the investigation were far louder that the settlement itself.
How much of this impact was easy to accept?
Snapchat was already building out their privacy and security program before the FTC came around. I’ll consider everything they were already destined to build a “desirable” regulation. This redundancy of work was a topic also mentioned in their blog post and should be viewed as a positive.
We have to assume there is some overlap between voluntary improvements and involuntary improvements.
Example: If the FTC forced me to eat breakfast every morning, I wouldn’t really count that as a cost. That’s not impact. I love breakfast! That’s desirable regulation.
Much of the settlement is likely to be desirable. Our focus should be on areas of the settlement that were not.
Line items of “undesirable” impact
I need to focus on what is undesirable about the settlement if I want to approximate its costs.
All estimates are subjective 90% credible intervals
Auditing: The imposed audits are hopefully useful and re-purposed with what self imposed auditing would already be happening if the FTC settlement never occurred. The most undesirable case is if these audits are wholly redundant with other security efforts and not useful in actual risk mitigation for the business. My interpretation of the settlement counts 11 unique audits over 20 years (Initial / Bi-annual). The settlement requires them to be external which will cost full consulting fees ($150–$600) and I imagine 2–6 consultants working 2–5 weeks (160hr-1200hr), giving me an interval of $24,000–$720,000 per audit.
A $600 fee is high for auditing — but I can’t assume that price gouging isn’t happening due to the FTC’s involvement or if they’re using a specialized legal firm with FTC experience. I am also leaving the $150 number in case they got a long term engagement and some kind of deal, if they found a way to make these audits minimal and efficient, or if they’re redundant with audits that would have occurred anyway.
$264,000–$7,920,000
Compliance Employee Time: FTC penalties are not a normal compliance area for any organization. It’s additional motions that need to be gone through. Employees likely need to manage these requirements in product development, for instance, shipping off privacy language to the FTC. In aggregate employee time, I’ll estimate it’s at least half a headcount (salary and benefits) to one full headcount annually for 20 years. ($250k * .5 * 20to $600k * 1 * 20).
$2.5M-$12M
Reputation: The settlement announcement looks like a very tolerable press event in hindsight. In Q1 2014, Snapchat had 46 Million active users and had a quarterly average revenue per user of less than 5 cents (SEC).
Considering that the FTC press was only a minor news cycle in the overall topics covered I think we’re working with extremely small numbers of immediate engagement losses influenced by the FTC case that would be less than $100k.
I do think that the FTC settlement put more of a spotlight on Snapchat and this increases the probability of separate regulation. Maryland, for instance, regulated Snapchat shortly afterward. I have to approximate the expected value of regulation efforts going forward that might be encouraged by this settlement. I imagine a small probability increase in all possible regulatory impacts going forward with our impact being the expectation value change across the board.
$100k-$2M
Revenue: I think the additional governance brought to the company has significant downstream effects on product’s ability to move quickly. Experiments and A/B testing will slow down any time privacy language is involved.
According to the settlement: any time a privacy statement is made, the FTC is CC’d.
This seems like a possibly large tax on product development that is highly undesirable from a product development standpoint. Under normal circumstances, these sorts of decisions would be made at Snapchat’s discretion with a more generalized concern for privacy and security. This adds a manual “check in” that normally wouldn’t exist.
If Snapchat found a way to minimize the interpretation of the rule or somehow make it efficient, then maybe it’s not such a big deal. I have a lot of uncertainty related to this because product velocity has a direct impact on revenue. This space is limited to only 5 years of regulation, not 20, like the other aspects of their regulation. This puts a smaller time frame around the uncertainties here.
$1M-$15M
Privacy and Security Organization: Security awareness training was likely happening or would have eventually. The settlement… required the settlement requirements (I know, it’s recursive!) to be included in the awareness training of employees. This adds a few minutes to all training material for twenty years that is specific to the regulation, including any software to implement, etc.
Other than that — I believe a lot of the FTC settlement requirements would have manifested themselves in some form, and we’re measuring the tax that being regulated would have on implementation.
I believe the extra workload could be abstracted as employees if you were aggregate the work. Same estimation as the compliance org.
$2.5M-$12M
M&A and Business Activity: It’s specifically mentioned that the FTC order be delivered to anyone involved in a corporate transaction. This likely ended up in the documents before IPO, potential acquirers, or acquisitions. A pain in the butt talking point and formality for corporate transactions. Was likely a minor dent, if any at all, on the overall valuation of the company which would have rewarded investors and employees later on at IPO.
$1M-$5M
Total estimated impact over 20 years
My total estimation of impact over 20 years is looking like $7.4M–$54M.
There are lots of issues with this that are exacerbated given the span of time and uncertainties associated. A valuation of impact on May 8th, 2014, is certainly not $54M, even if that’s eventually correct.
For example: The valuation of a startup has a huge discount in anticipation of its future value and the risks on the way. Similarly, this impact is not reasonable to consider as the “day of” impact at time of settlement.
Willingness to pay is our next step.
If the FTC offered Snapchat an alternative monetary settlement option on May 8th, 2014… how much would it cost?
We can reason through this with the approximations we’ve done. It’s important to consider that Snapchat had only raised $150M with minor revenue to show at this point, and capital is valuable resource.
The method we’ll use requires the suspension of disbelief for a couple reasons. First, we’ll need to approximate a reserve price for Snapchat to go into a settlement discussion (which, to our knowledge, was not offered) by the FTC. Second, it would also be bad optics to try and settle with cash. We’re not trying to avoid regulation, we’re just trying to manage the business impact, inform good decisions, and mitigate harm to users in the future.
We can start with absurd numbers. The FTC would obviously say no to zero, and Snapchat would rather follow regulation than give them all of their money.
There’s a reasonable middle ground and we have to find it.
It would take precious capital away from Snapchat’s investment in their business as we raise a reserve price. That pulls from runway hoping to validate a business model. They could raise outside money to cover a settlement (They did, raising $485M later in the year) but that would dilute the business for something that isn’t yet proven to generate sustainable revenue. This is the primary downward pressure on “willingness to pay” being a lower figure.
My instinct is to look at the non-monetary settlement as a tax on later growth as opposed to the immediate availability of funds to prove a business model.
We need to find a monetary value where Snapchat might become indifferent to either accepting regulation or paying a flat fee to settle instead.
Going even a single penny over the optimal monetary value should switch a decision to accepting regulation instead.
Approximating the optimal monetary settlement
My focus goes toward approximating more immediate impact from the settlement in the short term. I narrow in on a focused amount of Business Activity, a year of Revenue impact on product development, a couple years of undisturbed work by the Privacy and Security Organization, some and removing distractions from the first couple years of Audits are what would make up the decision.
If Snapchat were to succeed as a business, it would be capable of facing the more extensive costs longer term. A good problem to have. I’ll try to divide these with this tactic in mind.
I dropped these intervals into a dead simple monte carlo simulation. If I were to produce a “Willing to Pay” monetary settlement on May 8th 2014, I would pick $7 million, which was the 90% percentile of the simulation output. It also happens to be close to the minimum end the “total” interval I approximated earlier.
Any value greater than this and I suspect they might start considering the cost of a monetary penalty to be worse for the business than regulation. That’s the optimal point we are looking for to approximate how impactful this breach was in dollar value.
Conclusion: What does this help?
We’ve approximated a multi-million dollar impact from a single risk scenario that didn’t explicitly have any monetary impact.
Security organizations worry about many different scenarios and their probability of occuring. Analysis and understanding these impacts is an important aspect of measurement if we’re to make progress as an industry towards efficient security mitigation.
Multiple impact measurements can result in substantial amounts of expected risk in aggregate, but doesn’t compose the entire picture of why we do security.
A total aggregate of corporate risk will not entirely justify a security organization if it only measures the risk to itself. An organization must still prove that it is worthy of trust, which is willingness to delegate one’s risk to us. I think the concept of imposed risk can also be quantified and is additionally beneficial in justifying a security organization.
Hopefully I can write about that soon. There’s a lot that can be done.
Footnote: This model has problems! (as all do!)
I subscribe to “all models are wrong” advice and I’d probably agree with criticism of any methods aboved. Approximation methods are “always wrong” anyway.
There are some obvious issues with this model and it’s helpful to discuss them to reap the maximum usefulness from an approximation effort.
Some of these problems are covered in academic research. Often surfaced and discussed are the cognitive burdens on human prediction. I cover that subject often in other essays and won’t cover here. Forecasting approaches hope to minimize that error with calibration, training, practice, and diverse panels.
Protest values: Research into this field finds individuals proposing irrational contingent values on surveys. For instance, “I will never avoid regulation!” with a forecast of zero or infinite dollars. Given that this is not an ethical exercise, it’s an economic one, exercising the right mindset can help capture the value of consequential outcomes that ethical organizations can optimize reductions of risk with. If we’re capable of “thinking like criminals” in threat modeling, we can certainly think with economic approaches as well when discussing incident impact. The approach we describe relaxes the problem a bit since it’s not an open survey, but it does force us to rely on good faith approximations from a small group quite a bit.
Ability to pay: Willingness to pay is also tied to ability to pay. This was pronounced in this exercise. If Snapchat had a larger amount of disposable funds in the bank, it likely would have valued a monetary settlement even higher. The option to remove regulation in hindsight may eventually become valued far higher and confuse the “Willingness” discussion even further.
(In)Ability to pay: There’s an alternative, pessimistic angle: If the company wasn’t doing well… they might not be able to afford derailing any funds to a monetary settlement. In this case the reserve price would be absurdly small and indicate a death blow to the company.
Price Discovery: There is no market for settlements and this is a boutique situation applicable to only one party. so we are limited to setting a reserve price and opining about the utility of the monetary settlement. This goes down the rabbit hole on what “value” actually means and is wholly dependent on perspective. The FTC is not negotiating or looking to profit from the deal and it’s important to remember this fictional aspect of the model.
Risk of Violating the Settlement: We could continue this analysis and pull in expected risk of violating the non-monetary agreement. In fact, there’s news of this exact situation happening with Facebook right now. Similar has happened at Google. We’d have to similar decompose the likelihood of this agreement being violated over 20 years, and what the follow up penalty could look like. For essay brevity, I’m excluding it, but it’s entirely do-able.
Ryan McGeehan writes about security on scrty.io
