Cyber Security Performance Management

All businesses, job roles, departments and functions require performance management.  Security is no different and this is now being seen as critical within the compliance and audit functions as part of the management of cyber risk.

Management Systems

Security management processes typically revolve around understanding the level of spend, the resources deployed, the rate of alerts or incidents or service level measures around the length of the queue of security issues.  They also often relate to mapping of spreadsheets containing compliance standards to the perceived reality of the network – with non-compliance and policy non-conformity being tracked.

These are all security performance measures – details about response times and levels of coverage of the management system itself.  They can be high-level, they are often assessed manually and they can be infrequent.  More likely than not, they are where security reporting starts.

Technical Assessments

At the other end of the scale, are more technical assessments of security.  Driven by a range of techniques such as vulnerability scanning or penetration testing, these aim to find all of the technical issues at the most granular level; lists of vulnerabilities or mis-configurations of servers, details of missing patches and users or datasets with the wrong access rights.

The challenge with these is often that there are a high number of findings (depending on the day of the assessment) and that for each individual “issue” the risk might not be obvious as the attacker will make use of a sequence of smaller technical issues to compromise a network.  Some might appear benign, but affect critical servers or impact sensitive data sets, others might seem serious but might be little more than false positives.

As a way of creating a master “to do” list of individual items this has some value, but the volume or distribution of issues can be misleading (in particular if “known” issues are deliberately excluded because the business is already aware of them – even if this has been the case for some time).

Don’t trust to luck

The danger with these approaches is that they are often labour-intensive in terms of either the generation of the results (in the first case) or in dealing with the numbers of them (in the second).  This effort overhead can mask issues or make things appear better or worse than reality.

In security there is a danger that over-exaggerating problems makes the performance of a security function look bad (and hence less credible when asking for increases in budgets to address growing numbers of issues).

Likewise, reporting based on the numbers of policies or controls in place might mean that the effectiveness of processes and mechanisms designed to stop breaches is assumed to be higher than it is.

A “lucky” business might avoid breaches even if controls are missing or if they are poorly implemented and ineffective.  But being lucky isn’t a realistic way to handle one of the biggest risks facing the business.  An unlucky one will not avoid the risks and could face a big fine.

Continuous Security Control Measurement

If we consider what these approaches lack it is relatively easy to formulate a better way to track security performance.  Here is a list of widely documented perspectives from cyber security performance managers:

• We have growing reliance on technology, increasing regulatory pressures and complex supply chains
  We need an approach that will check the controls we have in place directly, rather from their (technical) effects or from their (policy) intentions.
  This implies a close connection to the environment itself – one not hampered by being launched externally from an Internet service or being discussed over spreadsheets of audit findings in a security management meeting.
• We see new vulnerabilities, new malware and new threats come out daily and take seconds to exploit
  We need a way to continuously monitor the state of controls, the cyber security risk posture if you will. One that isn’t annual, or quarterly, or monthly but is trusted to be happening all the time and reporting or flagging problems as soon as they are apparent or detected.
• The complexity of cyber risks is hard, even for a technical audience, to understand – the nature of an attack, the vulnerability, the level of data loss and the reputational damage and cost
  We need a way to create outputs, KPIs, metrics and performance indicators that show a clear view of cyber risk status or posture that can be understood by a range of audiences. These must be understandable as good or bad, and as improving or worsening.
• We know that people are subjective, possibly unaware of the real situation, prone to exaggerate or assume the best (or worst)
  We need a way to measure security that is objective and reduces the reliance on human intervention, interpretation, data analysis or summarisation.

As such, the future of security performance management MUST make greater use of systematic, direct, and more frequent or continuous measurement of security control configuration and performance.  From inspection, comparison of today/yesterday or actual settings versus “known good” states and using records of activity to build trust in the outputs.

Cyber Security Performance Tracking and Reporting

This is a burgeoning area of technology, called variously RegTech (especially if there is a compliance angle), Computer Assisted Audit Technology (CAAT) or Robotic Process Automation – RPA (if it has been driven by an excited marketing function).  Whatever it is called, the need for business to invest in technology that can better answer and report on these questions is self-evident and clear.

There simply isn’t a human-based way to provide the quality, accuracy and frequency of reporting on security performance that business managers require.