Our cyber security products span from our next gen SIEM used in the most secure government and critical infrastructure environments, to automated cyber risk reporting applications for commercial and government organisations of all sizes.
The Financial Reporting Council (FRC) is responsible for corporate governance, reporting and audit in the UK and has been consulting on the role of technology in audit processes.
On the one hand there is a recognition that technology can assist audits, providing the ability to automate data gathering or assessment to increase quality, remove subjectivity and make the process more trustworthy and consistent.
Both the Brydon review and the latest AQR thematic posit a link between enhanced audit quality and the increasing use of technology. This goes beyond efficiency gains from process automation and relates, in part, to the larger volume of data and evidence which can be extracted from an audited entity and the sophistication of the tools available to interrogate it.
As one example, the PCAOB in the US has for a while advocated for the provision of audit evidence and reports to be timely (which implies computerisation and automation) to assure that risks are being managed, and for the extent of human interaction with evidence or source data to be reflected to ensure influence is minimised (the more that can be achieved programmatically and objectively the better).
However, technology may obscure the nature of analysis and decision making and hence create a barrier to fully transparent audits compared to more manual (yet labour intensive) processes. There is also a competition aspect between larger firms and smaller ones as regards access to technology:
Brydon raised concerns about the ability of challenger firms to keep pace with the Big Four firms in the deployment of innovative new technology.
The discussion document and call for views (which has now passed) can be found here.
The consultation paper covers issues, and asks questions, in a number of areas. Examples include:
Clearly these are real issues – for a process that aims to provide trustworthy, objective, transparent and repeatable outputs – any use of technology to speed up or improve the process must maintain these standards.
The cyber security realm has grown to quickly become a major area of risk and hence a focus for boards, technologists and auditors alike.
The highly technical nature of threats and the adversarial nature of cybers attackers (who will actively try and find/exploit control failures) means that technology solutions that identify weaknesses and report on specific or overall vulnerabilities are becoming more entrenched in the assurance process within this discipline.
While the audit consultations and reports mentioned above cover the wider audit spectrum, similar challenges relate to cyber security as an inherently technology-focussed area of operation.
Huntsman Security found similar challenges when developing its Essential 8 Auditor product through discussions with customers and the national audit bodies in the territories it operates in, as well as in conversations with leading industry analysts on the wider GRC technology landscape.
The gains from using technology to conduct data gathering, analysis and reporting are obvious – removing the need for human questionnaires, interviews, inspections and manual number crunching. Increasing the speed of the process has a number of benefits:
The ability to conduct audits across different sites or scopes, to specify different thresholds of risk for different domains, the ease of conducting audits at remote locations or on suppliers networks (especially during period of restricted travel) are ALL factors that can make technology a useful tool for the auditor.
Different parts of an organisation might have varying risk thresholds for specific controls
One part of the FRC’s perceived problem space is that of transparency, you can ask a human how they derived a result, and they can probably tell you, or at least show you the audit trail of correspondence, meeting notes or spreadsheet calculations. But can you do this with software or technology?
Certainly, the use of AI and machine learning makes this hard, the learning nature and often black box calculations are not easy to either understand, recalculate in a repeatable way or to document. The system learns, so is always changing, and hence the rationale that a decision might not always be the same.
In technologies that are geared towards delivering audit outcomes this is easier. First, if you collect and retain data, provide an easy interface to go from results to the underlying cases in the source data, it is possible to take a score/rating/risk and reveal the specifics of what led to it. Secondly, it is vital that the calculations are transparent, i.e. that the methods of calculating risks or the way results are scored is decipherable.
Essential 8 Auditor – Application Control Score calculation screen
This is one obvious gain from technology, the logic is pre-programmed in. If you take two auditors and give them the same data sets or evidence case files they might draw different conclusions (possibly for valid reasons or due to them having different skill areas or experience), but the same algorithm operating on the same data will produce the same result every time.
Manual evidence gathering suffers a number of drawbacks – it relies on written notes, records of verbal conversations, email trails, spreadsheets, or questionnaire responses in different formats. Retaining all this in a coherent way is difficult and going back through it even harder.
Using a consistent toolset and consistent data format means that if you need to go back to a data source from a particular network domain three months ago, you will have information that is readily available and readable. And as stated above, if the source data and evidence is re-examined using a consistent solution, you will get the same calculations, decisions and results.
The outputs of any audit process need to provide details of the issues found so that the specific or general cases of the failures can be investigated and resolved. But for managers, operational teams and businesses, having a view of the KPIs for the security operations process is extremely useful.
Of course, following the “lines of defence” model, an internal or external “formal” audit might simply want the results and a level of trust in how they were calculated; however for operational management and ongoing continuous visibility, the need to derive performance statistics comes into its own.
It is worth noting that there are two dimensions to KPIs: The assessment of the strength or configuration of a control or policy (how good is the control) and the extent or level of coverage (how widely is it enforced).
To give a view of the technical maturity of a defence you really need to combine these two factors together. A weak control that is widely implemented or a strong control that provides only partial coverage are both causes for concern.
Essential 8 Auditor – Dashboard showing Application Control technical maturity calculation
The final area where technology can help is in allowing the separation and distribution of the data gathering, analysis and reporting processes. It is hard to take the data, evidence and meeting notes from someone else and analyse it. For one thing, is it trustworthy and reliable (in the case of third-party assurance questionnaires perhaps)? Then it is also hard to draw high-level conclusions about the analysis.
If technology allows the data gathering to be performed in a distributed way, say by local site administrators, third-party IT staff or non-expert users BUT in a trustworthy way, then the overhead of the audit process is much reduced. Instead of a team having to conduct multiple visits, interviews or data collection activities the toolset can be provided to the people nearest to the point of collection.
This allows the data analysis and interpretation to be performed centrally by the experts in a particular field or control area. So giving a non-expert user a way to collect and provide relevant and trustworthy audit evidence takes a large bite out of the resource overhead of conducting the audit, for both auditor and auditee.
It also means that a target organisation doesn’t have to manage the issue of allowing auditors to have access to networks, sites, data, accounts and systems to gather the audit evidence as this can be undertaken by existing administrators in the environment.
Technology solutions in the audit process can clearly deliver benefits, however if they are too simplistic or aim to be too clever, they can simply move the problem of providing high levels of audit quality. A rapidly generated AI-based risk score is useful, but if it’s not possible to understand the calculation it is hard to either correct the control issues or trouble shoot the underlying process.
Where technology can assist the audit process, speed up data gathering and analysis, and streamline the generation of high- and low-level outputs it can be a boon.
Technology allows organisations to put trustworthy assurance into the hands of operations teams and managers, consultants and auditors alike to provide flexible, rapid and frequent views of control data and understanding of risk posture. If this can be done in a way that is cognisant of the risks and challenges as we have shown, then auditors and regulators such as the FRC can be satisfied.
<<< Part 2a: Australia’s Essential Eight: Beyond Endpoint Control <<< Part 2b: Activating UK NCSC & US NIST Guidelines: Beyond Endpoint Control Part 4: Systematic Measurement of Cyber Controls >>> As much as we invest into cyber security controls, external threats are inevitable. In a recent Notifiable Data Breaches Report from the Office of the […]Read more
Keen campers, scouts and even the Swiss Army know – that a good penknife is indispensable. This simple device has mitigated many a disaster at one point in time or another. Whether it’s to cut through a bit of string, tighten a screw or simply to solve the problem of no bottle opener in the […]Read more
Supply chain risk is an area of cyber security that demands the ongoing attention of every enterprise; because it can make the difference between being resilient or not. It’s no surprise that insurers warn that the vulnerability of supply chains is potentially a systemic risk that can quickly propagate across supply chain dominated industries. Organisations […]Read more
It took a “tripartite cyber assessment” by the Australian Prudential Regulation Authority (APRA) to identify that a sample of financial organisations had inadequate cyber security: poor security control management, a lack of business recovery planning and inadequate 3rd party risk assessment. Why were there gaps? Where is the failure? Clearly the common practice of unsubstantiated […]Read more
The discussion over data-driven vs qualitative cyber security assessment has been going for some time. Nowadays, it is at the top of the priority list for many security and senior executive teams. Managing cyber security has always been a noble ambition but without reliable measurement, the lack of actionable information makes evidence-based management decisions almost […]Read more
Attack Surface Management (ASM) characterises a business’s security risks as the monitoring and risk mitigation of a constantly changing and vulnerable “risk-surface”. Importantly, this attack surface extends to both internal and external assets and services. Some ASM solutions deliver clear visibility across both Internet facing and internal assets. Others do not. Instead, they assess external […]Read more
The UK Government has released its annual “Cyber Security Breaches Survey 2023”. It provides some valuable insights into how cyber security is currently being managed in the UK, by a range of organisations. It also speaks to how current competing economic priorities are impacting the effectiveness of some cyber security management efforts. The full report […]Read more
Solving the mismatch between cyber security reporting and directors’ requirements You are undoubtedly familiar with the headlines; you may have even become in part desensitised to them: ‘Cyber-attacks are increasingly damaging’, or ‘large amounts of personal data are most at risk’. The important take-away, however, is that modern day thieves can easily gain access to […]Read more
A system to address the untrustworthy security environment Zero trust approaches to security have been talked about for a while; but in recent times they have certainly gained more currency. As a model for protecting data and services, the simplicity of the concept is its biggest strength – assume, as a default position, there is […]Read more
The ongoing protection of Critical Infrastructure from cyber-attacks has implications for us all – whether it’s supporting our health, well-being or simply our way of life, there is good reason to reflect on the effectiveness your cyber security. Cyber security risks are nothing new and the vulnerability of critical infrastructure to them (and the heightened […]Read more
Read by directors, executives, and security professionals globally, operating in the most complex of security environments.