Data collection and evidence gathering made easy

As businesses face continued disruption, cyber threats persist, as does the ongoing need to protect data. For consultants, travel to sites to conduct security audits and maturity assessments has been constrained, making traditional service delivery methods impractical. 

Huntsman Security’s automated security audit and assessment tools, Essential 8 Auditor and SmartCheck for Ransomware systematically undertake the data collection and evidence-gathering elements of the security audit process with a single click. They also have the functionality to save audit and assessment definitions, enabling quick and easy like-for-like audits.


How Huntsman Security solutions help

Audit and assessment processes involve several steps and the measurement of a number of ever-changing variables. Depending on the focus of an audit and the goal, the shape of the tasks might be different.

The way an assessment is performed, the depth and breadth of evidence gathering and the analysis and reporting, need to accurately reflect an organisation’s risk exposure. A typical audit process will involve these five stages:

Security Audit Process


Five areas where security review processes are transformed

Stage 1 – Planning, Scoping and Logistics

Huntsman Security solutions reduce site visits and travel. They also increase the breadth of the review scope.

The planning part of a security review will define the objectives, scope, work activities and logistics. This is where a balance has to be drawn between coverage and budget, and how best to efficiently deliver the outcomes. Decisions will need to be made on:

  1. The scope: What parts of the environment, what controls, the balance between technology, policies and processes and what activities will be involved.
  2. The methodology: The combination of manual assessments, onsite visits, interviews, questionnaires, audit tools and data/evidence requirements.
  3. Scheduling: Including the assignment of resources and the planning of site visits, inspections, meetings and the associated travel.
  4. Planning for the impact: On any operational systems and the involvement and collaboration with operational personnel.

If assurance in the outputs can be gained with reduced site visits then the cost in time and travel goes down, and if more accurate ways of collecting data can be used then the subjectivity and vagaries of interviews and questionnaires can be avoided. If the most relevant and highest impact controls can be prioritised, then the scope of coverage could be broader for an equivalent amount of budget/effort.

The other risk in the planning of any security review is the potential for selection bias – deciding to audit certain systems or sites because they are easier or because they are known to have data available. For example, choosing to conduct an audit externally with scanning tools because going to site is more expensive, or choosing controls that are well managed and generate detailed metrics instead of ones where metrics and records are harder to obtain.

The deciding factors here are the tools and facilities available as they will drive what site attendance is required, what data can be obtained, and how wide a scope can be covered for a given set of controls.

Download the Guide – 5 Ways to Improve Security Assessments


Stage 2 – Data Collection and Evidence Gathering

Huntsman Security solutions systematically gather review data and evidence of successes and failures.

The collection of data, metrics and audit evidence, the “fieldwork”, is the most resource intensive phase of the review process. It’s the easiest one to make mistakes in, omit key facts or rely on assumptions that are not really valid. The data should be gathered across as wide a scope as possible, ideally the entire environment. Manual approaches often rely on sampling and this can introduce biases or omissions.

Types of fieldwork


One obvious goal is to optimise the amount of labour involved in doing this; by minimising the time spent querying systems, gathering data and creating reports or spreadsheet extracts. This means that more systems and controls can be covered. Doing this programmatically also means the data collection can be done by less experienced or non-expert staff, or by local IT administrators. Taking data direct from the source also removes interpretation and subjectivity from the collection process around “how compliant” or “how well implemented” a control or system is.

Expert analysis of the collected data then benefits from better, more accurate and comprehensive inputs and less reliance on or overhead from travelling, collecting, collating and extracting – hence more time to understand and draw conclusions or examine areas of the cyber defence posture that are not embedded in technology.

Essential 8 Auditor and SmartCheck for Ransomware:

1. Confirm the presence of controls or the implementation of policies
2. Gather evidence and records of their operation or execution
3. Identify weaknesses, failures or omissions
4. Pinpoint failures and enables the analysis process to link these to identifiable root causes

Stage 3 – Analysis and Interpretation

Huntsman Security solutions accurately and consistently calculate results, analyse objectively and identify patterns. 

The analysis of collected control performance data can be split into two elements:

Mathematical or Statistical analysis

Using a large data set covering a large number of systems, controls and degrees of implementation requires some of the analysis that is best undertaken by computers. It is important that the toolset makes the calculations (i.e. produces outputs), rather than allowing the consultant or auditor to make the calculations (much as a spreadsheet would).

If a set of results will benefit from having a count of those values that meet certain criteria, or the minimum/maximum being derived, or the average – or the percentage equivalents of these – then the data collection/analysis process can derive this and feed it into the outputs. This will avoid someone having to manually operate a tool to filter the data into subsets, calculate totals and pivot them to derive the data representation they need.

Interpretation and expert analysis

The meaning of control failures, the omissions of implementation, the numbers of affected systems or degrees of compliance are all things that consultants, auditors or security experts, often with a view of the wider context around IT operations or business objectives, can best understand. This is where the consultant or auditor interprets the data (raw and processed) to derive the findings, insights and recommendations that the customer or end-user needs.

The performance metrics that result from the analysis of data from controls can be used to provide a view of cyber posture or residual risk and security operational performance that an auditor, manager or consultant can put into context for the business to use in decision-making.

The derivation of performance metrics from data through calculation can be automated so that the expert can focus on understanding what these actually mean, rather than trying to derive them in the first place.

4 – Reporting

Huntsman Security solutions enable you to distribute performance metrics to all stakeholders easily.

Every consultant, auditor and expert will have a way to report and present the findings and outputs of their work and the interpretation they have made as to the state of the cyber risk strategies that the controls have enforced.

Analysis and reporting are the culmination of the report author’s wisdom and experience, and it must be attuned to the audience for which it is intended.

Managers and risk owners need to understand the coverage, effectiveness and maturity of the controls that are in place, and the risks that their business or data is exposed to. The graphical representation of KPIs and the ability to understand how this translates into a maturity score that combines quality, coverage and effectiveness is vital. This audience doesn’t want lists of issues or pages of detailed analysis, they want business focused performance metrics and dashboard type outputs that show them the current posture and the trends over time, to help them understand if it is getting better or worse.

Those tasked with remediation activities or the prioritisation of improvement projects have a different output requirement, they need to know the specifics of which systems, controls or failures were detected so they can group these together to resolve them or put in place work orders or change controls to address the issues. This form of output will often be the input to other data sets or systems; therefore the appearance matters less, in fact the rawer the data is the better.

Huntsman Security solutions present easy to understand KPIs and outputs in dashboard form and a view of maturity that gives business unit owners, risk and compliance teams and the C-suite the information they need, at a glance, to understand the risks and the performance of the security function.

The technology can generate raw outputs for technical use, or for other systems that need lists of systems or control failures to be remediated. 

5 – Remediation

Huntsman Security solutions automatically verify that key issues have been addressed. 

Remediation is not a part of the security audit process itself, but it is a key element of the business improvement cycle that the audit initiates. Once findings have been made the relevant operations or technical functions must prioritise and resolve the issues.

This may be fixing point problems, investing in technology or addressing root causes so that expected controls and safeguards operate as intended.

The important part of this process is the re-evaluation and reassessment of the controls’ effectiveness and performance metrics AFTER the work has been conducted.

Here the use of technology and an optimised audit process is particularly useful. If the checking of the previous audit findings necessitates a complete rerun of the process – the site visits, field work, recalculation and reporting – it is unlikely to be viable. You are effectively doing the audit twice.

Where technology has been put in place to collect and gather data, to analyse it and present the identified issues and performance metrics and control maturity, then it becomes a much simpler exercise to rerun.

A full review with root cause analysis is unlikely to be needed in this instance, as the new and improved scores and the greatly reduced lists of identified problems, are sufficient output to show improvement.

A retest or follow-up check should be part of the planning of any review.

Get results fast

Essential 8 Auditor and SmartCheck for Ransomware can be deployed on site or on a remote network, without the need for a consultant or audit and risk manager to visit that site. No security expertise is needed to deploy the solutions; they can be installed and run by a network or system administrator locally. The gathered data can be reviewed off-line and extracted for additional analysis or report preparation.

Download Essential 8 Auditor BrochureDownload SmartCheck for Ransomware Brochure

Want to find out more?

If you’d like to request a demo or ask a question, please contact us by clicking on the button below.


Read by directors, executives, and security professionals globally, operating in the most complex of security environments.