How to measure cybersecurity effectiveness — before it’s too late

Are you measuring the value and effectiveness of your cybersecurity efforts? Most companies around the world are failing to do so, according to a recent security measurement index benchmark survey. Without establishing the proper metrics, you’re flying blind.

And even when organizations’ information security function does generate and deliver data about the business’ security, it typically never gets read.

“Many companies, while they’re making some effort in cybersecurity, they’re not looking at the effectiveness in terms of how it helps the business,” says Joseph Carson, chief security scientist at Thycotic, which created its Security Measurement Index (SMI) based on standards for security specified in ISO 27001 and best practices from industry experts and associations. “Many companies are not evaluating their risk versus their impact. They’re not looking at this from a business impact evaluation or perspective. They’re doing it to meet compliance and many of their security metrics were channeled toward that.”

“There’s a lack of collaboration between the two parties,” adds Steve Durbin, managing director of the Information Security Forum (ISF), a nonprofit association that researches and analyzes security and risk management issues. “What is the common language that we should be speaking? How could we, from a security standpoint, be focused on the right things from a business perspective?”

Where you’re failing at measuring cybersecurity effectiveness

Thycotic, a provider of privileged account management (PAM) and endpoint privilege management solutions, surveyed more than 400 global business and security executives to create the SMI benchmark survey. It found that 58 percent of respondents scored a failing grade when evaluating their organization’s efforts to measure their cybersecurity investments and performance against best practices.

The survey also found that while global companies spend more than $100 billion a year on cybersecurity defenses, 32 percent make business decisions and purchase cybersecurity technology blindly. Additionally, more than 80 percent of respondents failed to include business users in making cybersecurity purchase decisions. Nor have they established a steering committee to evaluate the business impact and risks associated with cybersecurity investments.

That jibes with what the ISF sees, according to Durbin. The ISF has found that many CISOs are reporting the wrong key performance indicators (KPIs) and key risk indicators (KRIs). Durbin attributes this to the fact that most CISOs have little or no interaction with the audiences to whom they report. As a result, they are guessing at what their audiences need and miss the mark when attempting to provide ongoing management reporting on topics like information security effectiveness, organizational risk and information security arrangements.

“If I don’t know what you’re doing, how can I help you? I’m going to make some assumptions about what you’re doing and I could be completely wrong,” Durbin says. “Security guys are always talking about cost. If we realign this, the security guys can now go to the business and say, ‘Look, if this is what is important to you, this is the role I can play in helping you protect that, but I don’t have the funding for a variety of reasons.’ The business can then make the call as to whether to find the funding for that problem. It’s no longer the security guy’s problem, it’s the business’s problem.”

While CISOs have to do much of the heavy lifting when it comes to cybersecurity, CIOs also have an important role to play, starting with providing the security function with the data it will need.

“The CIO’s core responsibility is to make sure the organization has the information they need to make the right decisions,” Carson says. “They need to identify what are the core, high-level assets of the organization, classify them. Then work with the CISO to protect them.”

4 steps to KPIs and KRIs

To help security departments align with the business, the ISF has developed a four-phase, practical approach to developing KPIs and KRIs. Durbin says this approach will help the information security function respond proactively to the needs of the business. The key, he says, is to have the right conversations with the right people.

The ISF’s approach was designed to be applied at all levels of an organization and consists of four phases:

Establish relevance by engaging to understand the business context, identify common interests and develop combinations of KPRs and KRIs
Generate insights by engaging to produce, calibrate and interpret KPI/KRI combinations
Create impact by engaging to make recommendations relating to common interests and make decisions about next steps
Learn and improve by engaging to develop learning and improvement plans

At the heart of the ISF’s approach is the idea of engagement. Engagement builds relationships and improves understanding, allowing the security function to better respond to the needs of the business.

Engagement begins with the right data

Engagement starts with establishing relevance. In the ISF’s approach, that means getting the right data, calibrated and supported by the right structures for the right audiences. That data must then be used consistently across the organization. Establishing relevance takes six steps, according to the ISF:

Understand the business context
Identify audiences and collaborators
Determine common interests
Identify the key information security priorities
Design KPI/KRI combinations
Test and confirm KPI/KRI combinations

Once you have the data, you need to generate insight from it. The ISF says reliable insights come from understanding KPIs and KRI. Generating insights involves the following three steps:

Gathering data
Producing and calibrating KPI/KRI combinations
Interpreting KPI/KRI combinations to develop insights

With the insights in hand, it’s time to create impact, ensuring that information is reported and presented in a way that is accepted and understood by all involved. This leads to decision and action, as follows:

Agree to conclusions, proposals and recommendations
Produce reports and presentations
Prepare to present and distribute reports
Present and agree on next steps

The final step is to develop learning and improvement plans based on everything learned from the previous steps. This, according to the ISF’s approach, will lead to informed decisions based on an accurate view of performance and risk, giving organizations assurance that the information security function is responding proactively to priorities and other needs of the business.

“You need to develop a continuous evolution mindset,” Carson says. “It’s a culture, an awareness project. It’s always ongoing.”