Security leaders told: test cyber confidence, stop buying isolated tools

The Rapid7 executive, writing about purple teaming exercises, said he has repeatedly seen gaps that nobody expected in organizations that invested heavily in their security stack: missing telemetry, misfiring detections, and attack paths nobody was watching. "The problem was never simply being under attack. It is being under-validated, and mistaking spend for assurance," he wrote. He described a common pattern in which every control checks out on paper and leadership is reassured, but cracks show almost immediately when a real attack runs through the environment. Some systems feed logs in real time while others barely do; detection rules are tuned to a generic attacker; and triggered alerts can sit unactioned for hours.

To address that, the Rapid7 executive calls for treating validation as an ongoing discipline rather than a one-off exercise. The output should be a living, prioritized backlog of gaps that are proven to matter, ranked against paths an attacker could realistically use. He also warned against relying purely on agent-based validation, because an agent embedded in the network "inherently shortcuts a lot of attack paths and bypasses defensive controls" and assumes the attacker has already achieved a foothold at that location. Shifting from ad hoc tests to an operationalized cycle of testing, validating, remediating, and retesting keeps the backlog tied to the environment as it is now, he added.

The Illumio executive argued that CISOs often have a clear answer for why they bought a specific security tool — it stops a technique, closes a gap, or satisfies compliance — but it is less clear how the tool fits into the rest of the stack. Research cited in the article shows that a majority of security professionals say they juggle too many tools, and more than half say those tools do not properly integrate together. He called this "additive by default," resulting in stacks that grow without a plan. Part of the problem, he said, is that most organizations never precisely define the security outcome they want to achieve, leaving no solid basis against which to measure new purchases.

Using asset management as an example, the Illumio director said labeling an asset answers only what it is, and says nothing about how it connects to everything around it or how policy is enforced against it. Labeling, visibility, and enforcement are three distinct jobs, and buying a separate tool for each leaves them uncoordinated. Citing a Gartner survey that found CISOs say only 20-30% of tool capability is used in some cases, he cautioned that the instinctive response to cut the stack down solves the wrong problem. Instead, every tool should be evaluated against three questions: Does it offer continuous validation against a given threat? Is it still operationally relevant? And is it effective in the environment today? VLAN-based segmentation, once effective in static data centers, is an example of a tool that still runs but whose effectiveness has dropped sharply because it offers none of the continuous validation that a hybrid, shifting estate requires.

The Illumio executive also noted that Gartner identified vendor consolidation as a strategy most organizations are pursuing, but consolidated tooling and consolidated security are not the same thing. Reducing the number of tools alone achieves little if underlying processes remain fragmented or teams work to different objectives. Comparing security to a data center built from racks, switches, storage, and cabling from a dozen suppliers that nevertheless operate as one coherent system, he said the goal is not necessarily fewer vendors, but every control — whoever built it — feeding into the same continuous picture.