Classical failures of vulnerability scanning, penetration testing, and red team exercises

In this section, we will focus on the limitations of classical Vscanning, pentesting, and red teaming exercises. Let's now discuss the actual meaning of these three methodologies in simple terms and look at their limitations:

• Vscanning: This is the process of identifying vulnerabilities or security loopholes in a system or network. The limitations with Vscanning are the potential vulnerabilities, including false positives, which can be confusing to the business owner.
• Pentesting: This is the process of safely exploiting vulnerabilities without much impact to the existing network or business. There are a fewer number of false positives, since the testers will try and simulate the exploit faithfully. A key limitation of pentesting is that the exploits it can detect are only those that are currently known and publicly available exploits. Also, most pentests are project-focused tests. In pentesting, we often hear "Yay, got root!", but we never then hear "What's next?" This could be due to various reasons, such as the project limiting the pentester to reporting only the high-risk issues immediately to the client, or the client being interested only in one segment of the network and wanting the pentester to compromise.
• RTEs: This is the process of evaluating the effectiveness of an organization's defenses against cyber threats and improving them; during RTEs, we notice multiple ways of achieving project goals, such as the complete coverage of all activities under a defined project goal. The key limitations with RTEs are that they are limited in terms of time and can only simulate specific predefined scenarios, and they have an assumed rather than a real environment.

Often, all three of these testing methodologies refer to the terms hack or compromise. "We will hack your network and show you where its weaknesses are –," but wait: does the client or business owner understand the terms hack or compromise? How do we measure hack or compromise? What are the criteria? When do we know that a hack or compromise is complete? All these questions point to only one thing: needing to know the primary goal.

The primary goal of pentesting and RTEs is determining the risk, differentiating the risk rating from the scanner, and performing a business risk value assessment of each asset, as well as the brand/image of the organization. It's not about how many threats there are, but how much risk the organization is exposed to. A risk does not really constitute a threat and doesn't necessarily need to be demonstrated. For example, a Cross-Site Scripting (XSS) attack on a brochure website may not have significant impact on the business; however, a client might put in a mitigation plan for the risk using a Web Application Firewall (WAF) to prevent the XSS attacks.