The main criterion for determining the quality of a security scanner is, of course, the number of vulnerabilities it can detect. But a scanner should be judged not by the number its manufacturer claims, but by the number of vulnerabilities the scanner actually detects. For example, one important feature is the scanner's ability to identify services using non-standard ports, without which vulnerabilities known to the scanner cannot actually be revealed.

A second criterion is the number of false detections. There is no doubt that it is better to be over-prepared, but with a high number of false detections, security specialists or system administrators waste enormous amounts of time verifying detections and filtering out what they don't need.

The third criterion is convenience. Although some may ignore this as a less important factor than the others, in the end it saves time and energy by minimizing human error.

In this survey the following products were tested:

Before testing, each scanner's vulnerability databases were updated through the Internet with the most recent versions.

Servers using the following operating systems were analyzed by the scanners:

The last server in the list was specially configured to make analysis more difficult. When connecting to all services besides HTTP, it responded with a service banner. Identical responses were sent to all subsequent queries. HTTP service responded to queries with identical replies, with a random choice of "200 OK" or "404 Not Found".

Vulnerability search quality is evaluated in points as follows:

For each false detection, 50% of points of the bonus for correct detection were subtracted based on the premise that a false response is not critical, but it does cause delays in the overall process of vulnerability removal.

Product user interface and operating convenience were assessed as follows:

Detailed tables with scan results are cited in APPENDIX A. Total points are as follows:

These data are easier to interpret when plotted on a chart:

Table of total functionality comparison results:

As you can see, all the scanners scored almost identically in convenience and most other features, with the exception of LanGuard and NetRecon.

For speed of operation, the slowest product was Internet Scanner, followed by. MaxPatrol and Nessus. In speed, the three leaders were LanGuard, Retina and NetRecon. Of course, speed in a security scanner is among the least significant criteria. The first and most important indicator was and remains the quality of vulnerability detection.