Container Security Tools Comparison for Vulnerability Scans

Recently, we took on a new challenge: compare 5 popular container security tools, including our solution. We wanted to see how the products stack up against each other. How did they do? Read this full container security tools comparison to find out!

TL; DR:

Containers have been causing waves in IT and dev circles since 2013, when Docker’s container technology was launched. They have revolutionized deployment, adding both speed and stability and have become critical for most IT operations, so securing them is a priority for all of us.

How well do various tools do that? See the table below:

But before we get into the details, it’s worthwhile to quickly revisit the importance of application container security in the modern-day development landscape.

The Importance of Container Security

Containers enable developers to run applications quickly and reliably when moved from one computing environment to another. But despite their many advantages – including increased application isolation – containers also amplify security risks.

Increasing adoption in production environments makes them attractive to malicious actors. Since traditional network security solutions cannot always protect against lateral attacks, a lot of effort goes into developing application container security solutions.

Container security refers to the tools (e.g. Docker container security solutions) and policies implemented to protect container integrity and reliability, mitigate risk, and minimize vulnerabilities.

Container Security Tools Compared

To protect containers from attacks, many security tools are available. Usually, they audit the Common Vulnerabilities and Exposures (CVE) set by the National Vulnerability Database (NVD), or the benchmarks set by the Center for Internet Security (CIS).

Most containerized applications and their underlying infrastructure are distributed widely and highly dynamic. In this scenario, manual vulnerability scanning can be time-consuming and resource-intensive. To reduce operational overhead, many tools offer automation. Some focus on specific aspects of the cloud-native ecosystem, e.g. runtime security.

For our analysis, we picked 5 popular automated container scanners:

Methodology

Before starting our analysis, we set up three images. The images are a logical progression where start with a vulnerable version of squid, then patch it and then add a vulnerable proprietary library, a proxy for applications you might produce and deploy in Docker images

1. Seeded with a vulnerable version of Squid, a caching and forwarding HTTP web proxy
2. Patch Squid to the latest safe version
3. Download a proprietary jar file that is vulnerable. Your own applications would typically fall in this category, and it is challenging for most container scanning tools to analyze these.

We expected that each application would find vulnerabilities in all these steps.

However, this is not quite what happened!

Before we reveal the results of our tool comparison, here’s a sequence of steps that shows the Docker files used to build the images. Also, for readers planning to replicate our experiment, bear in mind that vulnerability scanning is sensitive to the date of the scan.

We completed this experiment in early April 2021. New vulnerabilities may have been found and published since then, and security scanning tools themselves may have also changed.

Procedure to Build Images with Docker Files

The steps to build the Docker images used in this analysis.

If you need the build scripts, please ask us. We believe in transparency and are happy to provide them.

Results of Container Security Tools Comparison & Analysis

1. Aqua

For teams wondering how to secure Docker containers, Aqua claims to provide “enterprise-grade security for Docker environments” from development to production. Its tool scans images for vulnerabilities, malware, configuration issues, etc., for continuous image assurance.

Its vulnerabilities database is aggregated from multiple, constantly-updated data streams to increase detection accuracy and provide better protection.

Despite these claims, the tool didn’t quite make the cut in our test. In fact, Aqua found no vulnerabilities at all, raising doubts about its effectiveness.

2. Snyk

Snyk helps teams automatically find, prioritize and fix vulnerabilities in containers throughout the container lifecycle. It can detect vulnerable dependencies during coding, prevent new vulnerabilities from passing through the build process, and test the production environment for newly-disclosed vulnerabilities.

Snyk says that it has fixed over 5 million container vulnerabilities. But during our tests, Snyk found two vulnerabilities in Step 1:

• CVE-2020-25097 (Squid)
• CVE-2021-30139 (apk-tools)

ps. Snyk did not find any vulnerabilities from Steps 2 and 3.

3. Docker Hub

When a Docker image is pushed to Docker Hub, it automatically scans it for vulnerabilities. Teams can review the security state of images and fix identified issues for more secure deployments. The vulnerability report displays vulnerabilities and sorts them according to severity. It also displays information about the:

• Package containing the vulnerability
• Version in which it was introduced
• Whether the vulnerability is fixed in a later version

In our analysis, we found that this Docker container security scanner is not effective at finding all vulnerabilities. During testing, it only found one vulnerability from Step 1.

4. Quay

Quay automatically scans containers to provide a real-time view of known vulnerabilities. The scan report displays vulnerabilities by severity level: Low, Medium and High. It also specifies whether patches are available.

But in our vulnerability test, Quay found no vulnerabilities. For all 3 steps, the report displayed a “passed” status for the security scan.

And now, we have come to the final tool in our analysis: our own MergeBase tool.

5. MergeBase

In our analysis, only MergeBase found all vulnerabilities, including those the other tools missed:

• CVE-2021-28116 (Squid), for which no patch is available
• CVE-2016-5725 in the application, a directory traversal vulnerability in JCraft JSch before 0.1.54 on Windows, when the mode is ChannelSftp (source: CVE Mitre)

In summary, MergeBase found:

• 3 vulnerabilities in Step 1
• 1 vulnerability in Step 2
• 2 vulnerabilities in Step 3

Container Security Tools Comparison Conclusion

In containerized environments, the deployment pipeline is often standardized across different dev teams. Container scanning can help find vulnerabilities and take proactive action to fix security gaps. Securing containers and building security into the CI/CD their pipeline can help reduce the size of the attack surface.

However, different container scanning solutions yield inconsistent results in the same environment. Worse, many solutions fall short of their claims to help strengthen end-to-end container security.

In our analysis of 5 application container security tools, we found that our tool, MergeBase, was the only one that could find all vulnerabilities in our testing environment. Thus, compared to other tools, MergeBase provides complete DevSecOps coverage and reliable container security.

Want to know more about MergeBase?