DevSecOps: Enhancing Security With Vulnerability Scanning of Images and Source Code in CI/CD
This guide shows how to use Syft, Grype, and Trivy to scan container images for vulnerabilities in Gitlab pipelines during release.
Join the DZone community and get the full member experience.
Join For FreeMany companies strive to adopt the DevOps approach for software development and delivery. Alongside this, they face increasing security challenges, leading to the implementation of new innovative software development methods.
The need for security in the software deployment process is evident. Therefore, integrating security into CI/CD workflows should be done carefully to account for the ever-evolving technological landscape.
DevSecOps is an important concept that provides an automated approach to integrating security into the software delivery lifecycle. In the context of container solutions, there are specific challenges in adding security controls.
Furthermore, when using open-source containers, many of them may contain known and unknown vulnerabilities. For many organizations, it is difficult to determine the security of their containers confidently. Hence, these tools provide additional capabilities and features that facilitate faster implementation. However, not all of them always align with the organization's security goals. The reality is that lacking the skill to design secure deployment pipelines can come at a high cost to a company.
This article presents a guide on setting up, running, and using Syft, Grype, and Trivy in CI/CD. It describes how to automatically configure the release process to scan container images for vulnerabilities with Gitlab pipelines.
Solution Overview
Syft, Grype, and Trivy are popular vulnerability scanning tools used in the software development and deployment process. Here's a brief overview of each tool:
- Syft: Developed by Anchore, Syft is an open-source command-line tool that focuses on scanning container images for vulnerabilities. It provides a detailed analysis of the container's software bill of materials (SBOM) by inspecting package managers, libraries, and dependencies. Syft also has integrations with various container registries and orchestration platforms to simplify the scanning process.
- Grype: Also developed by Anchore, Grype is another open-source vulnerability scanner specifically designed for container images. It scans the image's software composition analysis (SCA) data to identify any known vulnerabilities. Grype's strength lies in its fast scanning capability and ability to handle complex image layers and formats.
- Trivy: Trivy is an open-source vulnerability scanner that specializes in container images, as well as operating systems and applications. It uses vulnerability databases from various sources, including NVD, Red Hat, and Ubuntu, to detect known vulnerabilities in container images. Trivy is easy to use, offers extensive configuration options, and provides rapid and reliable scanning results.
Get Started
For scanning, you can use an Alpine Linux image; let's assume the latest version. We embed an additional step before loading the collected image into the registry, saving dependency lists in the artifacts of the task, and storing the database in the cache.
Here is a complete set of scans and reports received:
variables:
GO_VERSION: '1.20'
stages:
- build
- scan
- upload
build:
stage: build
tags:
- docker
- dind
image: golang:1.20-bullseye
variables:
DOCKER_DRIVER: overlay2
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_VERIFY: 1
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
before_script:
- |
TIME=30; PASS=0
echo "Waiting for docker daemon..."
until [ -f "${DOCKER_CERT_PATH}/ca.pem" ] || [ ${TIME} -lt ${PASS} ]
do
echo -n ".";PASS=`expr $PASS + 1`; sleep 1
done
script:
- make gobuild
artifacts:
name: "$CONTAINER_NAME"
expire_in: 1h
paths:
- build/
upload:
stage: upload
needs:
- scan
dependencies: ["build"]
tags:
- docker
- dind
image: docker:23.0.3
variables:
DOCKER_DRIVER: overlay2
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_VERIFY: 1
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
before_script:
- |
TIME=30; PASS=0
echo "Waiting for docker daemon..."
until [ -f "${DOCKER_CERT_PATH}/ca.pem" ] || [ ${TIME} -lt ${PASS} ]
do
echo -n ".";PASS=`expr $PASS + 1`; sleep 1
done
script:
- |
echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER $CI_REGISTRY --password-stdin
docker build -t $CI_COMMIT_REF_SLUG:$CI_COMMIT_REF_SLUG .
docker push $CI_COMMIT_REF_SLUG:$CI_COMMIT_REF_SLUG
docker save $CI_COMMIT_REF_SLUG:$CI_COMMIT_REF_SLUG -o build/$CI_PROJECT_NAME.tar
artifacts:
name: "$CONTAINER_NAME"
expire_in: 1h
paths:
- build/
scan_cve:
stage: scan
image: alpine:latest
dependencies: ["build"]
variables:
GIT_STRATEGY: clone
SYFT_REGISTRY_AUTH_AUTHORITY: "$CI_REGISTRY"
SYFT_REGISTRY_AUTH_USERNAME: "$CI_REGISTRY_USER"
SYFT_REGISTRY_AUTH_PASSWORD: "$CI_REGISTRY_PASSWORD"
SYFT_FILE_CONTENTS_SKIP_FILES_ABOVE_SIZE: 20000000
GRYPE_REGISTRY_AUTH_AUTHORITY: "$CI_REGISTRY"
GRYPE_REGISTRY_AUTH_USERNAME: "$CI_REGISTRY_USER"
GRYPE_REGISTRY_AUTH_PASSWORD: "$CI_REGISTRY_PASSWORD"
GRYPE_DB_CACHE_DIR: "./.cache/grype/"
GRYPE_EXCLUDE: "./.cache"
TRIVY_USERNAME: "$CI_REGISTRY_USER"
TRIVY_PASSWORD: "$CI_REGISTRY_PASSWORD"
TRIVY_AUTH_URL: "$CI_REGISTRY"
TRIVY_NO_PROGRESS: "true"
TRIVY_CACHE_DIR: "./.cache/trivy/"
TRIVY_VERSION: "v0.43.1"
FULL_IMAGE_NAME: "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
allow_failure: true
cache:
key: $CI_JOB_NAME
paths:
- ./.cache/
before_script:
- apk add --update-cache curl openssl docker-cli
- mkdir -p ./.out
- export TRIVY_VERSION=${TRIVY_VERSION:-v0.43.1}
- curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
- curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin
- curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin #${TRIVY_VERSION}
- curl -sSL -o /tmp/trivy-gitlab.tpl https://github.com/aquasecurity/trivy/raw/main/contrib/gitlab.tpl
- curl -sSL -o /tmp/trivy-codequality.tpl https://github.com/aquasecurity/trivy/raw/main/contrib/gitlab-codequality.tpl
- curl -sSL -o /tmp/trivy-html.tpl https://github.com/aquasecurity/trivy/raw/main/contrib/html.tpl
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
script:
- echo "--- SYFT ---"
- time syft packages registry:$FULL_IMAGE_NAME --scope all-layers
- time syft packages dir:./ --scope all-layers -o table=.out/syft_sbom.txt -o spdx-json=.out/syft_sbom_spdx.json
- echo "--- GRYPE ---"
- time grype db update
- time grype dir:./ --add-cpes-if-none -o table > .out/grype_cve_table.txt
- time grype dir:./ --add-cpes-if-none -o json > .out/grype_cve_json.json
- time grype dir:./ --add-cpes-if-none -o cyclonedx > .out/grype_cyclonedx.xml ## this one is ok
- time grype sbom:.out/syft_sbom_spdx.json --add-cpes-if-none --fail-on critical --only-fixed
- time grype dir:./ --add-cpes-if-none
- time grype dir:./ --add-cpes-if-none --only-fixed
- time grype build/$CONTAINER_NAME --add-cpes-if-none -o table #--fail-on critical --only-fixed
- time grype registry:$FULL_IMAGE_NAME --add-cpes-if-none --fail-on critical --only-fixed
- echo "--- TRIVY ---"
- trivy --version
- time trivy image --clear-cache
- time trivy image --download-db-only
- time trivy image --exit-code 0 --format template --template "@/tmp/trivy-gitlab.tpl" -o .out/trivy_cve.json $IMAGE
- time trivy image --exit-code 0 --format template --template "@/tmp/trivy-codequality.tpl" -o .out/trivy_codequality.json $IMAGE
- time trivy image --exit-code 0 --format template --template "@/tmp/trivy-gitlab.tpl" -o .out/gl-container-scanning-report.json $IMAGE
- time trivy image --exit-code 0 --format template --template "@/tmp/trivy-html.tpl" -o .out/trivy_cve.html $IMAGE
- time trivy sbom --format spdx -o ".out/trivy_sbom_fs_spdx.spdx" --type fs .
- time trivy sbom --format spdx-json -o ".out/trivy_sbom_spdxjson.json" --type image "$FULL_IMAGE_NAME"
- time trivy image --format cyclonedx -o ".out/trivy_sbom_cyclonedx.json" "$FULL_IMAGE_NAME"
- time trivy image --exit-code 0 "$FULL_IMAGE_NAME"
- time trivy image --exit-code 0 --severity CRITICAL "$FULL_IMAGE_NAME"
- time trivy image --exit-code 1 --severity CRITICAL --ignore-unfixed "$FULL_IMAGE_NAME"
artifacts:
reports:
dependency_scanning: .out/grype_cve.json
container_scanning: .out/gl-container-scanning-report.json
codequality: .out/trivy_codequality.json
name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}
paths:
- .out/The result of Grype 's in syft_sbom_spdx.json work looks something like this:
#grype sbom:.out/syft_sbom_spdx.json --add-cpes-if-none --fail-on critical --only-fixed
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
go.mongodb.org/mongo-driver v1.1.2 1.5.1 go-module GHSA-f6mq-5m25-4r72 MediumThe result of Trivy in trivy_codequality.json work looks something like this:
#trivy image --exit-code 0 "$FULL_IMAGE_NAME"
2022-06-07T13:18:57.228Z INFO Detected OS: alpine
2022-06-07T13:18:57.229Z INFO Detecting Alpine vulnerabilities...
2022-06-07T13:18:57.230Z INFO Number of language-specific files: 1
2022-06-07T13:18:57.230Z INFO Detecting gobinary vulnerabilities...
Total: 2 (UNKNOWN: 1, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)
┌─────────────────────────────┬────────────────┬──────────┬───────────────────┬───────────────┬──────────────────────────────────────────────────────────────┐
│ Library │ Vulnerability │ Severity │ Installed Version │ Fixed Version │ Title │
├─────────────────────────────┼────────────────┼──────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ go.mongodb.org/mongo-driver │ CVE-2021-20329 │ MEDIUM │ v1.1.2 │ 1.5.1 │ mongo-go-driver: specific cstrings input may not be properly │
│ │ │ │ │ │ validated │
│ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2021-20329 │
├─────────────────────────────┼────────────────┼──────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ golang.org/x/text │ CVE-2021-38561 │ UNKNOWN │ v0.3.3 │ 0.3.7 │ Due to improper index calculation, an incorrectly formatted │
│ │ │ │ │ │ language tag can cause... │
│ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2021-38561 │
└─────────────────────────────┴────────────────┴──────────┴───────────────────┴───────────────┴──────────────────────────────────────────────────────────────┘Conclusion
Vulnerability scanning plays a crucial role in strengthening the security of software applications and ensuring a proactive approach to addressing vulnerabilities. By integrating vulnerability scanners into your CI pipeline for container images and source code, you can detect vulnerabilities early, maintain compliance, and foster a secure coding culture. Leveraging automation and comprehensive scanning tools allow teams to efficiently monitor and continuously improve the security of their software, ultimately delivering safer applications to end-users.
Opinions expressed by DZone contributors are their own.
Comments