diff --git a/docs/docs/advanced/air-gap.md b/docs/docs/advanced/air-gap.md index cf1df06f1bfe..138eee28bf75 100644 --- a/docs/docs/advanced/air-gap.md +++ b/docs/docs/advanced/air-gap.md @@ -1,94 +1,48 @@ -# Advanced Network Scenarios +# Air-Gapped Environments and Self-Hosting -Trivy needs to connect to the internet occasionally in order to download relevant content. This document explains the network connectivity requirements of Trivy and setting up Trivy in particular scenarios. +When you install Trivy, the installed artifact contains the scanner engine but is lacking relevant security information needed to make security detections and recommendations. These so called "databases" are fetched and maintained by Trivy automatically as needed. -## Network requirements +If your organizations blocks or restricts network traffic (from the machine Trivy runs on), Trivy might not be able to download the databases from public repositories. -Trivy's databases are distributed as OCI images via GitHub Container registry (GHCR): +This document explains how handle such advanced connectivity scenarios. -- -- -- - -The following hosts are required in order to fetch them: - -- `ghcr.io` -- `pkg-containers.githubusercontent.com` - -The databases are pulled by Trivy using the [OCI Distribution](https://github.com/opencontainers/distribution-spec) specification, which is a simple HTTPS-based protocol. - -[VEX Hub](https://github.com/aquasecurity/vexhub) is distributed from GitHub over HTTPS. -The following hosts are required in order to fetch it: - -- `api.github.com` -- `codeload.github.com` - -## Running Trivy in air-gapped environment - -An air-gapped environment refers to situations where the network connectivity from the machine Trivy runs on is blocked or restricted. - -In an air-gapped environment it is your responsibility to update the Trivy databases on a regular basis. - -## Offline Mode - -By default, Trivy will attempt to download latest databases. If it fails, the scan might fail. To avoid this behavior, you can tell Trivy to not attempt to download database files: - -- `--skip-db-update` to skip updating the main vulnerability database. -- `--skip-java-db-update` to skip updating the Java vulnerability database. -- `--skip-check-update` to skip updating the misconfiguration database. - -```shell -trivy image --skip-db-update --skip-java-db-update --offline-scan --skip-check-update myimage -``` +!!! note + Please familiarize yourself with the [Databases document](../configuration/db.md) that explains about the different databases used by Trivy and the different configuration options that control them. This guide assumes you are already familiar with the concepts explained there. ## Self-Hosting -### OCI Databases +You can host Trivy's databases in a container registry that is accessible to Trivy. -You can host the databases on your own local OCI registry. +First, make a copy of the databases into your container registry. The different databases, their use cases, and their locations are detailed in the [Trivy Databases](../configuration/db.md) document. -First, make a copy of the databases in a container registry that is accessible to Trivy. The databases are in: + !!! note + You will need to keep the databases updated in order to maintain relevant scanning results. -- `ghcr.io/aquasecurity/trivy-db:2` -- `ghcr.io/aquasecurity/trivy-java-db:1` -- `ghcr.io/aquasecurity/trivy-checks:0` +Then, tell Trivy to use the local registry using the relevant flags. -Then, tell Trivy to use the local registry: +For example, we if we scan a Java application, we copy the `trivy-db` and `trivy-java-db` databases to our local registry, and tell Trivy to use them. In this case we also need to turn off Trivy's [Java scanner external service](../coverage/language/java.md) with the `--offline-scan` flag: ```shell trivy image \ --db-repository myregistry.local/trivy-db \ --java-db-repository myregistry.local/trivy-java-db \ - --checks-bundle-repository myregistry.local/trivy-checks \ + --offline-scan \ myimage ``` -#### Authentication +### Registry Authentication If the registry requires authentication, you can configure it as described in the [private registry authentication document](../advanced/private-registries/index.md). -### VEX Hub - -You can host a copy of VEX Hub on your own internal server. - -First, make a copy of VEX Hub in a location that is accessible to Trivy. - -1. Download the [VEX Hub](https://github.com/aquasecurity/vexhub) archive from: . -1. Download the [VEX Hub Repository Manifest](https://github.com/aquasecurity/vex-repo-spec#2-repository-manifest) file from: . -1. Create or identify an internal HTTP server that can serve the VEX Hub repository in your environment (e.g `https://server.local`). -1. Make the downloaded archive file available for serving from your server (e.g `https://server.local/main.zip`). -1. Modify the downloaded manifest file's [Location URL](https://github.com/aquasecurity/vex-repo-spec?tab=readme-ov-file#locations-subfields) field to the URL of the archive file on your server (e.g `url: https://server.local/main.zip`). -1. Make the manifest file available for serving from your server under the `/.well-known` path (e.g `https://server.local/.well-known/vex-repository.json`). - -Then, tell Trivy to use the local VEX Repository: +### Pull through cache -1. Locate your [Trivy VEX configuration file](../supply-chain/vex/repo/#configuration-file) by running `trivy vex repo init`. Make the following changes to the file. -1. Disable the default VEX Hub repo (`enabled: false`) -1. Add your internal VEX Hub repository as a [custom repository](../supply-chain/vex/repo/#custom-repositories) with the URL pointing to your local server (e.g `url: https://server.local`). +You can install a pull-through cache service inside your environment that automatically serves consecutive pulls from cache instead of reaching the origin. +Here are some examples for pull-through cache solutions: -#### Authentication - -If your server requires authentication, you can configure it as described in the [VEX Repository Authentication document](../supply-chain/vex/repo/#authentication). +- [Docker Registry](https://docs.docker.com/docker-hub/mirror/) +- [Harbor](https://goharbor.io/docs/2.1.0/administration/configure-proxy-cache/) +- [Zot](https://zotregistry.dev/v2.1.0/articles/mirroring) +- [AWS ECR](https://docs.aws.amazon.com/AmazonECR/latest/userguide/pull-through-cache.html) ## Manual cache population @@ -157,6 +111,27 @@ For Java DB the process is the same, except for the following: ## Misconfigurations scanning -Note that the misconfigurations checks bundle is also embedded in the Trivy binary (at build time), and will be used as a fallback if the external database is not available. This means that you can still scan for misconfigurations in an air-gapped environment using the Checks from the time of the Trivy release you are using. +Misconfigurations checks are also embedded in the Trivy binary (at build time), and will be used as a fallback if the external database is not available. This means that you can still scan for misconfigurations in an air-gapped environment using the Checks from the time of the Trivy release you are using. + +## VEX Hub + +You can host a copy of VEX Hub on your own internal server. + +First, make a copy of VEX Hub in a location that is accessible to Trivy. + +1. Download the [VEX Hub](https://github.com/aquasecurity/vexhub) archive from: . +1. Download the [VEX Hub Repository Manifest](https://github.com/aquasecurity/vex-repo-spec#2-repository-manifest) file from: . +1. Create or identify an internal HTTP server that can serve the VEX Hub repository in your environment (e.g `https://server.local`). +1. Make the downloaded archive file available for serving from your server (e.g `https://server.local/main.zip`). +1. Modify the downloaded manifest file's [Location URL](https://github.com/aquasecurity/vex-repo-spec?tab=readme-ov-file#locations-subfields) field to the URL of the archive file on your server (e.g `url: https://server.local/main.zip`). +1. Make the manifest file available for serving from your server under the `/.well-known` path (e.g `https://server.local/.well-known/vex-repository.json`). + +Then, tell Trivy to use the local VEX Repository: -The misconfiguration scanner can be configured to load checks from a local directory, using the `--config-check` flag. In an air-gapped scenario you can copy the checks library from [Trivy checks repository](https://github.com/aquasecurity/trivy-checks) into a local directory, and load it with this flag. See more in the [Misconfiguration scanner documentation](../scanner/misconfiguration/index.md). +1. Locate your [Trivy VEX configuration file](../supply-chain/vex/repo/#configuration-file) by running `trivy vex repo init`. Make the following changes to the file. +1. Disable the default VEX Hub repo (`enabled: false`) +1. Add your internal VEX Hub repository as a [custom repository](../supply-chain/vex/repo/#custom-repositories) with the URL pointing to your local server (e.g `url: https://server.local`). + +### VEX Hub Authentication + +If your server requires authentication, you can configure it as described in the [VEX Repository Authentication document](../supply-chain/vex/repo/#authentication). diff --git a/docs/docs/configuration/db.md b/docs/docs/configuration/db.md index ccffae1e5302..50157e659d97 100644 --- a/docs/docs/configuration/db.md +++ b/docs/docs/configuration/db.md @@ -1,120 +1,137 @@ -# DB +# Trivy Databases -| Scanner | Supported | -|:----------------:|:---------:| -| Vulnerability | ✓ | -| Misconfiguration | | -| Secret | | -| License | | +When you install Trivy, the installed artifact contains the scanner engine but is lacking relevant security information needed to make security detections and recommendations. These so called "databases" are fetched and maintained by Trivy automatically as needed, so normally you shouldn't notice or worry about them. However, some situations might require your attention to Trivy's network connectivity. This section elaborates on the database management mechanism and it's configuration options. -The vulnerability database and the Java index database are needed only for vulnerability scanning. -See [here](../scanner/vulnerability.md) for the detail. +Trivy relies on the following databases: -## Vulnerability Database +DB | Artifact name | Contents | Purpose +--- | --- | --- | --- +Vulnerabilities DB | `trivy-db` | CVE information collected from various feeds | used only for [vulnerability scanning](../scanner/vulnerability.md) +Java DB | `trivy-java-db` | Index of Java artifacts and their hash digest | used to identify Java artifacts only in [Java vulnerability scanning](../coverage/language/java.md) +Misconfiguration DB | `checks-db` | Logic of misconfiguration checks | used only in [misconfiguration/IaC scanning](../scanner/misconfiguration/check/builtin.md) +VEX Hub | `vex-hub` | VEX statements | Used only in [VEX Hub is enabled](../supply-chain/vex/repo.md) in vulnerability scanning. -### Skip update of vulnerability DB -If you want to skip downloading the vulnerability database, use the `--skip-db-update` option. +## External Services -``` -$ trivy image --skip-db-update python:3.4-alpine3.9 -``` +In addition to the above, some specific scanning features might connect to external services, and have different connectivity requirements and configuration options. This document discusses only Trivy's own databases, but for your convenience here are use cases where external services are involved: -
-Result +- [Java vulnerability scanning](../coverage/language/java.md). -``` -2019-05-16T12:48:08.703+0900 INFO Detecting Alpine vulnerabilities... - -python:3.4-alpine3.9 (alpine 3.9.2) -=================================== -Total: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0) - -+---------+------------------+----------+-------------------+---------------+--------------------------------+ -| LIBRARY | VULNERABILITY ID | SEVERITY | INSTALLED VERSION | FIXED VERSION | TITLE | -+---------+------------------+----------+-------------------+---------------+--------------------------------+ -| openssl | CVE-2019-1543 | MEDIUM | 1.1.1a-r1 | 1.1.1b-r1 | openssl: ChaCha20-Poly1305 | -| | | | | | with long nonces | -+---------+------------------+----------+-------------------+---------------+--------------------------------+ -``` +## Locations + +Following are official locations of Trivy databases: + +| Registry | Image Address | Link +| --- | --- | --- +| GHCR | `ghcr.io/aquasecurity/trivy-db:2` | +| | `ghcr.io/aquasecurity/trivy-java-db` | +| | `aquaghcr.io/aquasecurity/ecurity/trivy-checks` | +| Docker Hub | `aquasec/trivy-db:2` | +| | `aquasec/trivy-java-db:1` | +| AWS ECR | `public.ecr.aws/aquasecurity/trivy-db:2` | +| | `public.ecr.aws/aquasecurity/trivy-java-db:1` | + + VEX Hub is fetched from VEX Hub GitHub Repository directly: . + +### Automatic fallback + +Trivy will attempt to pull images from the official registries in the order specified. In case of failure of pulling a database, Trivy will fall back to the next alternative registry in the order specified. +You can specify additional alternative repositories as explained in the [configuring database locations section](#locations). + +The Checks Database registry location option does not support fallback through multiple options. + +## Connectivity requirements + +| database | location | protocol | hosts +| --- | --- | --- | --- +|
  • `trivy-db`
  • `trivy-java-db`
  • `checks-db`
| GHCR | [OCI Distribution](https://github.com/opencontainers/distribution-spec) over HTTPS |
  • `ghcr.io`
  • `pkg-containers.githubusercontent.com`
+| `vexhub`| GitHub | Git over HTTPS |
  • `api.github.com`
  • `codeload.github.com`
+ +For more information about GitHub connectivity (including specific IP addresses), please refer to [GitHub's connectivity troubleshooting guide](https://docs.github.com/en/get-started/using-github/troubleshooting-connectivity-problems). + +### Rate limiting + +Trivy is an open source project that relies on public free infrastructure. In case of extreme load, you may encounter rate limiting when Trivy attempts to update its databases. If you are facing rate-limiting issues: +1. Consider self-hosting the databases, or implementing a proxy-cache in your organization. +2. Look into specific registry rate-limiting policies which might provide solution for your use-case (for example by authenticating with the registry). +3. Consider using a commercial service that provides the full [Aqua platform](https://www.aquasec.com/products/software-supply-chain-security/), which includes a stable and reliable scanning service. + +## DB Management Configuration -
+### Database Locations -### Only download vulnerability database -You can also ask `Trivy` to simply retrieve the vulnerability database. -This is useful to initialize workers in Continuous Integration systems. +You can configure Trivy to download databases from alternative locations by using the flags: + +- `--db-repository` +- `--java-db-repository` +- `--checks-bundle-repository` + +The value should be an image address in a container registry. + +For example: ``` -$ trivy image --download-db-only +trivy image --db-repository registry.gitlab.com/gitlab-org/security-products/dependencies/trivy-db alpine ``` -### DB Repository -`Trivy` could also download the vulnerability database from an external OCI registry by using `--db-repository` option. +The `--db-repository` flag accepts multiple values, which can be used to specify multiple alternative repository locations. In case of failure, Trivy will fall back to alternative registries in the order specified. An attempt to download from the next repository is only made if a temporary error is received (e.g. status 429 or 5xx). + +For example: ``` -$ trivy image --db-repository registry.gitlab.com/gitlab-org/security-products/dependencies/trivy-db +trivy image --db-repository my.registry.local/trivy-db,registry.gitlab.com/gitlab-org/security-products/dependencies/trivy-db alpine ``` -The media type of the OCI layer must be `application/vnd.aquasec.trivy.db.layer.v1.tar+gzip`. -You can reference the OCI manifest of [trivy-db]. - -
-Manifest - -```shell -{ - "schemaVersion": 2, - "mediaType": "application/vnd.oci.image.manifest.v1+json", - "config": { - "mediaType": "application/vnd.aquasec.trivy.config.v1+json", - "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a", - "size": 2 - }, - "layers": [ - { - "mediaType": "application/vnd.aquasec.trivy.db.layer.v1.tar+gzip", - "digest": "sha256:29ad6505b8957c7cd4c367e7c705c641a9020d2be256812c5f4cc2fc099f4f02", - "size": 55474933, - "annotations": { - "org.opencontainers.image.title": "db.tar.gz" - } - } - ], - "annotations": { - "org.opencontainers.image.created": "2024-09-11T06:14:51Z" - } -} -``` -
+!!! note + setting the repository location flags override the default values which include the official db locations. In case you want to preserve the default locations, you should include them in the list the you set as repository locations. !!!note - Trivy automatically adds the `trivy-db` schema version as a tag if the tag is not used: + When pulling `trivy-db` or `trivy-java-db`, if image tag is not specified, Trivy defaults to the db schema number instead of the `latest` tag. Currently, for `trivy-db` the default tag is `2`, for `trivy-java-db` the default tag is `1`, for `trivy-checks` the default tag is `0`. - `trivy-db-registry:latest` => `trivy-db-registry:latest`, but `trivy-db-registry` => `trivy-db-registry:2`. +VEX Hub repository locations can be configured separately using the [VEX configuration file](../supply-chain/vex/repo.md) +### Skip updates -## Java Index Database -The same options are also available for the Java index DB, which is used for scanning Java applications. -Skipping an update can be done by using the `--skip-java-db-update` option, while `--download-java-db-only` can be used to only download the Java index DB. +You can configure Trivy to not attempt to download database at all using the flags: -!!! Note - In [Client/Server](../references/modes/client-server.md) mode, `Java index DB` is currently only used on the `client` side. +- `--skip-db-update` +- `--skip-java-db-update` +- `--skip-check-update` -Downloading the Java index DB from an external OCI registry can be done by using the `--java-db-repository` option. +For example: ``` -$ trivy image --java-db-repository registry.gitlab.com/gitlab-org/security-products/dependencies/trivy-java-db --download-java-db-only +trivy image --skip-db-update --skip-java-db-update --offline-scan --skip-check-update myimage ``` -The media type of the OCI layer must be `application/vnd.aquasec.trivy.javadb.layer.v1.tar+gzip`. -You can reference the OCI manifest of [trivy-java-db]. +### Only update -!!!note - Trivy automatically adds the `trivy-java-db` schema version as a tag if the tag is not used: +You can ask `Trivy` to update the database without performing a scan using the flags: + +- `--download-db-only` +- `--download-java-db-only` + +For example: - `java-db-registry:latest` => `java-db-registry:latest`, but `java-db-registry` => `java-db-registry:1`. +``` +trivy image --download-db-only +``` + +### Remove Databases + +`trivy clean` command removes caches and databases. +You can select which cache component to remove: -## Remove DBs -"trivy clean" command removes caches and databases. +option | description +--- | --- +`-a`/`--all` | remove all caches +`--checks-bundle` | remove checks bundle +`--java-db` | remove Java database +`--scan-cache` | remove scan cache (container and VM image analysis results) +`--vex-repo` | remove VEX repositories +`--vuln-db` | remove vulnerability database + +Example: ``` $ trivy clean --vuln-db --java-db @@ -122,5 +139,13 @@ $ trivy clean --vuln-db --java-db 2024-06-24T11:42:31+06:00 INFO Removing Java database... ``` -[trivy-db]: https://github.com/aquasecurity/trivy-db/pkgs/container/trivy-db -[trivy-java-db]: https://github.com/aquasecurity/trivy-java-db/pkgs/container/trivy-java-db \ No newline at end of file +## Self-Hosting + +You can host all of Trivy's databases on your own local environment (to prevent external connectivity). For more information, refer to the [Air-Gapped Environments and Self-Hosting](../advanced/air-gap.md) document. + +When serving, proxying, or manipulating Trivy's databases, note that the media type of the OCI layer is not a standard container image type. + +DB | Media Type | Reference +--- | --- | --- +`trivy-db` | `application/vnd.aquasec.trivy.db.layer.v1.tar+gzip` | +`trivy-java-db` | `application/vnd.aquasec.trivy.javadb.layer.v1.tar+gzip` | https://github.com/aquasecurity/trivy-java-db/pkgs/container/trivy-java-db diff --git a/docs/docs/configuration/index.md b/docs/docs/configuration/index.md index b70163954beb..8c7aa3475b05 100644 --- a/docs/docs/configuration/index.md +++ b/docs/docs/configuration/index.md @@ -1,23 +1,21 @@ # Configuration -Trivy can be configured using the following ways. Each item takes precedence over the item below it: +Trivy's settings can be configured in any of the following methods, which will apply in the following precedence: -- CLI flags -- Environment variables -- Configuration file +1. CLI flags (overrides all other settings) +2. Environment variables (overrides config file settings) +3. Configuration file ## CLI Flags -You can view the list of available flags using the `--help` option. -For more details, please refer to [the CLI reference](../references/configuration/cli/trivy.md). +You can view the list of available flags by adding the `--help` flag to a Trivy command, or by exploring the [CLI reference](../references/configuration/cli/trivy.md). ## Environment Variables -Trivy can be customized by environment variables. -The environment variable key is the flag name converted by the following procedure. +Any CLI option can be set as an environment variable. The environment variable name are similar to the CLI option name, with the following augmentations: - Add `TRIVY_` prefix -- Make it all uppercase +- All uppercase letters - Replace `-` with `_` -For example, +For example: - `--debug` => `TRIVY_DEBUG` - `--cache-dir` => `TRIVY_CACHE_DIR` @@ -27,5 +25,6 @@ $ TRIVY_DEBUG=true TRIVY_SEVERITY=CRITICAL trivy image alpine:3.15 ``` ## Configuration File -By default, Trivy reads the `trivy.yaml` file. -For more details, please refer to [the page](../references/configuration/config-file.md). +Any setting can be set in a YAML file. By default, config file named `trivy.yaml` is read from the current directory where Trivy is run. To load configuration from a different file, use the `--config` flag and specify the config path to load: `trivy --config /etc/trivy/myconfig.yaml`. + +The structure and settings of the YAML config file is documented in the [Config file](../references/configuration/config-file.md) document. diff --git a/mkdocs.yml b/mkdocs.yml index 00e458e15a91..8fb16167e2b5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -121,7 +121,7 @@ nav: - Skipping Files: docs/configuration/skipping.md - Reporting: docs/configuration/reporting.md - Cache: docs/configuration/cache.md - - DB: docs/configuration/db.md + - Databases: docs/configuration/db.md - Others: docs/configuration/others.md - Supply Chain: - SBOM: docs/supply-chain/sbom.md