feat(misconf): Support arbitrary file types with custom schema

[simar7]

Currently by design Trivy has detection logic for a variety of configurations, e.g. Terraform, Cloudformation, etc. as implemented here. Custom checks today assume that the scanned input is one of supported configuration types.

When it comes to implementing support for arbitrary file parsing, there are a few things involved:

1. First we can determine the kind of configuration, that list can be found in the docs. This would be an easy win if the input file matches a format we are already aware of.
2. If we determine it isn't one of the supported file configuration types, we can fallback to a custom schema, if provided by the user.

Today Trivy limits itself to only scan known configurations. By enabling it to scan simple file types such as JSON or YAML as-is (without looking for the known configuration type) we can enable Trivy to scan newer formats that might not yet be officially supported by Trivy but are written in languages (e.g. JSON or YAML) that trivy knows about, as long as the user provides us with a custom schema to infer the input for evaluation with checks.

Originally posted by @simar7 in #7153 (reply in thread)

[nikpivkin]

@simar7 Is it proposed to use the schema only to determine that a file matches some configuration, such as Serverless Framework or Nginx? Or is it also proposed to use schema to pass to checks for type checking?

Is it worth allowing JSON or YAML files to be scanned without passing schemas, or is there a performance concern? I did some tests in a repository with a large number of JSON and YAML files and several large JSON files (30mb each). In the first test I scanned this repository without passing schema to check the configuration, in the second test with schema. And I didn't notice any problems with scanning speed. We can disable these scanners by default.

[simar7]

Is it proposed to use the schema only to determine that a file matches some configuration, such as Serverless Framework or Nginx? Or is it also proposed to use schema to pass to checks for type checking?

I would say both. If we are able to let users pass schemas for any arbitrary format (not the ones we support) they can also write their custom checks for it. I don't anticipate we would have a lot of users doing this on day 1, but still it's a nice to have if they wish to do so. It keeps Trivy open for future extensibility.

[nikpivkin]

@simar7 What do you think about the second question?

[itaysk]

that's a good question, IMO if possible we should allow people to write custom checks to their json/yaml file without having to provide a json schema. for more mature users they can should be able to add a schema file