feat(misconf): Improve support for scanning Terraform plan JSON input

[simar7]

Motivation

We've run into several occasions (see linked issues below) where we incorrectly flag (false positive) misconfigurations in Terraform scanning when the input is the Terraform plan in JSON. This issue takes place as we're unable to parse nested blocks and attributes past the first stage as currently there's no way to "walk" the JSON input. See the more on this here

Action items

Drop support for Terraform JSON until we have a proper way to walk the input. Flagging false positives creates misinformation.

Update Feb 2024

We've found a better approach to keep and improve the terraform plan scanning functionality. As it turns out, we can unzip the plan contents (it is a zip file) and parse the HCL directly from it. This allows us to have a functionally complete HCL input which we can scan and flag for misconfigurations.

Affected issues

• fix(terraform): panic when scanning a synthesized TF config using cdktf #5080
• fix(checks): Update Athena KMS argument to latest spec #5894
• fix(terraform): false positive AVD-AZU-0013 when scanning a synthesized TF config using cdktf #5081
• fix(misconf): false positive AVD-AWS-0131 #5099

[itaysk]

a couple of questions:

1. If this removing an existing advertised feature? should we mark it as break or feat?
2. is this also affecting tf plan scanning? if so, also need to properly communicate.

[simar7]

a couple of questions:

1. If this removing an existing advertised feature? should we mark it as break or feat?

Yes we're planning to drop scanning of Terraform JSON for the reasons above. I can change that.

2. is this also affecting tf plan scanning? if so, also need to properly communicate.

Yes this is just a placeholder issue to triage any future issues that are related. I'll open a discussion to field any questions or concerns and give enough time before we actually start working on it.

[simar7]

Discuss here #5950

[knqyf263]

This is a breaking change.

It is described above In #5950, so should we change the title prefix from "feat(misconf)" to "BREAKING CHANGE(miscong)"?
e.g. #1571

[simar7]

This is a breaking change.

It is described above In #5950, so should we change the title prefix from "feat(misconf)" to "BREAKING CHANGE(miscong)"? e.g. #1571

thanks, I updated it.

[simar7]

Update Feb 2024: We've found a better approach to keep and improve the terraform plan scanning functionality. As it turns out, we can unzip the plan contents (it is a zip file) and parse the HCL directly from it. This allows us to have a functionally complete HCL input which we can scan and flag for misconfigurations.

I've updated this issue's description and title to reflect the above. The PR to improve this functionality is here: #6176