From e8b53fbbd819d0c2f3698daba6e14ecf6c0b53b6 Mon Sep 17 00:00:00 2001
From: Jiri Jaburek <comps@nomail.dom>
Date: Wed, 9 Oct 2024 14:56:13 +0200
Subject: [PATCH] make audit-sample-rules into a diff-style tool

Based on recent discussions, turn the test into a diff-like tool
the output of which is to be always reviewed by a human.

This is in contrast to the old (current) state which treats the
test as a regression test, assuming it is meant to normally 'pass'.

And that would make sense if we wanted to keep audit rule lists stored
in the content identical with auditd samples, but we (CaC/content)
do not want that.

Instead, the intention is to just alert about any differences, and let
the humans decide whether they are significant enough to update the
CaC/content audit rules, and how.

Signed-off-by: Jiri Jaburek <comps@nomail.dom>
---
 conf/waivers/20-long-term                            | 12 ------------
 static-checks/audit-sample-rules/main.fmf            |  8 --------
 .../test.py => diff/audit-sample-rules.py}           |  0
 static-checks/diff/main.fmf                          |  6 ++++++
 4 files changed, 6 insertions(+), 20 deletions(-)
 delete mode 100644 static-checks/audit-sample-rules/main.fmf
 rename static-checks/{audit-sample-rules/test.py => diff/audit-sample-rules.py} (100%)

diff --git a/conf/waivers/20-long-term b/conf/waivers/20-long-term
index 3c38af1..af8fe6d 100644
--- a/conf/waivers/20-long-term
+++ b/conf/waivers/20-long-term
@@ -121,18 +121,6 @@
 /per-rule/.*/set_nftables_table/.*
     rhel == 9
 
-# /static-checks
-#
-# new arch= on RHEL-9.5
-# https://github.com/ComplianceAsCode/content/issues/12321
-/static-checks/audit-sample-rules/audit_ospp_general.*
-    rhel == 9.5
-# older issues with pkexec and grub2-set-bootflag, they seem to appear
-# only with older audit-3.0.x
-# https://github.com/ComplianceAsCode/content/issues/12321
-/static-checks/audit-sample-rules/audit_ospp_general.*
-    (rhel == 9 and rhel <= 9.3) or rhel == 8.8
-
 # RHEL10 - No official RHEL10 STIG/CIS benchmark yet
 /static-checks/rule-identifiers/(stig|cis.*)/.*
     rhel == 10
diff --git a/static-checks/audit-sample-rules/main.fmf b/static-checks/audit-sample-rules/main.fmf
deleted file mode 100644
index e06ed5b..0000000
--- a/static-checks/audit-sample-rules/main.fmf
+++ /dev/null
@@ -1,8 +0,0 @@
-summary: Compares audit.rules between Content and auditd samples
-test: python3 -m lib.runtest ./test.py
-result: custom
-environment+:
-    PYTHONPATH: ../..
-duration: 5m
-require+:
-  - audit
diff --git a/static-checks/audit-sample-rules/test.py b/static-checks/diff/audit-sample-rules.py
similarity index 100%
rename from static-checks/audit-sample-rules/test.py
rename to static-checks/diff/audit-sample-rules.py
diff --git a/static-checks/diff/main.fmf b/static-checks/diff/main.fmf
index 5c95a8b..b3c39b1 100644
--- a/static-checks/diff/main.fmf
+++ b/static-checks/diff/main.fmf
@@ -24,3 +24,9 @@ adjust:
 /profile-variables:
     summary: Diff datastreams, output profile variable refine differences
     test: python3 -m lib.runtest ./profile-variables.py
+
+/audit-sample-rules:
+    summary: Diff audit.rules between Content and installed auditd samples
+    test: python3 -m lib.runtest ./audit-sample-rules.py
+    require+:
+      - audit