add smart remediation system tests

[matthyx]

Type

enhancement, tests

---

Description

• Added SmartRemediationTests with test cases for various security controls.
• Implemented SmartRemediation class in test scripts to handle smart remediation scenarios.
• Introduced system test mappings for smart remediation tests.
• Minor formatting adjustments in backend_api.py.
• Added support for replacing workloads in Kubernetes test scripts.

---

Changes walkthrough

	Relevant files
Tests	tests.py Integrate Smart Remediation Tests into System Tests            configurations/system/tests.py Imported SmartRemediationTests for system tests. Added SmartRemediationTests to the list of all tests. Included SmartRemediationTests in the test retrieval logic. +4/-0      smart_remediation_tests.py Add Smart Remediation Test Cases for Various Security Controls configurations/system/tests_cases/smart_remediation_tests.py Introduced SmartRemediationTests class with multiple test methods for smart remediation scenarios. Each test method corresponds to a specific security control and includes a test configuration. +103/-0  smart_remediation.py Implement Smart Remediation Test Script                                    tests_scripts/helm/smart_remediation.py Implemented SmartRemediation class extending BaseHelm and BaseKubescape. Added methods for smart remediation test execution, including setup, cleanup, and verification. +122/-0
Formatting	backend_api.py Minor Formatting Adjustments in Backend API                            infrastructure/backend_api.py Minor formatting adjustments (whitespace changes). No significant functional changes detected. +111/-100
Enhancement	statics.py Add Path for Smart Remediation Tests                                          systest_utils/statics.py Added a path for smart remediation tests in statics. +3/-0      base_k8s.py Support Replacing Workloads in Kubernetes Test Scripts      tests_scripts/kubernetes/base_k8s.py Added replace parameter to apply_yaml_file method to support replacing existing workloads. +5/-2
Configuration changes	system_test_mapping.json Add System Test Mappings for Smart Remediation Tests          system_test_mapping.json Added system test mappings for various smart remediation tests. +128/-0

---

✨ PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[codiumai-pr-agent-free]

PR Description updated to latest commit (518b7c9)

[codiumai-pr-agent-free]

PR Review

⏱️ Estimated effort to review [1-5]	4, because the PR introduces a significant amount of new code across multiple files, including new test cases for smart remediation, updates to existing test utilities, and the addition of new Kubernetes workload configurations. The complexity of the changes, especially around the smart remediation tests, requires careful review to ensure correctness and adherence to best practices.
🧪 Relevant tests	Yes
🔍 Possible issues	Possible Bug: The implementation of apply_yaml_file in base_k8s.py now includes a replace parameter. However, the method to delete the workload before applying the new one (self.kubernetes_obj.delete_workload) might not always ensure the workload is fully terminated before the new one is applied. This could lead to race conditions or errors in environments with slower response times. Consider implementing a wait or check to ensure the workload is fully deleted before proceeding.
	Best Practice: Several of the fixed workload YAML files (c0016-fixed.yaml, c0017-fixed.yaml, etc.) include hard-coded values and configurations that may not be universally applicable or best practice for all environments. It's recommended to parameterize these where possible and ensure they adhere to security best practices.
	Code Quality: The SmartRemediation class in smart_remediation.py could benefit from more detailed docstrings explaining the purpose and usage of each method, especially for complex methods like check_smart_remediation and start. This would improve maintainability and understandability of the test cases.
🔒 Security concerns	No

---

✨ Review tool usage guide:

---

Overview:
The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

• When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

• With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

See the review usage page for a comprehensive guide on using this tool.

[codiumai-pr-agent-free]

PR Code Suggestions

Category	Suggestions
Enhancement	Implement exponential backoff for retry mechanism. Consider implementing a retry mechanism with exponential backoff instead of a fixed sleep interval. This approach can be more efficient and reduce the total waiting time for smart remediation to become available. tests_scripts/helm/smart_remediation.py [36-40] +wait_time = 1 for _ in range(retries): hl = self.backend.get_posture_resources_highlights(body) if len(hl["response"]) > 0 and (want == ("smartRemediations" in hl["response"][0])): return True - TestUtil.sleep(10, "wait for smart remediation") + TestUtil.sleep(wait_time, "wait for smart remediation") + wait_time *= 2
	Check workload existence before deletion in replace operation. Before deleting a workload as part of the replace operation, check if the workload exists to avoid potential errors from trying to delete non-existent resources. tests_scripts/kubernetes/base_k8s.py [352-354] if replace: - # cannot use patch because of https://github.com/kubernetes-client/python/issues/1359 - self.kubernetes_obj.delete_workload(application=workload, namespace=namespace) + # Check if workload exists before attempting to delete + if self.kubernetes_obj.workload_exists(application=workload, namespace=namespace): + self.kubernetes_obj.delete_workload(application=workload, namespace=namespace)
	Make sleep time configurable for flexibility. Instead of hardcoding the sleep time, consider making it a parameter or a class attribute to allow for easier adjustments and more flexible testing scenarios. tests_scripts/helm/smart_remediation.py [40] -TestUtil.sleep(10, "wait for smart remediation") +TestUtil.sleep(self.sleep_time, "wait for smart remediation")
Maintainability	Use descriptive variable names for better readability. Use a more descriptive variable name instead of hl to improve code readability. For example, highlights_response would make the code more understandable. tests_scripts/helm/smart_remediation.py [37] -hl = self.backend.get_posture_resources_highlights(body) +highlights_response = self.backend.get_posture_resources_highlights(body)
Robustness	Add error handling for backend service calls. To improve the robustness of the check_smart_remediation method, consider adding error handling for the call to self.backend.get_posture_resources_highlights(body). This can help manage unexpected issues, such as network errors or backend service downtime. tests_scripts/helm/smart_remediation.py [37] -hl = self.backend.get_posture_resources_highlights(body) +try: + hl = self.backend.get_posture_resources_highlights(body) +except Exception as e: + Logger.logger.error(f"Failed to get posture resources highlights: {e}") + return False
Security	Reduce security risks by limiting capabilities and avoiding privileged containers. Granting NET_ADMIN capabilities and setting privileged: true can expose the container to significant security risks. Consider using more specific, minimal capabilities required for your application or exploring other security mechanisms. configurations/k8s_workloads/smart-remediation/c0034-fixed.yaml [17-20] securityContext: capabilities: - add: ["NET_ADMIN"] - privileged: true + add: ["<specific-capabilities-needed>"] + privileged: false
	Avoid mounting the Docker socket to reduce security risks. Mounting the Docker socket inside a container can lead to security vulnerabilities, as it allows the container to control Docker on the host. Consider alternative ways to achieve your goals without direct Docker socket access. configurations/k8s_workloads/smart-remediation/c0057-fixed.yaml [22-23] volumeMounts: - - name: docker-socket - mountPath: /host-docker.sock + - name: <alternative-volume> + mountPath: /<alternative-path>
	Limit access to host files and prefer Kubernetes secrets or config maps for sensitive data. Using hostPath volumes can expose sensitive files on the host to the container. Ensure that the container only has access to the necessary files and consider using Kubernetes secrets or config maps for sensitive data. configurations/k8s_workloads/smart-remediation/nginx-deployment.yaml [31-34] volumes: - - name: host-volume - hostPath: - path: /etc - type: Directory + - name: <config-map-or-secret-name> + configMap: + name: <config-map-name>
	Avoid running containers in privileged mode to enhance security. Running containers in privileged mode grants them access to all devices on the host and removes many of the security mechanisms of containers. If possible, avoid running containers in privileged mode. configurations/k8s_workloads/smart-remediation/c0046-fixed.yaml [18] securityContext: - privileged: true + privileged: false
Best practice	Specify minimal necessary capabilities instead of broad ones like NET_ADMIN. Defining NET_ADMIN capabilities can be too broad and potentially dangerous. It's recommended to specify only the necessary capabilities your application needs to function properly. configurations/k8s_workloads/smart-remediation/c0074-fixed.yaml [18-19] securityContext: capabilities: - add: ["NET_ADMIN"] + add: ["<specific-capabilities-needed>"]

---

✨ Improve tool usage guide:

---

Overview:
The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.

• When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

• With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

See the improve usage page for a comprehensive guide on using this tool.