Remove duplicate EC2 cleanup jobs #1151
Conversation
| @@ -1,20 +0,0 @@ | |||
| { | |||
| "statements": [ | |||
There was a problem hiding this comment.
This is covered by autocleanups in models.aws.ec2.keypairs
There was a problem hiding this comment.
Really awesome that we can do this partial refactor
| @@ -1,10 +1,5 @@ | |||
| { | |||
| "statements": [ | |||
| { | |||
| "query": "MATCH (n:EC2SecurityGroup)<-[:RESOURCE]-(:AWSAccount{id: $AWS_ID}) WHERE n.lastupdated <> $UPDATE_TAG WITH n LIMIT $LIMIT_SIZE DETACH DELETE (n)", | |||
There was a problem hiding this comment.
This is covered by autocleanups in models.aws.ec2.securitygroups. Additionally the autocleanups also handle a previously unhandled case where we need to cleanup stale rels between SGs and instances.
| @@ -1,20 +0,0 @@ | |||
| { | |||
| "statements": [ | |||
There was a problem hiding this comment.
These are covered by autocleanups in models.aws.ec2.volumes.
| @@ -20,25 +20,10 @@ | |||
| "iterationsize": 100, | |||
| "iterative": true | |||
| }, | |||
| { | |||
| "query": "MATCH (:AWSAccount{id: $AWS_ID})-[:RESOURCE]->(:EC2Instance)-[r:PART_OF_SUBNET]->(:EC2Subnet) WHERE r.lastupdated <> $UPDATE_TAG WITH r LIMIT $LIMIT_SIZE DELETE (r)", | |||
There was a problem hiding this comment.
This is covered by autocleanups in models.aws.ec2.subnets.
| { | ||
| "query": "MATCH (:AWSAccount{id: $AWS_ID})-[:RESOURCE]->(:LoadBalancer)-[r:PART_OF_SUBNET]->(:EC2Subnet) WHERE r.lastupdated <> $UPDATE_TAG WITH r LIMIT $LIMIT_SIZE DELETE (r)", | ||
| "iterationsize": 100, | ||
| "iterative": true | ||
| }, | ||
| { | ||
| "query": "MATCH (:AWSAccount{id: $AWS_ID})-[:RESOURCE]->(n:NetworkInterface) WHERE n.lastupdated <> $UPDATE_TAG WITH n LIMIT $LIMIT_SIZE DETACH DELETE (n)", |
There was a problem hiding this comment.
This is covered by autocleanups in models.aws.ec2.networkinterfaces.
| @@ -308,11 +308,6 @@ def cleanup(neo4j_session: neo4j.Session, common_job_parameters: Dict[str, Any]) | |||
| logger.debug("Running EC2 instance cleanup") | |||
| GraphJob.from_node_schema(EC2ReservationSchema(), common_job_parameters).run(neo4j_session) | |||
| GraphJob.from_node_schema(EC2InstanceSchema(), common_job_parameters).run(neo4j_session) | |||
| GraphJob.from_node_schema(EC2SubnetSchema(), common_job_parameters).run(neo4j_session) | |||
There was a problem hiding this comment.
These are all moved from here to their respective modules. Specifically,
- EC2SubnetSchema -> intel.aws.ec2.subnets.cleanup_subnets
- EC2SecurityGroupSchema -> intel.aws.ec2.security_groups.cleanup_ec2_security_groupinfo
- EC2KeyPairSchema -> intel.aws.ec2.key_pairs.cleanup_ec2_key_pairs
- EC2NetworkInterfaceSchema -> intel.aws.ec2.network_interfaces.cleanup_network_interfaces
- EBSVolumeSchema -> intel.aws.ec2.cleanup_volumes
| @@ -1,20 +0,0 @@ | |||
| { | |||
| "statements": [ | |||
There was a problem hiding this comment.
Really awesome that we can do this partial refactor
| @@ -146,6 +148,7 @@ def cleanup_ec2_security_groupinfo(neo4j_session: neo4j.Session, common_job_para | |||
| neo4j_session, | |||
| common_job_parameters, | |||
| ) | |||
| GraphJob.from_node_schema(EC2SecurityGroupSchema(), common_job_parameters).run(neo4j_session) | |||
There was a problem hiding this comment.
We still need to keep both, for now, right?
There was a problem hiding this comment.
Yes this is intentional because the other resources cleaned up in aws_import_ec2_security_groupinfo_cleanup.json have not been refactored to the new model yet.
Fast follow of cartography-cncf#1146 - removes duplicate calls to cleanup jobs. All these manual jobs will soon be deprecated as we refactor modules to the new data model.
Fast follow of #1146 - removes duplicate calls to cleanup jobs. All these manual jobs will soon be deprecated as we refactor modules to the new data model.