Handling changing registration purpose #2436

andrea-williams · 2024-10-31T17:41:38Z

Note that this is only for the backend changes required. The frontend functionality to allow for changing registration purpose after the operation has already registered will follow in a subsequent PR.

added test_changing_registration_purposes.py file that tests for every possible scenario of changing registration purpose from A to B, for both operation types (LFO and SFO). Also accounts for whether the operation has a status of "Registered" (in which case no-longer-relevant database records will be archived) or not registered (in which case no-longer-relevant records are deleted).

BCerki

Thanks for taking on this monster, Andrea! If you want to go over anything, let me know--I ~think I followed things ok, but I may have made irrelevant comments where my brain couldn't keep up with yours

bc_obps/service/operation_service_v2.py

BCerki · 2024-12-18T22:45:06Z

bc_obps/registration/tests/integration/test_changing_registration_purpose.py

+        assert self.operation.updated_by == self.user
+        assert self.operation.updated_at > self.operation.created_at
+
+        match original_purpose:


If we go with a reorg like I suggested above (or a different, better one you come up with), I think we can avoid the match/case and the test will be easier to read

BCerki · 2024-12-18T22:50:53Z

bc_obps/registration/tests/integration/test_changing_registration_purpose.py

+        if new_purpose == Operation.Purposes.OPTED_IN_OPERATION:
+            self._set_opted_in_operation_detail()
+        elif new_purpose == Operation.Purposes.NEW_ENTRANT_OPERATION:
+            self._set_new_entrant_info()


Just want to confirm I understand how this would work in the app to make sure this test is representative. Say I'm on the operation page and click edit. I change the registration purpose, and at that point the opt-in/new entrant fields pop up. If I try to submit, I won't be able to until I fill in those fields. Correct? (Vs. how it is in the registration workflow where purpose and opt-in/new entrant are in separate steps and separate API calls.)

There's 2 possible scenarios. If registration_status == REGISTERED, this test mimics changing the purpose from the Operation Details page, in which case the new fields should display in the "Registration Information" section of the page and will need to be filled in before the user can click Save.

If registration_status == DRAFT, this test mimics a user going through the registration workflow, and at some point (before submitting their registration) going back to the first step to change the registration purpose, in which case a new section might be added to the flow, depending on which new purpose is chosen.

BCerki · 2024-12-18T22:52:23Z

bc_obps/registration/tests/integration/test_changing_registration_purpose.py

+    def _create_new_entrant_payload(self):
+        return {"date_of_first_shipment": "On or after April 1, 2024", "new_entrant_application": MOCK_DATA_URL}
+
+    def _prepare_test_data(self, registration_purpose, type):


I'm not convinced we need this function or _set_operation_information. I think we could achieve the same setup with baker recipes

I agree that we probably could achieve the same with baker recipes. In creating them this way, I was trying to (a) be more consistent with the structure of the other integration test (for operation registration) and (b) replicate the ordering of steps as they happen when a user is going through the registration workflow.

bc_obps/registration/tests/integration/test_changing_registration_purpose.py

bc_obps/service/operation_service_v2.py

BCerki · 2025-01-15T20:16:55Z

bc_obps/service/tests/test_operation_service_v2.py

@@ -65,6 +66,7 @@ def set_up_valid_mock_operation(purpose: Operation.Purposes):
            'utils.document', type=DocumentType.objects.get(name='new_entrant_application')
        )
        operation.documents.add(new_entrant_application)
+        operation.date_of_first_shipment = "On or after April 1, 2024"


Do we need an operation.save() after this?

no, we don't even need the operation.save() I inserted a few lines below, since the function is returning the operation. The caller of the this function saves the operation.

BCerki · 2025-01-15T20:23:18Z

bc_obps/service/tests/data_access_service/test_data_access_opted_in_operation_detail_service.py

+            approved_user_operator.user.user_guid, users_operation.id
+        )
+        users_operation.refresh_from_db()
+        # at the moment there's no easy way to tell whether a record has been deleted or archived, so not testing any further than


The way you did it in cas-registration/bc_obps/service/tests/test_document_service.py might work here too?

if registration_status == Operation.Statuses.REGISTERED: """if the registration has been completed, the document should have been archived""" b_map.refresh_from_db() assert b_map.archived_at is not None assert b_map.archived_by is not None elif registration_status == Operation.Statuses.DRAFT: """if the registration wasn't completed, the document should have been deleted""" with pytest.raises(Document.DoesNotExist): b_map.refresh_from_db() # this should raise an exception because it no longer exists in the db

BCerki · 2025-01-15T20:28:45Z

bc_obps/service/operation_service_v2.py

@@ -235,7 +274,7 @@ def create_or_update_operation_v2(
        operation.documents.add(*operation_documents)

        if operation.registration_purpose == Operation.Purposes.OPTED_IN_OPERATION:
-            operation = cls.create_opted_in_operation_detail(user_guid, operation)
+            operation = cls.create_opted_in_operation_detail(user_guid, operation.id)


If I'm following correctly, handle_change_of_registration_purpose only archives/deletes data that's no longer needed for an old purpose. It doesn't add any of the new data--that adding happens in this create_or_update_operation_v2.

I think this function may be missing the update for operation detail (unless it's in another service somewhere; I only see create here).

see comment https://github.com/bcgov/cas-registration/pull/2436/files#r1917469882 - opted_in_operation detail is only ever being created here because we don't have any opt-in data available at this point - that gets called in a different function later in the workflow.

BCerki · 2025-01-15T20:40:08Z

bc_obps/service/operation_service_v2.py

@@ -261,10 +300,10 @@ def register_operation_information(
        )

        if operation.registration_purpose == Operation.Purposes.OPTED_IN_OPERATION:
-            # TODO in ticket 2169 - replace this with create_or_update_-----


This note makes me think the update should be here too

I don't think so. I think the source of confusion is that when we're calling create_opted_in_operation_detail(), we only have the operation's id and know that it's RP is Opt-in. We don't have any data about the opt-in yet, so this is only creating a blank OptedInOperationDetail record in the database - it doesn't get populated with data until later (when update_opted_in_operation_detail gets called).

This is why create_opted_in_operation_detail is a completely separate function from update_opted_in_operation_detail, because the payload data is so different. (I know this is different behaviour from what we've typically done elsewhere in Reg, where create_or_update_x is just one function. But this strategy for opt-ins was written before this PR, and I think it's out of scope to refactor it now.

Makes sense, thanks!

BCerki · 2025-01-15T20:40:56Z

bc_obps/service/operation_service_v2.py

+        Logic to handle the situation when an industry user changes the selected registration purpose (RP) for their operation.
+        Changing the RP can happen during or after submitting the operation's registration info.
+        Depending on what the old RP was, some operation data may need to be removed.
+        Depending on what the new RP is, some new operation data may need to be added.


Depending on what the new RP is, some new operation data may need to be added.--I only see things being archived/deleted, not added. If that's the intention, we could drop this line from the docstring

Fair - the only data being "added" in switching the opt_in flag to false. I'll remove this line from the docstring.

BCerki · 2025-01-15T21:17:54Z

bc_obps/registration/tests/integration/test_changing_registration_purpose.py

+        operation_information_payload = {
+            "boundary_map": MOCK_DATA_URL,
+            "process_flow_diagram": MOCK_DATA_URL,
+            "activities": [2, 3],
+            "naics_code_id": mock_naics_codes[1].id,
+            "secondary_naics_code_id": mock_naics_codes[2].id,
+            "name": self.operation.name,
+            "type": self.operation.type,
+            "registration_purpose": self.operation.registration_purpose,
+            "regulated_products": [1, 2],
+        }


I think this comment is still relevant--this payload shouldn't be used to set up Potential Reporting, Reporting, or EIO because it contains things those purposes don't have (e.g. regulated products)

BCerki · 2025-01-15T21:29:25Z

bc_obps/registration/tests/integration/test_changing_registration_purpose.py

+    def _test_new_entrant_to_reporting(self):
+        """
+        Tests operation registration data for situation where registration_purpose changes from New Entrant to Reporting Operation.
+        Registration data specific to new entrants (date of first shipment, new entrant application document) should be removed.


~~b don't see~~ this was a note to self I forgot to remove, sorry for being confusing

BCerki

Amazing work!
Re my flip-floppy thoughts on the test structure:

There are so many combinations of tos and froms that I'm not confident I noticed if a check for a particular field was missing. I commented when I saw the missing regulated products, but I don't know that I got every one, and as we try to maintain this test, that might continue to be an issue. The mock data wasn't set up completely correctly at first and everything still passed
this PR is huge and has taken a lot of time already, so maybe ^ is tech debt to kick down the road. I don't mind tackling it--I ~think I have a strategy that isn't too overwhelming

andrea-williams · 2025-01-17T00:22:29Z

Amazing work! Re my flip-floppy thoughts on the test structure:

There are so many combinations of tos and froms that I'm not confident I noticed if a check for a particular field was missing. I commented when I saw the missing regulated products, but I don't know that I got every one, and as we try to maintain this test, that might continue to be an issue. The mock data wasn't set up completely correctly at first and everything still passed

this PR is huge and has taken a lot of time already, so maybe ^ is tech debt to kick down the road. I don't mind tackling it--I ~think I have a strategy that isn't too overwhelming

thanks Brianna! I greatly appreciate how much time and effort you put into reviewing this. You did a fantastic job catching my goofs!

…n multiples are required; fixed bug where reporting ops were also being marked as regulated erroneously

… of changing reg purpose - more refinement likely needed. Lots of failures

…O as original RP bc of doc count that's haunting me

…lete

…rvice

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch 5 times, most recently from a862acb to 4735cb1 Compare November 7, 2024 00:30

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch from 4735cb1 to 40d36f0 Compare November 14, 2024 21:02

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch 3 times, most recently from ae804fb to 491151c Compare November 27, 2024 01:28

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch from 491151c to 4541316 Compare November 29, 2024 01:13

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch 4 times, most recently from ec92785 to b082037 Compare December 13, 2024 01:44

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch 2 times, most recently from 330dd49 to ddf6d7a Compare December 17, 2024 00:47

BCerki reviewed Dec 18, 2024

View reviewed changes

andrea-williams marked this pull request as ready for review January 6, 2025 16:32

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch 3 times, most recently from 4a36d22 to 34213f3 Compare January 10, 2025 02:12

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch 2 times, most recently from 01abcc9 to aa8f2eb Compare January 15, 2025 01:12

BCerki reviewed Jan 15, 2025

View reviewed changes

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch 2 times, most recently from e25b3f3 to f0053e1 Compare January 16, 2025 01:14

BCerki approved these changes Jan 16, 2025

View reviewed changes

andrea-williams added 2 commits January 16, 2025 16:22

Chore: refactor to simplify logic of setting registration purpose whe…

d56fe15

…n multiples are required; fixed bug where reporting ops were also being marked as regulated erroneously

Rebase

f634534

andrea-williams added 27 commits January 16, 2025 16:23

Chore: in progress

fd6d8d3

chore: integration test for changing reg purpose - in progress

51a9c4c

chore: integration test for changing reg purpose - in progress

fcb28a5

chore: approx. 50% finished writing unit tests for changing reg purposes

940f92d

chore: completed first attempt at making unit tests for all scenarios…

b4d221d

… of changing reg purpose - more refinement likely needed. Lots of failures

chore: backend logic to change reg purpose in progress

3ec3a7c

Chore: in progress - deletion of OptedInOperationDetail object

ff73d8c

chore: in progress

9a953cf

chore: changing RP in progress. All tests pass except for wretched EI…

36f796d

…O as original RP bc of doc count that's haunting me

chore: in progress

713bde7

chore: in progress

8104d57

chore: all tests pass - is mypy happy now?

fb5bd67

chore: remove changes to FE code - will be addressed in subsequent PR

013cdd6

chore: removed reg_purpose ComboBox - to be addressed in subsequent PR

252f3dd

chore: addressed TODO comments

58e5036

chore: implemented some of the code review feedback

a22b86e

chore: implemented more PR review feedback

380e89d

chore: added unit tests for is_operation_new_entrant_information_comp…

ff5d2ca

…lete

chore: added unit test for archive_or_delete_opted_in_operation_detail

fe164db

chore: simplified test_remove_opted_in_operation_detail

01b3c7b

chore: refactors to operation_service_v2 based on PR feedback

04d31e5

chore: added basic unit tests for handle_change_of_registration_purpose

d3cd749

chore: implemented more PR feedback

0c8b422

chore: addressed all PR review feedback (I think)

13cf582

chore: addressed some PR review feedback

7caa442

chore: updated unit test for data_access_opted_in_operation_detail_se…

009be86

…rvice

chore: PR feedback - don't set NAICS codes for EIOs

b4be968

andrea-williams force-pushed the task/changing-reg-purpose/2169 branch from 903d941 to b4be968 Compare January 17, 2025 00:24

andrea-williams merged commit a287ae4 into develop Jan 17, 2025
42 checks passed

andrea-williams deleted the task/changing-reg-purpose/2169 branch January 17, 2025 00:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handling changing registration purpose #2436

Handling changing registration purpose #2436

andrea-williams commented Oct 31, 2024 •

edited

Loading

BCerki left a comment

BCerki Dec 18, 2024

BCerki Dec 18, 2024

andrea-williams Jan 15, 2025

BCerki Dec 18, 2024

andrea-williams Jan 15, 2025

BCerki Jan 15, 2025

andrea-williams Jan 15, 2025

BCerki Jan 15, 2025

andrea-williams Jan 16, 2025

BCerki Jan 15, 2025

andrea-williams Jan 16, 2025

BCerki Jan 15, 2025

andrea-williams Jan 15, 2025

BCerki Jan 16, 2025

BCerki Jan 15, 2025

andrea-williams Jan 15, 2025

BCerki Jan 15, 2025

BCerki Jan 15, 2025 •

edited

Loading

BCerki left a comment

andrea-williams commented Jan 17, 2025

Handling changing registration purpose #2436

Handling changing registration purpose #2436

Conversation

andrea-williams commented Oct 31, 2024 • edited Loading

BCerki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BCerki Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

BCerki left a comment

Choose a reason for hiding this comment

andrea-williams commented Jan 17, 2025

andrea-williams commented Oct 31, 2024 •

edited

Loading

BCerki Jan 15, 2025 •

edited

Loading