💩 🔧 Enable suspending data collection for certain subgroups

[shankari]

This implements a hack for e-mission/e-mission-docs#1104
The request in the issue was to implement the ability to suspend and resume
data collection.

As a first step for implementing this, we:

• add a new section to the opcode in the dynamic config that lists the
  suspended subgroups, and
• skip all calls to usercache/put for users that are in suspended subgroup

This is a hack because the real fix is to stop data collection on the phone.

But even with this implementation:

• I have copied over the implementation of getSubgroupFromToken from the phone and manually ported it over to python. Instead, we should have all the implementations for parsing the opcode in e-mission-common and just use them here
• ideally, we would check for the subgroup being suspended in the putIntoCache method. But, by design, we only see the opcode in the auth module; the rest of the code works only with UUIDs. We should figure out a way to return the non-random parts of the opcode (program and subgroup) as a context so we can do additional validation downstream if needed. We can return this as a context variable, or store it into the profile and have the code check it from there.

@JGreenlee @TeachMeTW we should discuss if we want to continue with this hack on the server, or just go to the principled fix, in which case, most of these 💩 changes can be reverted.

[shankari]

Testing done:

After changing the default config

STUDY_CONFIG = os.getenv('STUDY_CONFIG', "dev-emulator-study")

and changing the dev-emulator-study config to have the new-style opcode with
a suspended subgroup

    "opcode": {
        "autogen": true,
        "subgroups": [
          "test",
          "default"
        ],
        "suspended_subgroups": ["default"]
    },

The suspended subgroup skipped all put messages

2025-02-02 18:57:35,723:DEBUG:6363951104:START POST /usercache/put
2025-02-02 18:57:35,724:DEBUG:6363951104:Called userCache.put
2025-02-02 18:57:35,742:DEBUG:6363951104:subgroup default found in list ['test', 'default']
2025-02-02 18:57:35,743:INFO:6363951104:Received put message for subgroup default in suspended
_subgroups=['default'], returning uuid = None
2025-02-02 18:57:35,743:DEBUG:6363951104:retUUID = None
2025-02-02 18:57:35,743:DEBUG:6380777472:START POST /usercache/get
2025-02-02 18:57:35,743:DEBUG:6363951104:END POST /usercache/put  0.023092985153198242

The non-suspended subgroup used the default implementation of put

2025-02-02 19:15:42,723:DEBUG:13035925504:START POST /usercache/put
2025-02-02 19:15:42,726:DEBUG:13035925504:Called userCache.put
2025-02-02 19:15:42,750:DEBUG:13035925504:subgroup test found in list ['test', 'default']
2025-02-02 19:15:42,759:DEBUG:13035925504:Updated result for user = feb70456-abf4-444b-8848-1515fc3470cf, key = stats/client_time, write_ts = 1738551626.711804 = {'n': 1, 'nModified': 0, 'upserted': ObjectId('67a034de46756f8244ede842'), 'ok': 1.0, 'updatedExisting': False}
2025-02-02 19:15:42,761:DEBUG:13035925504:Updated result for user = feb70456-abf4-444b-8848-1515fc3470cf, key = stats/client_time, write_ts = 1738551626.71355 = {'n': 1, 'nModified': 0, 'upserted': ObjectId('67a034de46756f8244ede844'), 'ok': 1.0, 'updatedExisting': False}

After modifying the config to remove suspended_subgroups (default case)

    "opcode": {
        "autogen": true,
        "subgroups": [
          "test",
          "default"
        ],
    },

2025-02-02 21:08:26,082:DEBUG:13052751872:START POST /usercache/put
2025-02-02 21:08:26,082:DEBUG:13052751872:Called userCache.put
2025-02-02 21:08:26,088:DEBUG:13052751872:subgroup default found in list ['test', 'default']
2025-02-02 21:08:26,088:DEBUG:13052751872:methodName = skip, returning <class 'emission.net.auth.skip.SkipMethod'>
2025-02-02 21:08:26,088:DEBUG:13052751872:Using the skip method to verify id token nrelop_dev-emulator-study_default_123 of length 37

2025-02-02 21:08:26,110:DEBUG:13052751872:Updated result for user = eb4a7aae-f2d4-4480-ba85-c568a45591b5, key = background/location, write_ts = 1738559116.0185199 = {'n': 1, 'nModified': 0, 'upserted': ObjectId('67a04f4a46756f8244ee3a88'), 'ok': 1.0, 'updatedExisting': False}
2025-02-02 21:08:26,112:DEBUG:13052751872:Updated result for user = eb4a7aae-f2d4-4480-ba85-c568a45591b5, key = background/filtered_location, write_ts = 1738559116.020091 = {'n': 1, 'nModified': 0, 'upserted': ObjectId('67a04f4a46756f8244ee3a8a'), 'ok': 1.0, 'updatedExisting': False}
...
2025-02-02 21:08:26,930:DEBUG:13052751872:Updated result for user = eb4a7aae-f2d4-4480-ba85-c568a45591b5, key = stats/client_nav_event, write_ts = 1738559306.0461679 = {'n': 1, 'nModified': 0, 'upserted': ObjectId('67a04f4a46756f8244ee3da8'), 'ok': 1.0, 'updatedExisting': False}
2025-02-02 21:08:26,930:DEBUG:13052751872:END POST /usercache/put eb4a7aae-f2d4-4480-ba85-c568a45591b5 0.849830150604248

[shankari]

@mattwigway the hack has been merged now. It will be on staging for a couple of days, and then we will move it to production, likely by mid-week. After it is in production, you can choose to edit the config and suspend the subgroup at any time.

[mattwigway]

Great, thanks so much!

…

On Mon, Feb 3, 2025, 1:54 AM K. Shankari ***@***.***> wrote: Merged #1016 <#1016> into master. — Reply to this email directly, view it on GitHub <#1016 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEKNLVXKXFRBEXBF4DUTPL2N4HDFAVCNFSM6AAAAABWLH7FNOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJWGE2TMOBXGUZDSMQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>