(DOCSP-32047) New push/pull Admin API endpoints #625

nlarew · 2023-11-06T18:07:59Z

Pull Request Info

Note: also cleans up the nonexistent /services/config endpoints and combines them with the correct /services/{serviceId}/config - this fixes the lint error I was getting!

Jira

(DOCSP-32047) New push/pull Admin API endpoints

Staged Changes

Push
Pull

dacharyc

It looks like there are some discrepancies here with what the API is actually returning. I started listing them out, but they got bigger when I got to data_sources. Ping me on Slack if you'd like and I can share my response object with you, or you can try hitting the pull endpoint and see what you get back.

source/openapi-admin-v3.yaml

dacharyc

Back to you with a handful of questions, including what appears to me to be some bigger discrepancies in the data_sources.

dacharyc · 2023-11-28T15:54:52Z

source/openapi-admin-v3.yaml

+          $ref: "#/components/schemas/CustomUserDataConstructor"
+        data_api_config:
+          $ref: "#/components/schemas/DataApiConfig"
+        data_sources:


According to our API reference, the DataSourceConstructor schema contains one of AtlasClusterConstructorWithRules or AtlasFederatedInstanceConstructor.

My response body for data_sources included the details in the AtlasClusterConstructorWithRules schema, AND the ServiceResponse schema (I had type, name, version, but I did not have _id). Should both of those things be included in the DataSourceConstructorSchema?

dacharyc · 2023-11-28T16:00:59Z

source/openapi-admin-v3.yaml

@@ -5603,7 +5688,24 @@ components:
        error_code:
          type: string
          description: The error type.
+    AtlasClusterConstructorWithRules:
+      allOf:
+        - $ref: "#/components/schemas/AtlasClusterConstructor"


My response body did not match the AtlasClusterConstructor schema config object. My config only had clusterName, readPreference, and wireProtocolEnabled. It did not have clusterName, clusterId, clusterType, groupName, etc. My response body had a sync config outside of this config object - it wasn't in the Atlas cluster config. I'm not sure if this schema is used elsewhere, but it doesn't match what I got from the pull endpoint.

source/openapi-admin-v3.yaml

dacharyc · 2023-11-28T16:30:00Z

source/openapi-admin-v3.yaml

+          description: |-
+            If true, clients may [connect to the app over the
+            MongoDB Wire
+            Protocol](https://www.mongodb.com/docs/atlas/app-services/mongodb/wire-protocol/#connect-over-the-wire-protocol).
    FlexibleSync:


I notice we don't have required/optional on these fields. I'm missing collection_queryable_fields_names and is_recovery_mode_disabled.

I also don't see anything about data ingest, and realize I don't know if you configure data ingest, how it shows up in the config object. I looked around and we don't seem to have anything documented for that. But I assume it shows up in the config somehow.

Should we be more explicit about which of these fields may be absent? For a lot of our other config options, if there are no values, we just get an empty array - for example, my pull had empty arrays for triggers, values, log_forwarders, etc. So I might assume that collection_queryable_fields_names would show up as an empty array, and I'm not sure why is_recovery_mode_disabled is omitted.

Added a bunch of required annotations. I think we'll want to do a separate ticket to look at ingest / sync config stuff

…shpull

dacharyc

Back to you, @nlarew !

source/openapi-admin-v3.yaml

dacharyc · 2024-01-04T13:46:08Z

source/openapi-admin-v3.yaml

+          type: array
+          items:
+            $ref: "#/components/schemas/AuthProviderConstructor"
+        custom_user_data:


My custom_user_data and data_api_config responses are both null. We say in the API doc that these are required properties, and that they use these objects. Should we explicitly state they can be null?

Good catch - not actually required and can be null if unconfigured.

dacharyc · 2024-01-04T13:49:30Z

source/openapi-admin-v3.yaml

      type: object
+      required:
+        - clusterName
+        - clusterType


We list clusterType as a required field, but it's not present in the pull response I got. I am guessing this is not required, or is maybe required for push but not guaranteed to be present in pull?

I think this one snuck in - it's not present anywhere we use this schema. Removed!

dacharyc · 2024-01-04T13:58:27Z

source/openapi-admin-v3.yaml

+              type: string
+              description: The role's unique ObjectId identifier.
+        - $ref: "#/components/schemas/RoleConstructor"
+    RoleConstructor:


The response I got from the pull endpoint had an object that isn't present in our RoleConstructor here: document_filters - which has a read bool and a write object where I've got "owner_id": "%%user.id"

I'm guessing that's optional but I would expect to see it covered here.

I've also got a read bool (which I guess is probably an expression?) at the same level as this write bool, which we don't list here.

I believe document_filters is sync-specific and we don't have good Admin API docs for that atm. I added a reusable schema for them (can reuse when we doc the sync config better) and included that in RoleConstructor + added the non-sync read permission + noted that you should use document_filters if sync is enabled.

dacharyc · 2024-01-04T14:11:05Z

source/openapi-admin-v3.yaml

@@ -7298,9 +7528,12 @@ components:

            - the relationship may point to many foreign documents.
            - the local field must be defined as an array in the collection schema.
+    SchemaDefinition:
+      type: object
+      description: A valid [schema](https://www.mongodb.com/docs/realm/schemas) for the collection.


Is there a reason we don't document the schema object here? The page we link out to has a "Define a Schema" section with an example object: https://www.mongodb.com/docs/atlas/app-services/schemas/#define-a-schema

It struck me as weird that I can view all the other object details here in the API spec, but not schema.

Historically because it's highly dependent on the user's definition. That said there are some invariants that we can include here and use them in a recursive definition - good catch!

dacharyc · 2024-01-04T14:17:51Z

source/openapi-admin-v3.yaml

@@ -6851,7 +7056,7 @@ components:
            type: string
        collection_queryable_fields_names:
          type: object
-          description:  |-
+          description: |-


My Sync config has a handful of fields that aren't represented here (type, development_mode_enabled, service_name, database_name). I notice there's still an outstanding comment from the last review where you suggested a separate ticket to look at ingest/sync config stuff. Does that ticket exist somewhere so we can capture these details?

https://jira.mongodb.org/browse/DOCSP-36931

…shpull

dacharyc

This looks great, Nick! I know this is a massive thing that touched a lot of individual components, and I really appreciate your thoroughness here. Thanks so much!

docs-builder-bot · 2024-02-20T15:10:20Z

✨ Staging URL: https://preview-mongodbmongodb.gatsbyjs.io/atlas-app-services/master/

🪵 Logs

job log: 60c0c04cbad6dd4323d7500737dd6ec9124ed048

nlarew added 4 commits November 1, 2023 16:17

WIP - pushpull schema

58df2f8

WIP - fixing schemas for pushpull

c56f692

WIP - fixing schemas for pushpull

db227fe

(DOCSP-32047) New push/pull Admin API endpoints

a08114b

nlarew force-pushed the pushpull branch from 56a1737 to a08114b Compare November 6, 2023 19:15

nlarew and others added 4 commits November 6, 2023 15:42

Add sync to PushPull

cf15e4b

Merge branch 'master' into pushpull

79bf0a7

plural data_sources

9728882

Merge branch 'master' into pushpull

d766d56

dacharyc reviewed Nov 8, 2023

View reviewed changes

nlarew and others added 7 commits November 27, 2023 12:50

csv and json in description

98825a6

pushpull config updates

676976b

better example

c7cd2fb

pushpull config updates + spec error fixes

ad5dafa

Merge branch 'master' into pushpull

d2caca3

note create secret

627e40c

fix request body

a76c531

dacharyc reviewed Nov 28, 2023

View reviewed changes

nlarew and others added 4 commits December 19, 2023 12:06

Merge branch 'master' of github.com:mongodb/docs-app-services into pu…

b986ad5

…shpull

fix data source constructor + required annotations

6c532c2

more required + fix title

7bb9003

Merge branch 'master' into pushpull

d51044b

nlarew requested a review from dacharyc January 3, 2024 17:00

dacharyc reviewed Jan 4, 2024

View reviewed changes

nlarew added 3 commits February 15, 2024 16:50

anonymous -> anon-user

Loading
Loading status checks…

d6caeaa

DC Review

f184e23

Merge branch 'master' of github.com:mongodb/docs-app-services into pu…

Loading
Loading status checks…

768e299

…shpull

dacharyc approved these changes Feb 16, 2024

View reviewed changes

nlarew merged commit 60c0c04 into mongodb:master Feb 20, 2024
4 checks passed

dacharyc mentioned this pull request Feb 26, 2024

DOCSP-37028 Add docs release notes for w/e Feb 23 #688

Merged

MongoCaleb pushed a commit to MongoCaleb/docs-app-services that referenced this pull request Jun 3, 2024

(DOCSP-32047) New push/pull Admin API endpoints (mongodb#625)

47d887e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(DOCSP-32047) New push/pull Admin API endpoints #625

(DOCSP-32047) New push/pull Admin API endpoints #625

nlarew commented Nov 6, 2023 •

edited

Loading

dacharyc left a comment

dacharyc left a comment

dacharyc Nov 28, 2023

dacharyc Nov 28, 2023

dacharyc Nov 28, 2023

nlarew Dec 19, 2023

dacharyc left a comment

dacharyc Jan 4, 2024

nlarew Feb 16, 2024

dacharyc Jan 4, 2024

nlarew Feb 16, 2024

dacharyc Jan 4, 2024

dacharyc Jan 4, 2024

nlarew Feb 16, 2024

dacharyc Jan 4, 2024

nlarew Feb 16, 2024

dacharyc Jan 4, 2024

nlarew Feb 16, 2024

dacharyc left a comment

docs-builder-bot commented Feb 20, 2024

(DOCSP-32047) New push/pull Admin API endpoints #625

(DOCSP-32047) New push/pull Admin API endpoints #625

Conversation

nlarew commented Nov 6, 2023 • edited Loading

Pull Request Info

Jira

Staged Changes

dacharyc left a comment

Choose a reason for hiding this comment

dacharyc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dacharyc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dacharyc left a comment

Choose a reason for hiding this comment

docs-builder-bot commented Feb 20, 2024

🪵 Logs

nlarew commented Nov 6, 2023 •

edited

Loading