Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to generate client schema when adding external RDS database #13885

Closed
3 tasks done
outraday opened this issue Oct 4, 2024 · 1 comment
Closed
3 tasks done

Unable to generate client schema when adding external RDS database #13885

outraday opened this issue Oct 4, 2024 · 1 comment
Labels
Gen 2 Issues related to Gen 2 Amplify projects GraphQL Related to GraphQL API issues

Comments

@outraday
Copy link

outraday commented Oct 4, 2024

Before opening, please confirm:

JavaScript Framework

React

Amplify APIs

GraphQL API, DataStore

Amplify Version

v6

Amplify Categories

api

Backend

Amplify Gen 2 (Preview)

Environment information

# Put output below this line
System:
    OS: Windows 10 10.0.19045
    CPU: (4) x64 AMD Ryzen 3 3200G with Radeon Vega Graphics
    Memory: 1.74 GB / 15.95 GB
  Binaries:
    Node: 18.20.3 - C:\Program Files\nodejs\node.EXE
    npm: 10.8.3 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Chromium (127.0.2651.74)
    Internet Explorer: 11.0.19041.4355
  npmPackages:
    %name%:  0.1.0
    @aws-amplify/backend: ^1.2.1 => 1.2.1
    @aws-amplify/backend-cli: ^1.2.5 => 1.2.5
    @aws-amplify/ui-react: ^6.1.12 => 6.1.13
    @aws-amplify/ui-react-internal:  undefined ()
    @aws-amplify/ui-react-storage: ^3.1.4 => 3.1.4
    @aws-sdk/client-cognito-identity-provider: ^3.614.0 => 3.614.0
    @babel/preset-env: ^7.24.7 => 7.24.7
    @babel/preset-react: ^7.24.7 => 7.24.7
    @babel/preset-typescript: ^7.24.7 => 7.24.7
    @dnd-kit/core: ^6.1.0 => 6.1.0
    @dnd-kit/modifiers: ^7.0.0 => 7.0.0
    @dnd-kit/sortable: ^8.0.0 => 8.0.0
    @dnd-kit/utilities: ^3.2.2 => 3.2.2
    @emoji-mart/react: ^1.1.1 => 1.1.1
    @emotion/styled: ^11.11.0 => 11.11.5
    @floating-ui/dom: ^1.5.3 => 1.6.7 (0.5.4)
    @floating-ui/react: ^0.26.5 => 0.26.19 (0.19.2)
    @fluentui/react-context-selector: ^9.1.65 => 9.1.65
    @lexical/clipboard: ^0.12.6 => 0.12.6
    @lexical/code: ^0.12.5 => 0.12.6
    @lexical/hashtag: ^0.12.6 => 0.12.6
    @lexical/link: ^0.12.5 => 0.12.6
    @lexical/list: ^0.12.5 => 0.12.6
    @lexical/mark: ^0.12.5 => 0.12.6
    @lexical/markdown: ^0.12.5 => 0.12.6
    @lexical/react: ^0.12.5 => 0.12.6
    @lexical/rich-text: ^0.12.5 => 0.12.6
    @lexical/selection: ^0.12.5 => 0.12.6
    @lexical/utils: ^0.12.5 => 0.12.6
    @lingui/cli: ^4.11.2 => 4.11.2
    @lingui/core: ^4.5.0 => 4.11.2
    @lingui/detect-locale: ^4.5.0 => 4.11.2
    @lingui/format-po: ^4.5.0 => 4.11.2
    @lingui/macro: ^4.5.0 => 4.11.2
    @lingui/react: ^4.5.0 => 4.11.2
    @lingui/vite-plugin: ^4.11.2 => 4.11.2
    @reduxjs/toolkit: ^2.2.6 => 2.2.6
    @reduxjs/toolkit-query:  1.0.0
    @reduxjs/toolkit-query-react:  1.0.0
    @reduxjs/toolkit-react:  1.0.0
    @tailwindcss/line-clamp: ^0.4.4 => 0.4.4
    @tanstack/eslint-plugin-query: ^5.35.6 => 5.50.1
    @tanstack/query-codemods:  4.24.3
    @tanstack/react-query: ^5.40.0 => 5.50.1
    @tanstack/react-table: ^8.12.0 => 8.19.2
    @tanstack/react-virtual: ^3.3.0 => 3.8.2
    @types/emoji-mart: ^3.0.14 => 3.0.14
    @types/ip: ^1.1.3 => 1.1.3
    @types/lodash: ^4.14.200 => 4.17.6
    @types/node: ^20 => 20.14.10
    @types/react-resizable: ^3.0.8 => 3.0.8
    @types/ua-parser-js: ^0.7.39 => 0.7.39
    @typescript-eslint/eslint-plugin: ^7.13.1 => 7.16.0 (5.62.0)
    @typescript-eslint/parser: ^7.13.1 => 7.16.0 (5.62.0, 6.21.0)
    @vitejs/plugin-react: ^4.3.1 => 4.3.1
    alphanumeric-id: ^1.0.1 => 1.0.1
    antd: ^5.11.1 => 5.19.1
    aws-amplify: ^6.6.0 => 6.6.0
    aws-amplify/adapter-core:  undefined ()
    aws-amplify/analytics:  undefined ()
    aws-amplify/analytics/kinesis:  undefined ()
    aws-amplify/analytics/kinesis-firehose:  undefined ()
    aws-amplify/analytics/personalize:  undefined ()
    aws-amplify/analytics/pinpoint:  undefined ()
    aws-amplify/api:  undefined ()
    aws-amplify/api/server:  undefined ()
    aws-amplify/auth:  undefined ()
    aws-amplify/auth/cognito:  undefined ()
    aws-amplify/auth/cognito/server:  undefined ()
    aws-amplify/auth/enable-oauth-listener:  undefined ()
    aws-amplify/auth/server:  undefined ()
    aws-amplify/data:  undefined ()
    aws-amplify/data/server:  undefined ()
    aws-amplify/datastore:  undefined ()
    aws-amplify/in-app-messaging:  undefined ()
    aws-amplify/in-app-messaging/pinpoint:  undefined ()
    aws-amplify/push-notifications:  undefined ()
    aws-amplify/push-notifications/pinpoint:  undefined ()
    aws-amplify/storage:  undefined ()
    aws-amplify/storage/s3:  undefined ()
    aws-amplify/storage/s3/server:  undefined ()
    aws-amplify/storage/server:  undefined ()
    aws-amplify/utils:  undefined ()
    aws-cdk: ^2.148.0 => 2.148.0
    aws-cdk-lib: ^2.148.0 => 2.148.0
    aws-sdk: ^2.1667.0 => 2.1667.0
    axios: ^1.7.2 => 1.7.2
    babel-plugin-macros: ^3.1.0 => 3.1.0
    clsx: ^2.1.0 => 2.1.1 (1.1.1, 1.2.1)
    constructs: ^10.3.0 => 10.3.0
    dayjs: ^1.11.10 => 1.11.11
    emoji-mart: ^5.5.2 => 5.6.0
    emoji-picker-react: ^4.5.16 => 4.11.1
    esbuild: ^0.23.0 => 0.23.0 (0.17.19, 0.21.5)
    eslint: ^8.57.0 => 8.57.0
    eslint-config-airbnb: ^19.0.4 => 19.0.4
    eslint-config-airbnb-typescript: ^18.0.0 => 18.0.0
    eslint-config-prettier: ^8.10.0 => 8.10.0
    eslint-config-react-app: ^7.0.1 => 7.0.1
    eslint-import-resolver-typescript: ^3.6.1 => 3.6.1
    eslint-plugin-css-modules: ^2.11.4 => 2.12.0
    eslint-plugin-filename-rules: ^1.3.1 => 1.3.1
    eslint-plugin-flowtype: ^8.0.3 => 8.0.3
    eslint-plugin-html: ^7.1.0 => 7.1.0
    eslint-plugin-import: ^2.28.1 => 2.29.1
    eslint-plugin-jsx-a11y: ^6.7.1 => 6.9.0
    eslint-plugin-lingui: ^0.2.0 => 0.2.2
    eslint-plugin-no-barrel-files: ^1.1.1 => 1.1.1
    eslint-plugin-prettier: ^5.1.3 => 5.1.3
    eslint-plugin-react: ^7.33.2 => 7.34.3
    eslint-plugin-react-hooks: ^4.6.2 => 4.6.2
    eslint-plugin-react-refresh: ^0.4.7 => 0.4.7
    eslint-plugin-simple-import-sort: ^10.0.0 => 10.0.0
    fast-xml-parser: ^4.4.1 => 4.4.1 (4.2.5)
    flexlayout-react: ^0.7.15 => 0.7.15
    framer-motion: ^12.0.0-alpha.0 => 12.0.0-alpha.0
    history: ^5.3.0 => 5.3.0
    immer: ^10.0.4 => 10.1.1 (9.0.21, 9.0.6)
    iso3166-2-db: ^2.3.10 => 2.3.10
    jotai: ^2.8.0 => 2.9.0
    jotai-devtools: ^0.8.0 => 0.8.0
    jotai-effect: ^1.0.0 => 1.0.0
    jotai-optics: ^0.4.0 => 0.4.0
    jotai-scope: ^0.5.1 => 0.5.2
    lexical: ^0.12.5 => 0.12.6
    lexical-beautiful-mentions: ^0.1.36 => 0.1.36
    lodash: ^4.17.21 => 4.17.21
    optics-ts: ^2.4.1 => 2.4.1
    prettier: ^3.2.5 => 3.3.2 (2.3.2, 2.8.8, 1.19.1)
    prettier-eslint: ^16.1.1 => 16.3.0
    prettier-plugin-tailwindcss: ^0.5.12 => 0.5.14
    react: ^19.0.0-rc-163365a0-20240717 => 19.0.0-rc-163365a0-20240717
    react-dom: ^19.0.0-rc-163365a0-20240717 => 19.0.0-rc-163365a0-20240717
    react-draggable: ^4.4.6 => 4.4.6
    react-dropzone: ^14.2.3 => 14.2.3
    react-error-boundary: ^4.0.13 => 4.0.13 (3.1.4)
    react-hot-toast: ^2.4.1 => 2.4.1
    react-icons: ^5.0.1 => 5.2.1
    react-image-file-resizer: ^0.4.8 => 0.4.8
    react-loading-skeleton: ^3.3.1 => 3.4.0
    react-redux: ^9.1.2 => 9.1.2
    react-resizable: ^3.0.5 => 3.0.5
    react-resize-detector: ^9.1.0 => 9.1.1
    react-router-dom: ^6.20.0 => 6.24.1
    react-spinners: ^0.14.1 => 0.14.1
    react-table: ^7.8.0 => 7.8.0
    react-tag-autocomplete: ^7.2.0 => 7.3.0
    react-textarea-autosize: ^8.5.3 => 8.5.3 (8.3.4)
    react-virtuoso: ^4.10.3 => 4.10.3 (2.19.1)
    redux-first-history: ^5.2.0 => 5.2.0
    redux-persist: ^6.0.0 => 6.0.0
    redux-persist/integration/react:  undefined ()
    redux-thunk: ^3.1.0 => 3.1.0
    reselect: ^5.1.1 => 5.1.1
    stream-chat: ^8.14.4 => 8.37.0
    stream-chat-react: ^11.2.1 => 11.23.1
    stylelint: ^16.3.1 => 16.6.1
    stylelint-config-recess-order: ^5.0.0 => 5.0.1
    stylelint-config-recommended: ^13.0.0 => 13.0.0 (14.0.1)
    stylelint-config-standard-scss: ^13.0.0 => 13.1.0
    stylelint-config-tailwindcss: ^0.0.7 => 0.0.7
    stylelint-order: ^6.0.4 => 6.0.4
    stylelint-prettier: ^5.0.0 => 5.0.0
    tailwind-merge: ^2.4.0 => 2.4.0
    tailwind-scrollbar-hide: ^1.1.7 => 1.1.7
    tailwindcss: ^3.4.1 => 3.4.4
    ts-enum-util: ^4.1.0 => 4.1.0
    tsx: ^4.16.2 => 4.16.2
    types-react: ^19.0.0-rc.1 => 19.0.0-rc.1
    types-react-dom: ^19.0.0-rc.1 => 19.0.0-rc.1 (19.0.0)
    typescript: ^5.2.2 => 5.5.3 (4.4.4, 4.9.5)
    typescript-eslint: ^7.15.0 => 7.16.0
    typescript-plugin-css-modules: ^5.0.2 => 5.1.0
    typesense: ^1.8.2 => 1.8.2
    ua-parser-js: ^1.0.38 => 1.0.38
    vite: ^5.3.1 => 5.3.3 
    vite-plugin-babel: ^1.2.0 => 1.2.0
    vite-plugin-babel-macros: ^1.0.6 => 1.0.6
    vite-plugin-eslint: ^1.8.1 => 1.8.1
    vite-tsconfig-paths: ^4.3.2 => 4.3.2
  npmGlobalPackages:
    aws-cdk: 2.144.0
    corepack: 0.28.0
    create-vite: 5.3.0
    dpdm: 3.14.0
    eslint: 9.9.1
    firebase-tools: 13.11.0
    npm: 10.8.3
    rimraf: 5.0.10

Describe the bug

When adding an external postgresql schema with combine I get an error in function excludeDisabledOps in clientUtils.ts file:

TypeError: coarseToFineDict[key] is not iterable (cannot read property undefined)

I digged into it and found out that models that are generated after configuring the external database have an attribute of type model where the only key in the properties object is timestamp. The function iterates over the keys and tries to spread out coarseToFineDict[key]. The only keys in this dict are queries, mutations, subscriptions. No key timestamp exists, therefore the value is undefined and trying to spread it gives the error.
On regular schema models the properties object is empty and no error occurs.

Expected behavior

The external database schema from postgres can be genereated without any errors and then be combined with the regular datastore schema.

Reproduction steps

Following the steps from the guide:
https://docs.amplify.aws/react/build-a-backend/data/connect-to-existing-data-sources/connect-postgres-mysql-database/#step-2---generate-typescript-representation-of-your-database-schema

  1. Typescript representation of the database schema:
    npx ampx generate schema-from-database --connection-uri-secret SQL_CONNECTION_STRING --out amplify/data/schema.sql.ts

  2. Combining the schemas with the combine method

  3. Run ampx sandbox

  4. Start the application

Code Snippet

The schema.sql.file

export const generatedSqlSchema = configure({
    database: {
        identifier: "XXX",
        engine: "mysql",
        connectionUri: secret("PSQL_BACKEND_CONNECTION_STRING"),
        vpcConfig: {
            vpcId: "vpc-XXX",
            securityGroupIds: ["sg-XXX"],
            subnetAvailabilityZones: [
                {
                    subnetId: "subnet-XXX",
                    availabilityZone: "eu-central-1b"
                },
                {
                    subnetId: "subnet-XXX",
                    availabilityZone: "eu-central-1c"
                },
                {
                    subnetId: "subnet-XXX",
                    availabilityZone: "eu-central-1a"
                }
            ]
        }
    }
}).schema({
    Test: a
        .model({
            id: a.string().required()
        })
        .identifier(["id"])
});

Combining the schemas:

const dataBackendSchema = generatedSqlSchema.authorization((allow) => allow.guest());

...<regular schema>...

const combinedSchema = a.combine([schema, dataBackendSchema]);

export type Schema = ClientSchema<typeof combinedSchema>;

export const data = defineData({
    schema: combinedSchema,
    authorizationModes: {
        defaultAuthorizationMode: "userPool"
    }
});

Log output

TypeError: coarseToFineDict[key] is not iterable (cannot read property undefined)
at excludeDisabledOps (clientUtils.ts:47:29)
at generateModelsProperty (generateModelsProperty.ts:17:33)
at addSchemaToClient (addSchemaToClient.ts:6:21)
at generateClient (generateClient.ts:35:9)
at generateClient2 (API.ts:10:12)
at amplify-client.ts:10:30

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

@github-actions github-actions bot added pending-triage Issue is pending triage pending-maintainer-response Issue is pending a response from the Amplify team. labels Oct 4, 2024
@outraday
Copy link
Author

outraday commented Oct 4, 2024

For everybody else having this error. It is not a bug but a missing package.

The autogenerated .sql file has eslint disabled which did not show me that I missed @aws-amplify/data-schema package. Installed it as dev dependency and then it worked.

@outraday outraday closed this as completed Oct 4, 2024
@github-actions github-actions bot added pending-maintainer-response Issue is pending a response from the Amplify team. and removed pending-triage Issue is pending triage pending-maintainer-response Issue is pending a response from the Amplify team. labels Oct 4, 2024
@chrisbonifacio chrisbonifacio added GraphQL Related to GraphQL API issues Gen 2 Issues related to Gen 2 Amplify projects and removed pending-maintainer-response Issue is pending a response from the Amplify team. labels Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Gen 2 Issues related to Gen 2 Amplify projects GraphQL Related to GraphQL API issues
Projects
None yet
Development

No branches or pull requests

2 participants