Mongodb connector fails to refresh schema

[Ishankoradia]

Connector Name

source-mongodb-v2

Connector Version

1.5.14

What step the error happened?

Other

Relevant information

The mongo db source connector takes a long time (~9mins) to refresh source schema and then errors out

The airbyte version we are using is 1.1.0 with kubernetes deployment.

Added this under server's extraEnv in helm values.yaml

- name: HTTP_IDLE_TIMEOUT  
  value: 20m  
- name: READ_TIMEOUT  
  value: 30m

Relevant log output

Discover schema failed logs

2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CLAIM — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover) — (dataplaneId = local)
2025-02-11 08:12:43 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):166 - 
2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.c.WorkloadApiClient(claim):75 - Claimed: true for b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover via API for local
2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CHECK_STATUS — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover) — (dataplaneId = local)
2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.p.s.CheckStatusStage(applyStage):59 - No pod found running for workload b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover
2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: BUILD — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover) — (dataplaneId = local)
2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: MUTEX — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover) — (dataplaneId = local)
2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.p.s.EnforceMutexStage(applyStage):50 - No mutex key specified for workload: b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover. Continuing...
2025-02-11 08:12:33 �[32mINFO�[m i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: LAUNCH — (workloadId = b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover) — (dataplaneId = local)
2025-02-11 08:12:38 �[32mINFO�[m i.a.w.l.c.WorkloadApiClient(updateStatusToLaunched):60 - Attempting to update workload: b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover to LAUNCHED.
2025-02-11 08:12:38 �[32mINFO�[m i.a.w.l.p.h.SuccessHandler(accept):60 - Pipeline completed for workload: b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover.
2025-02-11 08:12:43 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):166 - ----- START DISCOVER -----
2025-02-11 08:12:43 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):166 - 
2025-02-11 08:21:43 �[33mWARN�[m i.a.c.ConnectorWatcher(waitForConnectorOutput):102 - Failed to find output files from connector within timeout of 9 minute(s). Is the connector still running?
2025-02-11 08:21:43 �[32mINFO�[m i.a.c.ConnectorWatcher(failWorkload):309 - Failing workload b2e713cd-cc36-4c0a-b5bd-b47cb8a0561e_ec16ae84-c0ff-4fd8-abe7-f02b88fe5f4f_0_discover.
2025-02-11 08:21:43 �[32mINFO�[m i.a.c.ConnectorWatcher(exitFileNotFound):227 - Deliberately exiting process with code 2.

Contribute

• Yes, I want to contribute