Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc updates for egress IP addresses #1566

Merged
merged 1 commit into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 33 additions & 24 deletions site/docs/concepts/connectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ or any other public image registry provider.
To interface with a connector, the Flow runtime needs to know:

1. The specific image to use, through an image name such as `ghcr.io/estuary/source-postgres:dev`.
Notice that the image name also conveys the specific image registry and version tag to use.
Notice that the image name also conveys the specific image registry and version tag to use.

2. Endpoint configuration such as a database address and account, with meaning that is specific to the connector.

Expand Down Expand Up @@ -153,8 +153,9 @@ user: postgres
</Tabs>

Storing configuration in separate files serves two important purposes:
* Re-use of configuration across multiple captures or materializations
* The ability to protect sensitive credentials

- Re-use of configuration across multiple captures or materializations
- The ability to protect sensitive credentials

### Protecting secrets

Expand Down Expand Up @@ -208,13 +209,13 @@ host: ENC[AES256_GCM,data:K/clly65pThTg2U=,iv:1bNmY8wjtjHFBcXLR1KFcsNMGVXRl5LGTd
password: ENC[AES256_GCM,data:IDDY+fl0/gAcsH+6tjRdww+G,iv:Ye8st7zJ9wsMRMs6BoAyWlaJeNc9qeNjkkjo6BPp/tE=,tag:EPS9Unkdg4eAFICGujlTfQ==,type:str]
user: ENC[AES256_GCM,data:w+F7MMwQhw==,iv:amHhNCJWAJnJaGujZgjhzVzUZAeSchEpUpBau7RVeCg=,tag:62HguhnnSDqJdKdwYnj7mQ==,type:str]
sops:
# Some items omitted for brevity:
gcp_kms:
- resource_id: projects/your-project-id/locations/us-central1/keyRings/your-ring/cryptoKeys/your-key-name
created_at: "2022-01-05T15:49:45Z"
enc: CiQAW8BC2GDYWrJTp3ikVGkTI2XaZc6F4p/d/PCBlczCz8BZiUISSQCnySJKIptagFkIl01uiBQp056c
lastmodified: "2022-01-05T15:49:45Z"
version: 3.7.1
# Some items omitted for brevity:
gcp_kms:
- resource_id: projects/your-project-id/locations/us-central1/keyRings/your-ring/cryptoKeys/your-key-name
created_at: "2022-01-05T15:49:45Z"
enc: CiQAW8BC2GDYWrJTp3ikVGkTI2XaZc6F4p/d/PCBlczCz8BZiUISSQCnySJKIptagFkIl01uiBQp056c
lastmodified: "2022-01-05T15:49:45Z"
version: 3.7.1
```

You then use this `config.yaml` within your Flow specification.
Expand All @@ -235,6 +236,7 @@ which is:
```
[email protected]
```

:::

#### Example: Protect portions of a configuration
Expand All @@ -259,21 +261,21 @@ Next, encrypt only values which have that suffix:
$ sops --encrypt --in-place --encrypted-suffix "_sops" --gcp-kms projects/your-project-id/locations/us-central1/keyRings/your-ring/cryptoKeys/your-key-name config.yaml
```

`sops` re-writes the file, wrapping only values having a "_sops" suffix and adding its `sops` metadata section:
`sops` re-writes the file, wrapping only values having a "\_sops" suffix and adding its `sops` metadata section:

```yaml title="config.yaml"
host: my.hostname
password_sops: ENC[AES256_GCM,data:dlfidMrHfDxN//nWQTPCsjoG,iv:DHQ5dXhyOOSKI6ZIzcUM67R6DD/2MSE4LENRgOt6GPY=,tag:FNs2pTlzYlagvz7vP/YcIQ==,type:str]
user: my-user
sops:
# Some items omitted for brevity:
encrypted_suffix: _sops
gcp_kms:
- resource_id: projects/your-project-id/locations/us-central1/keyRings/your-ring/cryptoKeys/your-key-name
created_at: "2022-01-05T16:06:36Z"
enc: CiQAW8BC2Au779CGdMFUjWPhNleCTAj9rL949sBvPQ6eyAC3EdESSQCnySJKD3eWX8XrtrgHqx327
lastmodified: "2022-01-05T16:06:37Z"
version: 3.7.1
# Some items omitted for brevity:
encrypted_suffix: _sops
gcp_kms:
- resource_id: projects/your-project-id/locations/us-central1/keyRings/your-ring/cryptoKeys/your-key-name
created_at: "2022-01-05T16:06:36Z"
enc: CiQAW8BC2Au779CGdMFUjWPhNleCTAj9rL949sBvPQ6eyAC3EdESSQCnySJKD3eWX8XrtrgHqx327
lastmodified: "2022-01-05T16:06:37Z"
version: 3.7.1
```

You then use this `config.yaml` within your Flow specification.
Expand All @@ -286,7 +288,14 @@ In some cases, your source or destination endpoint may be within a secure networ
to allow direct access to its port due to your organization's security policy.

:::tip
If permitted by your organization, a quicker solution is to whitelist the Estuary IP address, `34.121.207.128`.
If permitted by your organization, a quicker solution is to whitelist the Estuary IP addresses:

```
34.121.207.128
35.226.75.135
34.68.62.148
```

For help completing this task on different cloud hosting platforms,
see the documentation for the [connector](../reference/Connectors/README.md) you're using.
:::
Expand Down Expand Up @@ -352,10 +361,10 @@ Users are empowered to write their own connectors for esoteric systems not alrea

Furthermore, implementing a Docker-based community specification brings other important qualities to Estuary connectors:

* Cross-platform interoperability between Flow, Airbyte, and any other platform that supports the protocol
* The abilities to write connectors in any language and run them on any machine
* Built-in solutions for version management (through image tags) and distribution
* The ability to integrate connectors from different sources at will, without the centralized control of a single company, thanks to container image registries
- Cross-platform interoperability between Flow, Airbyte, and any other platform that supports the protocol
- The abilities to write connectors in any language and run them on any machine
- Built-in solutions for version management (through image tags) and distribution
- The ability to integrate connectors from different sources at will, without the centralized control of a single company, thanks to container image registries

:::info
In order to be reflected in the Flow web app and used on the managed Flow platform,
Expand Down
Loading
Loading