diff --git a/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md b/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md index eb2d3b49..f5f07a54 100644 --- a/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md +++ b/docs/Using AWS GCP private VPC connections/Setting up AWS Private Link.md @@ -1,118 +1,105 @@ # Setting up AWS Private Link -!!! Note - AWS PrivateLink is used for private connections within the same AWS region. - - Prior to configuring AWS PrivateLink on a SkySQL service, you must have created a VPC with a private subnet that will be used to communicate with private IP addresses. - - For detailed information about AWS PrivateLink, see ["AWS PrivateLink" (Amazon documentation)](https://aws.amazon.com/privatelink/). +AWS PrivateLink is an AWS service that enables secure and private connectivity between Virtual Private Clouds (VPCs) and third-party services. By using PrivateLink with SkySQL services, traffic does not traverse the public internet, which enhances security and reduces exposure to potential threats. -The default endpoint mechanism is "`nlb`", which supports accessing the SkySQL service via the public internet. Use of AWS PrivateLink occurs when the endpoint mechanism is changed to "`privateconnect`". +For detailed information about AWS PrivateLink, see ["AWS PrivateLink" (Amazon documentation)](https://aws.amazon.com/privatelink/). -AWS PrivateLink can be enabled using the SkySQL Portal, SkySQL DBaaS API, or SkySQL Terraform Provider. -## **Note the following** +## **Considerations** -- When a SkySQL service has the `privateconnect` endpoint mechanism, all connections occur through private endpoints. SkySQL does offer users to setup a “secondary” public IP endpoint also (available in the Portal UI when you provision a new service). -- Client connections can originate from private IP addresses in the linked VPC. -- AWS PrivateLink supports DNS propagation which can be configured to resolve the Database FQDN to the private IP for an internal DNS request. The private DNS names feature must be enabled on the customer's VPC. - -- The SkySQL IP Allowlist is not used with the `privateconnect` endpoint mechanism. -- With AWS PrivateLink, connections are restricted to the list of allowed accounts that were specified when configuring the SkySQL endpoint. -- Further restrictions are controlled through the customer's AWS account. VPC security group policies apply. For detailed information, see ["Control traffic to resources using security groups"](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) (Amazon documentation). -- Some customers using AWS PrivateLink may choose to disable TLS, but MariaDB recommends keeping TLS enabled for extra security. -- Endpoint changes can be destructive, resulting in downtime. When you change the connection type from private to public, or public to private, the endpoint must be destroyed and recreated. Changing the SkySQL endpoint between `nlb` and `privateconnect` will result in a service interruption. +- AWS PrivateLink is used for private connections within the same AWS region. The SkySQL service and the connection VPC must be in the same region. +- When using SkySQL with AWS PrivateLink, all connections occur through private endpoints. If you need to connect to the service from outside your VPC, you will need to use a VPN or other mechanism to go through the connected VPC. Alternatively, SkySQL can be configured to provide a second, public endpoint for an additional fee. +- A list of AWS Account IDs that will be allowed to connect to the SkySQL service must be provided when enabling AWS PrivateLink. This list can be updated at any time. +- The SkySQL IP Allowlist is not used with AWS PrivateLink connections. Access to the SkySQL service will be controlled by Security Groups in the connecting VPC. For detailed information, see ["Control traffic to resources using security groups"](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) (Amazon documentation). - Connections to SkySQL services by features such as SkySQL backups, and monitoring do not depend on AWS PrivateLink. - Query Editor is not supported when AWS PrivateLink is enabled. + ### **Enable AWS PrivateLink on Service Launch** -To enable AWS PrivateLink when launching a new service via the SkySQL Portal all you need to do is select the 'Enable Private link' option in the 'Security' section. +
+Enable Privatelink via the SkySQL Portal +
-For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page. +To enable AWS PrivateLink when launching a new service via the SkySQL Portal select the 'Enable Private link' option in the 'Security' section. +After the service completes provisioning, you will see a new option to "Set up Private Link" in the service's context menu. Click this option to add one or more AWS account IDs to the allowlist. -### **Enable AWS PrivateLink on Existing SkySQL Service** +
-To enable AWS PrivateLink for an existing service via the SkySQL Portal: +
+Enable Privatelink via the SkySQL DBaaS API +
-1. Log in to the SkySQL Portal -2. Click the "MANAGE" button (at right) for the desired service. -3. In the context menu, choose the "Set up Private Link" menu item. -4. In the popup window, add one or more AWS account IDs. -5. Click the "OK" button to confirm this operation. +To enable AWS PrivateLink when launching a new service via the SkySQL DBaaS API, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes to service creation JSON payload. -For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page. +``` +{ + "name": "my-skysql-service", + ... + "endpoint_mechanism": "privateconnect", + "allowed_accounts": [ + "AWS-ACCOUNT-ID-1", + "AWS-ACCOUNT-ID-2" + ] +} +``` +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a JSON array of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service. -### **Disabling AWS PrivateLink** +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/post_provisioning_v1_services). +
-To disable AWS PrivateLink via the SkySQL Portal: +
+Enable Privatelink via the SkySQL Terraform Provider +
-1. Visit the [SkySQL Portal](https://mariadb.com/docs/skysql-dbaas/working/nr-portal/) -2. Find the service that you would like to modify. -3. Click "MANAGE" on the far right side of the service listing. -4. In the context menu, select "Manage PrivateLink". -5. In the popup window, click "I want to disconnect my Private Link". -6. In the popup window, select "Disconnect". -7. After the service restarts, PrivateLink is disabled. -8. Since the service's allowlist was cleared when AWS PrivateLink was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling PrivateLink. +To enable AWS PrivateLink when launching a new service via the SkySQL DBaaS API, set the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes on the `skysql_service` resource. -## AWS Endpoint Setup +```hcl +resource "skysql_service" "example" { + name = "my-skysql-service" + ... + endpoint_mechanism = "privateconnect" + endpoint_allowed_accounts = ["123456789012"] +} +``` -In addition to switching the SkySQL service to the `privateconnect` endpoint mechanism, an AWS Endpoint must be created using the customer's AWS account in order for the SkySQL service to become accessible. +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a list of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service. -1. Log in to the AWS console. -2. Confirm the correct region is selected. -3. Navigate to the "VPC" page, then the "Endpoint" section. -4. Click the "Create Endpoint" button. -5. Fill in the name to help you to identify it. (This is optional.) -6. Set the Service category to "Other endpoint services". -7. The value for the "Service name" field must be set to the value of the "Fully Qualified Domain Name" in the "Connect" window from SkySQL portal. -8. Click "Verify service". AWS should find the service and auto-populate the rest of the form. -9. In the VPC search field, find the VPC that you want to use for the interconnect between the clients and the SkySQL service. -10. In the Subnets section, it is suggested that you select all the Availability Zones in the list, entering the proper subnet ID for each one. If you are unsure, view the details of your running instances to see the Subnet ID that they have configured. -11. Select IPv4 for "IP address type". -12. For the "Security Groups" section, we suggest that you create a new security group that will regulate which instances can make a connection to the database. - - In a new browser tab, use the AWS console to visit the "Security Groups" settings (under Network & Security). - - Click on the "Create security group" button. - - Fill in the group's name and (optionally) its description. - - Under "Inbound rules" click the "Add rule" button. - - Set the value for the "Port range" to be the port number of the "Read-write port" in the "Connect" window of the SkySQL portal. - - Set the Source to either a list of private (internal) IPv4 addresses that you want to authorize (adding a "/32" suffix to each one), or set it to an existing security group name that can be used to authorize all instances that have that security group in their configuration. - - Press the "Create security group" button. -13. Back on the endpoint tab, click the refresh button on the "Security Groups" section and choose the newly created security group. -14. Optionally add a "Name" tag for easy identification. -15. Press the "Create endpoint" button. Endpoint creation may take several minutes. When complete, status will change from "Pending" to "Available". -16. In the details of the new endpoint will be a list of DNS names. Copy the first name in the list and use that as the hostname for your clients to use when they connect. +A complete example Terraform template that creates a new SkySQL service with AWS PrivateLink enabled can be found in the [terraform provider examples](https://github.com/skysqlinc/terraform-provider-skysql/tree/main/examples/privateconnect). -The newly created endpoint now authorizes the internal IPs or security groups that you specified in the Source values to access the SkySQL service's connection port. When testing a client connection, ensure that the client host is authorized by the security group's Source settings and that you're using the "`readwrite`" port plus the appropriate username and password (either the default values or the value for any user you have created). -## **Setting up AWS Private link using SkySQL REST API** +For more information on using the SkySQL Terraform Provider, see ["SkySQL Terraform Provider"](https://registry.terraform.io/providers/skysqlinc/skysql/latest/docs). -### **Enable AWS PrivateLink on Service Launch** +
-To enable AWS PrivateLink when launching a new service via the SkySQL DBaaS API: - -1. Initiate service launch using the procedure at "[DBaaS API Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20REST%20API.md). -2. When you are creating the request, add the `"endpoint_mechanism"` and `"endpoint_allowed_accounts"` attributes to the JSON payload: - 1. - - ```json - "endpoint_mechanism": "privateconnect", - "endpoint_allowed_accounts": [ - "AWS-ACCOUNT-ID-1", - "AWS-ACCOUNT-ID-2" - ] - ``` - - - Set `"endpoint_mechanism"` to `"privateconnect"` - - Set `"endpoint_allowed_accounts"` to a JSON array of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service +**For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page.** -For the next step, go through the AWS Endpoint setup section. -### **Enable AWS PrivateLink on Existing SkySQL Service** +### **Enable AWS PrivateLink on an Existing SkySQL Service** -To enable AWS PrivateLink on an existing service via the SkySQL DBaaS API, create a JSON payload such as: +> [!CAUTION] +> Enabling PrivateLink on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the public endpoint is replaced with the new PrivateLink endpoint. + +
+Enable AWS PrivateLink on an existing service via the SkySQL Portal: +
+ +1. Log in to the SkySQL Portal +2. Click the "MANAGE" button (on the right) for the desired service. +3. In the context menu, choose the "Set up AWS PrivateLink" menu item. +4. In the popup window, add one or more AWS account IDs. +5. Click the "OK" button to confirm this operation. + +
+ +
+Enable AWS PrivateLink on an existing service via the SkySQL DBaaS API: +
+ +To enable AWS PrivateLink on an existing service, you will need to update the service endpoints with a payload similar to the following: ```json [ @@ -126,201 +113,117 @@ To enable AWS PrivateLink on an existing service via the SkySQL DBaaS API, creat ] ``` -- Set `"mechanism"` to `"privateconnect"` -- Set `"allowed_accounts"` to a JSON array of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service. +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +
-```json -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` +**For the next step, see the [AWS Endpoint Setup](#aws-endpoint-setup) section on this page.** -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": -```json -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` +## AWS Endpoint Setup -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): +To connect to a SkySQL service using AWS PrivateLink, you must create an endpoint in your VPC that connects to the SkySQL service. The endpoint will be used by clients in your VPC to connect to the SkySQL service. + +### Pre-requisites +- You must have a VPC in the same region as the SkySQL service. +- You must create a security group that will be used to control access to the SkySQL service endpoint. + - This security group should contain rules to allow traffic from your client instances to the port of the SkySQL service (usually 3306). + - You must create a rule in your security group for each IP range or other security group that will be allowed to connect to the SkySQL service. + - The security group must be associated with the VPC that you will use to connect to the SkySQL service. +- You will need to lookup the Endpoint Service ID that SkySQL provisioned for you when you created your SkySQL Service. + - This ID can be found in the "Connect" window of the SkySQL portal. + - If using the SkySQL DBaaS API, the ID can be found in the response of the service details API call. + ``` + curl https://api.skysql.com/provisioning/v1/services/{SERVICE_ID} | jq ".endpoints[0].endpoint_service" + ``` -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the PrivateLink endpoint setup is complete. - - A status of "`pending_upgrade`" is reflected while PrivateLink endpoint setup is in-progress. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. +### VPC Endpoint Creation Steps -The output will look something like this, though your values will vary: +1. Log in to the AWS console. +2. Confirm the correct region is selected. +3. Navigate to the "VPC" page, then the "Endpoints" section. +4. Click the "Create Endpoint" button. +5. In the "Name tag" field, enter a name for the new endpoint. This name can be anything you like. +6. Set the Service category to "Other endpoint services". +7. The value for the "Service name" field must be set to the value of the Endpoint Service ID provided to you by SkySQL. See [Pre-requisites](#pre-requisites) for more information on how to find this ID. +8. Click "Verify service". AWS should find the service and auto-populate the rest of the form. +9. In the VPC search field, find the VPC that you want to use for the interconnect between the clients and the SkySQL service. +10. In the Subnets section, it is suggested that you select all the Availability Zones in the list, entering the proper subnet ID for each one. If you are unsure, view the details of your running instances to see the Subnet ID that they have configured. +11. Select IPv4 for "IP address type". +12. For the "Security Groups" section, assign the security groups that will allow your client instance to connect to your VPC endpoint. See [Pre-requisites](#pre-requisites) for more information on setting up security groups. +13. Press the "Create endpoint" button. Endpoint creation may take several minutes. When complete, status will change from "Pending" to "Available". -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"com.amazonaws.vpce.AWS_REGION.vpce-svc-AWS_SERVICE_ID" -``` +After creation, the Endpoint will be in `Pending` status while AWS provisions the new endpoint. Once the endpoint is `Available`, you can connect to your SkySQL service using the new endpoint. +The newly created endpoint now authorizes the internal IPs or security groups that you specified in the Source values to access the SkySQL service's connection port. When testing a client connection, ensure that the client host is authorized by the security group's Source settings and that you're using the "`readwrite`" port plus the appropriate username and password (either the default values or the value for any user you have created). -If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. -For the next step, go through the AWS Endpoint Setup section. +### Connecting to your SkySQL Service -### **Disable AWS PrivateLink** +After creating your VPC endpoint, AWS will create a number of DNS records that will resolve to the private IP addresses of your Privatelink Endpoint. +- The first DNS name in the list can be used from any availability zone in your VPC and will resolve to the private IP address of the endpoint in the same availability zone. +- The following DNS names provided are availability zone specific and rely on the user to match the correct DNS name to the availability zone of the client instance. +- If connecting via these DNS names, we recommend using the first DNS name in the list to ensure that the connection is routed to the correct availability zone. -To disable AWS PrivateLink via the SkySQL DBaaS API, create a JSON payload such as: -```json -[ - { - "mechanism": "nlb" - } -] -``` +> [!NOTE] +> The DNS names provided by AWS will always be in the domain `amazonaws.com`. If connecting to your SkySQL service using SSL/TLS, the database certificate will not match the VPC endpoint name. Due to this, we recommend [Enabling Private DNS for AWS PrivateLink](#enabling-private-dns-for-aws-privatelink). -Set `"mechanism"` to `"nlb"` -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +### Enabling Private DNS for AWS PrivateLink -```json -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` +In order to connect to your SkySQL service using the skysql.com service name provided in the SkySQL portal, you must enable Private DNS for the VPC endpoint. This will allow the service name to resolve to the private IP address of the SkySQL service. -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": +The following requirements must be met to enable Private DNS for the VPC endpoint: +- Private DNS must be enabled for the VPC. +- Your client instances must use the default AWS DNS server provided by the VPC (this is usually turned on by default). -```json -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` +To enable Private DNS for the VPC endpoint: +1. Log in to the AWS console. +2. Confirm the correct region is selected. +3. Navigate to the "VPC" page, then the "Endpoints" section. +4. Select the VPC endpoint that you created for your SkySQL service. +5. Click the "Actions" button, then select "Modify Private DNS Name". +6. In the popup window, select the checkbox to "Enable Private DNS Name". +7. Click the "Save changes" button. -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): +After a short period of time, the service name provided in the SkySQL portal should resolve to the private IP address of the SkySQL service via PrivateLink. You can test this by connecting to the service using the service name provided in the portal. -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the endpoint modification is complete. - - A status of "`pending_upgrade`" is reflected while PrivateLink is being disabled. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. -The output will look something like this, though your values will vary: +### **Disabling AWS PrivateLink** -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"com.amazonaws.vpce.AWS_REGION.vpce-svc-AWS_SERVICE_ID" -``` +> [!CAUTION] +> Disabling PrivateLink on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the private endpoint is replaced with the new public endpoint. -If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. +
+Disable AWS PrivateLink via the SkySQL Portal +
-Since the service's allowlist was cleared when AWS PrivateLink was previously enabled, you will need to update the allowlist to allow clients to connect after disabling PrivateLink. +1. Visit the [SkySQL Portal](https://app.skysql.com/) +2. Find the service that you would like to modify. +3. Click "MANAGE" on the far right side of the service listing. +4. In the context menu, select "Manage PrivateLink". +5. In the popup window, click "I want to disconnect my Private Link". +6. In the popup window, select "Disconnect". +7. Since the service's allowlist was cleared when AWS PrivateLink was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling PrivateLink. -To add an address to the allowlist via the SkySQL DBaaS API, create a JSON payload such as: +
-```json -**{** **"ip_address":** "192.0.2.10/32"**}** -``` +
+Disable AWS PrivateLink via the SkySQL DBaaS API +
-And then use the API to update the `allowlist` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +To disable AWS PrivateLink on an existing service, you will need to update the service endpoints with a payload similar to the following: -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X POST -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/security/allowlist \ - | jq . +```json +[ + { + "mechanism": "nlb" + } +] ``` -## Setup AWS Private link using Terraform Provider - -For general instructions on using the SkySQL Terraform Provider, see "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) - -For an example Terraform configuration that enables AWS PrivateLink, see Resources section [here](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md). - - -### **Enable AWS PrivateLink on Service Launch** +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). -To enable AWS PrivateLink when launching a new service via the SkySQL Terraform provider: - -1. Initiate service launch using the procedure at "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) . -2. When you are configuring the `skysql_service` resource, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes. - - For example, the attributes can be placed after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["AWS-ACCOUNT-ID-1","AWS-ACCOUNT-ID-2"] - ``` - - - Set `endpoint_mechanism` to `"privateconnect"` - - Set `endpoint_allowed_accounts` to a comma-separated list of one or more customer account IDs in AWS that will be allowed to establish a private connection to the SkySQL service -3. Continue the rest of the procedure. - -For the next step, see the AWS Endpoint Setup section on this page. - -### **Enable AWS PrivateLink on Existing SkySQL Service** - -To enable AWS PrivateLink for an existing service via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by adding the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes. - - Follow the same instructions as above. - -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): - - ```bash - $ terraform apply -var-file="skysql-nr-quickstart.tfvars" - ``` - - -For the next step, see the AWS Endpoint Setup section on this page. - -### **Disable AWS PrivateLink** - -To disable AWS PrivateLink via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by doing the following: - - Changing the `endpoint_mechanism` attribute to `"nlb"` - - Removing the `endpoint_allowed_accounts` attribute - - Adding the `allow_list` attribute - - For example, if you started with the following attributes after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["AWS-ACCOUNT-ID-1","AWS-ACCOUNT-ID-2"] - ``` - - You could disable AWS PrivateLink by changing the attributes to the following: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "nlb" - allow_list = [ - { - "ip" : var.ip_address, - "comment" : var.ip_address_comment - } - ] - ``` - -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): +
diff --git a/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md b/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md index 3a49025e..a7fbdba3 100644 --- a/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md +++ b/docs/Using AWS GCP private VPC connections/Setting up GCP Private Service Connect.md @@ -1,70 +1,142 @@ # Setting up GCP Private Service Connect -!!! Note: - GCP Private Service Connect (PSC) is used for private connections within the same GCP region. - - Prior to configuring PSC on a SkySQL service, you must have created a VPC with a private subnet that will be used to communicate with private IP addresses. - - For detailed information about GCP PSC, see ["Private Service Connect" (Google documentation)](https://cloud.google.com/vpc/docs/private-service-connect). +Google Private Service Connect (PSC) is a Google Cloud service that enables secure and private connectivity between Virtual Private Clouds (VPCs) and third-party services. By using PSC with SkySQL services, traffic does not traverse the public internet, which enhances security and reduces exposure to potential threats. -The default endpoint mechanism is "`nlb`", which supports accessing the SkySQL service via the public internet. Use of GCP PSC occurs when the endpoint mechanism is changed to "`privateconnect`". +For detailed information about Google PSC, see ["Private Service Connect" (Google documentation)](https://cloud.google.com/vpc/docs/private-service-connect). -GCP PSC can be enabled using the SkySQL Portal, SkySQL DBaaS API, or SkySQL Terraform Provider. -## **Note the following** +## **Considerations** -- When a SkySQL service has the `privateconnect` endpoint mechanism, all connections occur through private endpoints. SkySQL does offer users to setup a “secondary” public IP endpoint also (available in the Portal UI when you provision a new service). -- With Private Service Connect, projects are allowlisted and auto-accept of connections from any account is disabled. -- Access restrictions are controlled through the customer's GCP account. VPC firewall rules apply. For detailed information, see ["Cloud Firewall overview" (Google documentation)](https://cloud.google.com/vpc/docs/firewalls). -- Some customers using Private Service Connect may choose to disable TLS, but MariaDB recommends keeping TLS enabled for extra security. -- Database connection limits are independent from PSC connection limits. The limit for PSC connections is 10. -- Endpoint changes can be destructive, resulting in downtime. When you change the connection type from private to public, or public to private, the endpoint must be destroyed and recreated. Changing the SkySQL endpoint between `nlb` and `privateconnect` will result in a service interruption. -- Connections to SkySQL services by features such as SkySQL backups, and monitoring do not depend on Private Service Connect. -- Query Editor is not supported when Private Service Connect is enabled. +- PSC is used for private connections within the same Google Cloud region. The SkySQL service and the connection VPC must be in the same region. +- When using SkySQL with PSC, all connections occur through private endpoints. If you need to connect to the service from outside your VPC, you will need to use a VPN or other mechanism to go through the connected VPC. Alternatively, SkySQL can be configured to provide a second, public endpoint for an additional fee. +- A list of Google Cloud project IDs that will be allowed to connect to the SkySQL service must be provided when enabling PSC. This list can be updated at any time. +- The SkySQL IP Allowlist is not used with PSC connections. Access to the SkySQL service can be controlled by setting up firewall rules inside the connecting VPC. +- Connections to SkySQL services by features such as SkySQL backups, and monitoring do not depend on PSC. +- Query Editor is not supported when PSC is enabled. +- PSC has connection limits which refer to the number of endpoints that can be created to a single PSC service within Google Cloud. Database connection limits are independent from PSC connection limits. The limit for PSC connections is 10. -### **Enable GCP Private Service Connect on Service Launch** -To enable GCP Private Service Connect when launching a new service via the SkySQL Portal: +### **Enable Private Service Connect on Service Launch** -- When you get to the final "Security" section, select "Enable Private Service Connect". +
+Enable Google PSC via the SkySQL Portal +
-For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. +To enable PSC when launching a new service via the SkySQL Portal select the 'Google Private Service Connect' option in the 'Security' section. +After the service completes provisioning, you will see a new option to "Manage Google Private Service Connect" in the service's context menu. Click this option to add one or more Google project IDs to the allowlist. +
-### **Enable GCP Private Service Connect on Existing SkySQL Service** +
+Enable Google PSC via the SkySQL DBaaS API +
-To enable GCP Private Service Connect for an existing service via the SkySQL Portal: +To enable Google PSC when launching a new service via the SkySQL DBaaS API, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes to service creation JSON payload. -1. Log in to the SkySQL Portal. -2. Click the "MANAGE" button (at right) for the desired service. -3. In the context menu, choose the "Set up Private Service Connect" menu item. +``` +{ + "name": "my-skysql-service", + ... + "endpoint_mechanism": "privateconnect", + "allowed_accounts": [ + "GCP-PROJECT-ID-1", + "GCP-PROJECT-ID-2" + ] +} +``` +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a JSON array of one or more client project IDs in Google Cloud that will be allowed to establish a private connection to the SkySQL service. + +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/post_provisioning_v1_services). +
+ +
+Enable Google PSC via the SkySQL Terraform Provider +
+ +To enable Google PSC when launching a new service via the SkySQL DBaaS API, set the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes on the `skysql_service` resource. + +```hcl +resource "skysql_service" "example" { + name = "my-skysql-service" + ... + endpoint_mechanism = "privateconnect" + endpoint_allowed_accounts = ["GCP-PROJECT-ID-1", "GCP-PROJECT-ID-2"] +} +``` + +- The `endpoint_mechanism` field must be set to `privateconnect` +- The `endpoint_allowed_accounts` field must be set to a list of one or more customer project IDs in Google Cloud that will be allowed to establish a private connection to the SkySQL service. + +A complete example Terraform template that creates a new SkySQL service with Google PSC enabled can be found in the [terraform provider examples](https://github.com/skysqlinc/terraform-provider-skysql/tree/main/examples/private-service-connect). + + +For more information on using the SkySQL Terraform Provider, see ["SkySQL Terraform Provider"](https://registry.terraform.io/providers/skysqlinc/skysql/latest/docs). + +
+ +**For the next step, see the [PSC Endpoint Setup](#private-service-connect-endpoint-setup) section on this page.** + + +### **Enable Google PSC on an Existing SkySQL Service** + +> [!CAUTION] +> Enabling PSC on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the public endpoint is replaced with the new PSC endpoint. + +
+Enable Google PSC on an existing service via the SkySQL Portal: +
+ +1. Log in to the SkySQL Portal +2. Click the "MANAGE" button (on the right) for the desired service. +3. In the context menu, choose the "Set up Google Private Service Connect" menu item. 4. In the popup window, add one or more GCP project IDs. 5. Click the "OK" button to confirm this operation. -For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. +
-### **Disable GCP Private Service Connect** +
+Enable Google PSC on an existing service via the SkySQL DBaaS API: +
-To disable GCP Private Service Connect via the SkySQL Portal: +To enable Google PSC on an existing service, you will need to update the service endpoints with a payload similar to the following: -!!! Note: - Adding a new PSC will fail unless existing PSC to deleted projects are deleted. +```json +[ + { + "mechanism": "privateconnect", + "allowed_accounts": [ + "GOOGLE-PROJECT-ID-1", + "GOOGLE-PROJECT-ID-2" + ] + } +] +``` + +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). + +
+ +**For the next step, see the [PSC Endpoint Setup](#private-service-connect-endpoint-setup) section on this page.** -1. Visit the SkySQL Portal. -2. Find the service that you would like to modify. -3. Click "MANAGE" on the far right side of the service listing. -4. In the context menu, select "Manage Private Service Connect". -5. In the popup window, click "I want to disconnect my Private Service Connect". -6. In the popup window, select "Disconnect". -7. After the service restarts, Private Service Connect is disabled. -8. Since the service's allowlist was cleared when GCP Private Service Connect was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling Private Service Connect. -## GCP Endpoint Setup +## Private Service Connect Endpoint Setup -Endpoint setup is performed using the customer's GCP account. +To connect to a SkySQL service using Google PSC, you must create an endpoint in your VPC that connects to the SkySQL service. The endpoint will be used by clients in your VPC to connect to the SkySQL service. + + +### Pre-requisites +- You must have a VPC in the same region as the SkySQL service. +- You will need to lookup the Endpoint Service ID that SkySQL provisioned for you when you created your SkySQL Service. + - This ID can be found in the "Connect" window of the SkySQL portal. + - If using the SkySQL DBaaS API, the ID can be found in the response of the service details API call. + ``` + curl https://api.skysql.com/provisioning/v1/services/{SERVICE_ID} | jq ".endpoints[0].endpoint_service" + ``` -### **Create a Subnet (optional)** +### Create a Subnet (optional) We recommend use of a subnet dedicated to Private Service Connect endpoints in the same VPC where the application is running. @@ -79,7 +151,8 @@ We recommend use of a subnet dedicated to Private Service Connect endpoints in t - Optionally configure Flow logs - Click "ADD". -### **Create a Static Internal IP Address** + +### Create a Static Internal IP Address 1. In the GCP console, navigate VPC network → VPC networks → → STATIC INTERNAL IP ADDRESSES → RESERVE STATIC ADDRESS. - Replace with the name of the VPC where the application is running. @@ -90,108 +163,59 @@ We recommend use of a subnet dedicated to Private Service Connect endpoints in t - Purpose: Non-shared - Click "RESERVE". -### ****Connect to the Published Service**** + +### VPC Endpoint Creation Steps + 1. In the GCP console, navigate Network services → Private Service Connect → CONNECTED ENDPOINTS → CONNECT ENDPOINT. 2. Configure the endpoint connection: - Target: Published service - - Target service: the value of the "Fully Qualified Domain Name" in the "Connect" window from SkySQL portal. + - Target service: the value of the Endpoint Service ID. See [Pre-requisites](#pre-requisites) for more information on how to find this ID. - Endpoint name: set to the Database ID from SkySQL (dbxxxxxxxx) - Network: select the VPC network where the application is running - Subnetwork: select the subnet where the static internal IP address is reserved - IP address: select the reserved internal IP address from the prior step - Click "ADD ENDPOINT". -## Setting up GCP PSC using SkySQL REST API - -### **Enable GCP PSC on Service Launch** - -To enable GCP Private Service Connect when launching a new service via the SkySQL DBaaS API: - -1. Initiate service launch using the procedure at "[DBaaS API Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20REST%20API.md)". -2. When you are creating the request, add the `"endpoint_mechanism"` and `"endpoint_allowed_accounts"` attributes to the JSON payload: - - ```json - "endpoint_mechanism": "privateconnect", - "endpoint_allowed_accounts": [ - "GCP-PROJECT-ID-1", - "GCP-PROJECT-ID-2" - ] - ``` - - - Set `"endpoint_mechanism"` to `"privateconnect"` - - Set `"endpoint_allowed_accounts"` to a JSON array of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service - -For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. - -### **Enable GCP Private Service Connect on Existing SkySQL Service** - -To enable GCP Private Service Connect on an existing service via the SkySQL DBaaS API, create a JSON payload such as: - -```json -[ - { - "mechanism": "privateconnect", - "allowed_accounts": [ - "GCP-PROJECT-ID-1", - "GCP-PROJECT-ID-2" - ] - } -] -``` +After creation, the Endpoint should have a status of `Accepted`. If this status is not present, please ensure your Google Project ID is added to the list of allowed accounts in the SkySQL portal for this service. -- Set `"mechanism"` to `"privateconnect"` -- Set `"allowed_accounts"` to a JSON array of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: +### Connecting to your SkySQL Service -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` +After creating your PSC endpoint, your service should be available within your VPC at the Private IP Address you assigned to the endpoint. +- DNS propagation from SkySQL to the Private IP address is not supported when using PSC. +- The hostname when connecting to your SkySQL service should always be the Private IP address of the PSC endpoint. -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": -```bash -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` +> [!NOTE] +> When using PSC with SSL/TLS, there will be a hostname mismatch since the hostname provisioned by SkySQL will not match your internal IP Address. This can be ignored as the connection is still secure. -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the Private Service Connect endpoint setup is complete. - - A status of "`pending_upgrade`" is reflected while Private Service Connect endpoint setup is in-progress. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. +### **Disabling Google PSC** -The output will look something like this, though your values will vary: +> [!CAUTION] +> Disabling PSC on an existing service will cause all existing connections to be dropped. The service will be unavailable for a short period of time while the private endpoint is replaced with the new public endpoint. -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"GCP_ENDPOINT_HOSTNAME" -``` +
+Disable Google PSC via the SkySQL Portal +
-If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. +1. Visit the [SkySQL Portal](https://app.skysql.com/) +2. Find the service that you would like to modify. +3. Click "MANAGE" on the far right side of the service listing. +4. In the context menu, select "Manage your Private Service Connect". +5. In the popup window, click "I want to disconnect my Private Service Connect". +6. In the popup window, select "Disconnect". +7. Since the service's allowlist was cleared when Goolge PSC was previously enabled, you will need to [update the allowlist](../Security/Configuring%20Firewall.md) to allow clients to connect after disabling PSC. -For the next step, see the [GCP Endpoint Setup](#gcp-endpoint-setup) sections on this page. +
-### **Disable GCP PSC** +
+Disable Google PSC via the SkySQL DBaaS API +
-To disable GCP Private Service Connect via the SkySQL DBaaS API, create a JSON payload such as: +To disable Google PSC on an existing service, you will need to update the service endpoints with a payload similar to the following: ```json [ @@ -201,157 +225,7 @@ To disable GCP Private Service Connect via the SkySQL DBaaS API, create a JSON p ] ``` -- Set `"mechanism"` to `"nlb"` - -And then use the API to update the `endpoints` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: - -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X PATCH -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/endpoints \ - | jq . -``` - -We will need the endpoint's service name later on in the setup, so use the API to query the endpoint once the service has finished its modifications and is back to "ready": - -```bash -curl -H "X-API-Key: ${API_KEY}" -X GET \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID} \ - | jq '.status,.endpoints[0].ports,.endpoints[0].endpoint_service' -``` - -Note that the `jq` command is filtering the returned JSON data to extract three things (which are separated by commas in the command above): - -- The status of the service, which must be "ready" for the other values to be available. - - The "`ready`" status means that the endpoint modification is complete. - - A status of "`pending_upgrade`" is reflected while Private Service Connect is being disabled. -- The "ports" data for the first item in the `endpoints` array. - - Some topologies include multiple port options, including ports for read/write and read-only connections, or for connections to the NoSQL listener. - - Choose the desired port, such as "`readwrite`". -- The `endpoint_service` value for the first item in the `endpoints` array. - -The output will look something like this, though your values will vary: - -```json -"ready" -[ - { - "name": "readwrite", - "port": 3306, - "purpose": "readwrite" - } -] -"GCP_ENDPOINT_HOSTNAME" -``` - -If you are not using `jq`, scan (or parse) the full returned JSON data to ensure the service status is "ready" and find the associated values described above. - -Since the service's allowlist was cleared when GCP Private Service Connect was previously enabled, you will need to update the allowlist to allow clients to connect after disabling Private Service Connect. - -To add an address to the allowlist via the SkySQL DBaaS API, create a JSON payload such as: - -```json -**{** **"ip_address":** "192.0.2.10/32"**}** -``` - -And then use the API to update the `allowlist` information of the service. The following example of using `curl` expects the above payload to be in a file named `data.json` and also expects the variables `API_KEY` and `SERVICE_ID` to be set: - -```bash -curl -H "X-API-Key: ${API_KEY}" \ - -X POST -d @data.json \ - https://api.skysql.com/provisioning/v1/services/${SERVICE_ID}/security/allowlist \ - | jq . -``` - -## **Terraform Provider** - -GCP Private Service Connect can be enabled with Terraform using the SkySQL Terraform provider. - -For general instructions on using the SkySQL Terraform Provider, see "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) . - -For an example Terraform configuration that enables GCP Private Service Connect, see Resources section [here](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md) . - -### **Enable GCP Private Service Connect on Service Launch** - -To enable GCP Private Service Connect when launching a new service via the SkySQL Terraform provider: - -1. Initiate service launch using the procedure at "[Terraform Launch Walkthrough](../Quickstart/Launch%20DB%20using%20the%20Terraform%20Provider.md). -2. When you are configuring the `skysql_service` resource, add the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes: - - For example, the attributes can be placed after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["GCP-PROJECT-ID-1","GCP-PROJECT-ID-2"] - ``` - - - Set `endpoint_mechanism` to `"privateconnect"` - - Set `endpoint_allowed_accounts` to a comma-separated list of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service -3. Continue the rest of the procedure. - -For the next step, see the [Connectivity](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#Connectivity), [Controls](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#Controls), & [GCP Endpoint Setup](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#GCP_Endpoint_Setup) sections on this page. - -### **Enable GCP Private Service Connect on Existing SkySQL Service** - -To enable GCP Private Service Connect for an existing service via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by adding the `endpoint_mechanism` and `endpoint_allowed_accounts` attributes. - - For example, the attributes can be placed after `ssl_enabled`: - - ```bash - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["GCP-PROJECT-ID-1","GCP-PROJECT-ID-2"] - ``` - - - Set `endpoint_mechanism` to `"privateconnect"` - - Set `endpoint_allowed_accounts` to a comma-separated list of one or more customer project IDs in GCP that will be allowed to establish a private connection to the SkySQL service -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): - - ```bash - $ terraform apply -var-file="skysql-nr-quickstart.tfvars" - ``` - - -For the next step, see the [GCP Endpoint Setup](https://mariadb.com/docs/skysql-dbaas/security/nr-private-connections/nr-gcp-private-service-connect/#GCP_Endpoint_Setup) sections on this page. - -### **Disable GCP Private Service Connect** - -To disable GCP Private Service Connect via the SkySQL Terraform provider: - -1. Alter the `skysql_service` resource configuration by doing the following: - - Changing the `endpoint_mechanism` attribute to `"nlb"` - - Removing the `endpoint_allowed_accounts` attribute - - Adding the `allow_list` attribute - - For example, if you started with the following attributes after `ssl_enabled`: - - ```json - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "privateconnect" - endpoint_allowed_accounts = ["GCP-PROJECT-ID-1","GCP-PROJECT-ID-2"] - ``` - - You could disable GCP Private Service Connect by changing the attributes to the following: - - ```json - ssl_enabled = var.ssl_enabled - endpoint_mechanism = "nlb" - allow_list = [ - { - "ip" : var.ip_address, - "comment" : var.ip_address_comment - } - ] - ``` - -2. Execute the `[terraform apply` command](https://developer.hashicorp.com/terraform/cli/commands/apply): - - ```json - $ terraform apply -var-file="skysql-nr-quickstart.tfvars" - ``` - +This payload should then be sent to the API `PATCH` https://api.skysql.com/provisioning/v1/services/{SERVICE_ID}/endpoints where `{SERVICE_ID}` is the ID of the service you are updating. +For more information on using the SkySQL DBaaS API, see ["SkySQL DBaaS API"](https://apidocs.skysql.com/#/Services/patch_provisioning_v1_services__service_id__endpoints). ---- +