From d7b49c192ad82104173269030c2d879e022dc78d Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 17 Dec 2015 11:16:00 -0800 Subject: [PATCH 001/110] Update bootstrap doc based on PM feedback - Replace `||` with `;` to ensure all start commands are executed - Add `-n` to avoid user prompts - Add example bosh output of final desired state - Re-add manual bootstrap steps in case the errand fails [#109827548] Signed-off-by: Ben Calegari --- docs/bootstrapping.md | 121 +++++++++++++++++++++++++++++++++++------- 1 file changed, 102 insertions(+), 19 deletions(-) diff --git a/docs/bootstrapping.md b/docs/bootstrapping.md index 304fd187..f763d99b 100644 --- a/docs/bootstrapping.md +++ b/docs/bootstrapping.md @@ -1,6 +1,6 @@ # Bootstrapping a Galera Cluster -Bootstrapping is the process of (re)starting a Galera cluster. +Bootstrapping is the process of (re)starting a Galera cluster. ## When to Bootstrap @@ -23,6 +23,7 @@ If quorum has *not* been lost, individual unhealthy nodes should automatically r | wsrep_cluster_status | non-Primary | +----------------------+-------------+ ``` + - All responsive nodes respond with `ERROR 1047` when queried with most statement types. ```sh @@ -38,24 +39,106 @@ As part of cf-mysql-release v25, we provide an auto-bootstrap feature which runs #### How to run -Run ```bosh run errand bootstrap``` from the terminal. When done, this should successfully bootstrap the cluster, and all jobs should report as ```running```. Note that: - -1. If the cluster was already healthy to begin with (i.e. quorum was never lost), the errand will error out saying ```bootstrap is not required```. -1. If one or more nodes are not reachable (i.e. the VM exists but in an unknown state), it will error out saying ```nodes are not reachable```. In this situation, follow the steps below: - 1. ```bosh -n stop mysql_z1 && bosh -n stop mysql_z2 && bosh -n stop mysql_z3``` - 1. ```bosh edit deployment``` - 1. Set ```update.canaries``` to 0, ```update.max_in_flight``` to 3, and ```update.serial``` to false. - 1. ```bosh deploy``` - 1. ```bosh -n start mysql_z1 || bosh -n start mysql_z2 || bosh -n start mysql_z3``` (This will throw several errors, but it ensures that all the jobs are present on the VM) - 1. ```bosh instances``` to verify that all jobs report as failing. - 1. Try running the errand again using ```bosh run errand bootstrap``` as above. - 1. Once the errand succeeds, the cluster is synced, although some jobs might still report as failing. - 1. ```bosh edit deployment``` - 1. Set ```update.canaries``` to 1, ```update.max_in_flight``` to 1, and ```update.serial``` to true. - 1. Verify that deployment succeeds and all jobs are healthy. - +Run `bosh run errand bootstrap` from the terminal. When done, this should successfully bootstrap the cluster, and all jobs should report as `running`. Note that: + +If the cluster was already healthy to begin with (i.e. quorum was never lost), the errand will error out saying `bootstrap is not required`. + +If one or more nodes are not reachable (i.e. the VM exists but in an unknown state), it will error out saying `nodes are not reachable`. In this situation, follow the steps below: +1. `bosh -n stop mysql_z1 && bosh -n stop mysql_z2 && bosh -n stop mysql_z3` +1. `bosh edit deployment` +1. Set `update.canaries` to 0, `update.max_in_flight` to 3, and `update.serial` to false. +1. `bosh deploy` +1. `bosh -n start mysql_z1 ; bosh -n start mysql_z2 ; bosh -n start mysql_z3` (This will throw several errors, but it ensures that all the jobs are present on the VM) +1. `bosh instances` to verify that all jobs report as failing. +1. Try running the errand again using `bosh -n run errand bootstrap` as above. +1. Once the errand succeeds, the cluster is synced, although some jobs might still report as failing. +1. `bosh edit deployment` +1. Set `update.canaries` to 1, `update.max_in_flight` to 1, and `update.serial` to true. +1. Verify that deployment succeeds and all jobs are healthy. A healthy deployment should look like this: + + ``` +$ bosh vms cf-warden-mysql | egrep 'mysql_z./0' +Acting as user 'admin' on deployment 'cf-warden-mysql' on 'Bosh Lite Director' +| mysql_z1/0 | running | mysql_z1 | 10.244.7.2 | +| mysql_z2/0 | running | mysql_z2 | 10.244.8.2 | +| mysql_z3/0 | running | mysql_z3 | 10.244.9.2 | + ``` + +If these steps did not work for you, please refer to the [Manual Bootstrap Process](#manual-bootstrap-process) below. + ## How it works -The bootstrap errand simply automates the steps in the manual bootstrapping process documented in previous releases of cf-mysql. It finds the node with the highest transaction sequence number, and asks it to start up by itself (i.e. in bootstrap mode), then asks the remaining nodes to join the cluster. +The bootstrap errand simply automates the steps in the manual bootstrapping process documented below. It finds the node with the highest transaction sequence number, and asks it to start up by itself (i.e. in bootstrap mode), then asks the remaining nodes to join the cluster. + +The sequence number of a stopped node can be retained by either reading the node's state file under `/var/vcap/store/mysql/grastate.dat`, or by running a mysqld command with a WSREP flag, like `mysqld --wsrep-recover`. + +## Manual Bootstrap Process + +The following steps are prone to user-error and can result in lost data if followed incorrectly. +Please follow the [Auto-bootstrap](#auto-bootstrap-errand) instructions above first, and only resort to the manual process if the errand fails to repair the cluster. + +1. SSH to each node in the cluster and, as root, shut down the mariadb process. + + ```sh + $ monit stop mariadb_ctrl + ``` + + Re-bootstrapping the cluster will not be successful unless all other nodes have been shut down. + +1. Choose a node to bootstrap. + + Find the node with the highest transaction sequence number (seqno): + + - If a node shutdown gracefully, the seqno should be in the galera state file. + + ```sh + $ cat /var/vcap/store/mysql/grastate.dat | grep 'seqno:' + ``` + + - If the node crashed or was killed, the seqno in the galera state file should be `-1`. In this case, the seqno may be recoverable from the database. The following command will cause the database to start up, log the recovered sequence number, and then exit. + + ```sh + $ /var/vcap/packages/mariadb/bin/mysqld --wsrep-recover + ``` + + Scan the error log for the recovered sequence number (the last number after the group id (uuid) is the recovered seqno): + + ```sh + $ grep "Recovered position" /var/vcap/sys/log/mysql/mysql.err.log | tail -1 + 150225 18:09:42 mysqld_safe WSREP: Recovered position e93955c7-b797-11e4-9faa-9a6f0b73eb46:15 + ``` + + Note: The galera state file will still say `seqno: -1` afterward. + + - If the node never connected to the cluster before crashing, it may not even have a group id (uuid in grastate.dat). In this case there's nothing to recover. Unless all nodes crashed this way, don't choose this node for bootstrapping. + + Use the node with the highest `seqno` value as the new bootstrap node. If all nodes have the same `seqno`, you can choose any node as the new bootstrap node. + + **Important:** Only perform these bootstrap commands on the node with the highest `seqno`. Otherwise the node with the highest `seqno` will be unable to join the new cluster (unless its data is abandoned). Its mariadb process will exit with an error. See [cluster behavior](cluster-behavior.md) for more details on intentionally abandoning data. + +1. On the new bootstrap node, update state file and restart the mariadb process: + + ```sh + $ echo -n "NEEDS_BOOTSTRAP" > /var/vcap/store/mysql/state.txt + $ monit start mariadb_ctrl + ``` + + You can check that the mariadb process has started successfully by running: + + ```sh + $ watch monit summary + ``` + + It can take up to 10 minutes for monit to start the mariadb process. + +1. Once the bootstrapped node is running, start the mariadb process on the remaining nodes via monit. + + ```sh + $ monit start mariadb_ctrl + ``` + +1. Verify that the new nodes have successfully joined the cluster. The following command should output the total number of nodes in the cluster: -The sequence number of a stopped node can be retained by either reading the node's state file under ```/var/vcap/store/mysql/grastate.dat```, or by running a mysqld command with a WSREP flag, like ```mysqld --wsrep-recover```. + ```sh + mysql> SHOW STATUS LIKE 'wsrep_cluster_size'; + ``` From a368f76d423e92591ff603d9938f02338b92d81d Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 17 Dec 2015 11:21:56 -0800 Subject: [PATCH 002/110] Update bootstrap doc to render properly on GitHub [#109827548] --- docs/bootstrapping.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/bootstrapping.md b/docs/bootstrapping.md index f763d99b..ce2db52b 100644 --- a/docs/bootstrapping.md +++ b/docs/bootstrapping.md @@ -44,6 +44,7 @@ Run `bosh run errand bootstrap` from the terminal. When done, this should succes If the cluster was already healthy to begin with (i.e. quorum was never lost), the errand will error out saying `bootstrap is not required`. If one or more nodes are not reachable (i.e. the VM exists but in an unknown state), it will error out saying `nodes are not reachable`. In this situation, follow the steps below: + 1. `bosh -n stop mysql_z1 && bosh -n stop mysql_z2 && bosh -n stop mysql_z3` 1. `bosh edit deployment` 1. Set `update.canaries` to 0, `update.max_in_flight` to 3, and `update.serial` to false. @@ -56,13 +57,13 @@ If one or more nodes are not reachable (i.e. the VM exists but in an unknown sta 1. Set `update.canaries` to 1, `update.max_in_flight` to 1, and `update.serial` to true. 1. Verify that deployment succeeds and all jobs are healthy. A healthy deployment should look like this: - ``` +``` $ bosh vms cf-warden-mysql | egrep 'mysql_z./0' Acting as user 'admin' on deployment 'cf-warden-mysql' on 'Bosh Lite Director' | mysql_z1/0 | running | mysql_z1 | 10.244.7.2 | | mysql_z2/0 | running | mysql_z2 | 10.244.8.2 | | mysql_z3/0 | running | mysql_z3 | 10.244.9.2 | - ``` +``` If these steps did not work for you, please refer to the [Manual Bootstrap Process](#manual-bootstrap-process) below. From 2dab6caa0574a97ba87d0a95773af6b6f84ac138 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Tue, 8 Dec 2015 17:39:38 -0800 Subject: [PATCH 003/110] Added arbitrator VM running garbd job and took out mysql_z3 node [#109744228] Signed-off-by: Morgan Fine --- jobs/arbitrator/monit | 5 +++ jobs/arbitrator/spec | 33 +++++++++++++++++ jobs/arbitrator/templates/garbd_config.erb | 4 +++ jobs/arbitrator/templates/garbd_ctl.erb | 41 ++++++++++++++++++++++ templates/cf-infrastructure-warden.yml | 2 +- templates/cf-mysql-template.yml | 18 +++------- 6 files changed, 89 insertions(+), 14 deletions(-) create mode 100644 jobs/arbitrator/monit create mode 100644 jobs/arbitrator/spec create mode 100644 jobs/arbitrator/templates/garbd_config.erb create mode 100644 jobs/arbitrator/templates/garbd_ctl.erb diff --git a/jobs/arbitrator/monit b/jobs/arbitrator/monit new file mode 100644 index 00000000..48a3e270 --- /dev/null +++ b/jobs/arbitrator/monit @@ -0,0 +1,5 @@ +check process garbd + with pidfile /var/vcap/sys/run/garbd/garbd.pid + start program "/var/vcap/jobs/arbitrator/bin/garbd_ctl start" with timeout 60 seconds + stop program "/var/vcap/jobs/arbitrator/bin/garbd_ctl stop" with timeout 10 seconds + group vcap diff --git a/jobs/arbitrator/spec b/jobs/arbitrator/spec new file mode 100644 index 00000000..147843c5 --- /dev/null +++ b/jobs/arbitrator/spec @@ -0,0 +1,33 @@ +--- +name: arbitrator + +templates: + garbd_ctl.erb: bin/garbd_ctl + garbd_config.erb: config/garbd_config + +packages: +- mariadb +- common + +properties: + admin_username: + description: 'Username for the MySQL server admin user' + default: 'root' + admin_password: + description: 'Password for the MySQL server admin user' + port: + description: 'Port the mysql server should bind to' + default: 3306 + galera_port: + description: 'Port which garbd listens on' + default: 4567 + healthcheck_port: + description: 'Port used by healthcheck process to listen on' + default: 9200 + cluster_ips: + description: 'List of nodes. Must have the same number of ips as there are nodes in the cluster' + + bootstrap_endpoint.username: + description: 'Username used by the bootstrap endpoints for Basic Auth' + bootstrap_endpoint.password: + description: 'Password used by the bootstrap endpoints for Basic Auth' diff --git a/jobs/arbitrator/templates/garbd_config.erb b/jobs/arbitrator/templates/garbd_config.erb new file mode 100644 index 00000000..cd993740 --- /dev/null +++ b/jobs/arbitrator/templates/garbd_config.erb @@ -0,0 +1,4 @@ +group = cf-mariadb-galera-cluster +<% galera_port = p('galera_port') %> +<% node_addresses = p('cluster_ips').map { |ip| "#{ip}:#{galera_port}" } %> +address = gcomm://<%= node_addresses.join(',') %> diff --git a/jobs/arbitrator/templates/garbd_ctl.erb b/jobs/arbitrator/templates/garbd_ctl.erb new file mode 100644 index 00000000..463cec31 --- /dev/null +++ b/jobs/arbitrator/templates/garbd_ctl.erb @@ -0,0 +1,41 @@ +#!/bin/bash -e + +job_dir=/var/vcap/jobs/arbitrator +run_dir=/var/vcap/sys/run/garbd +log_dir=/var/vcap/sys/log/garbd +package_dir=/var/vcap/packages/mariadb +pidfile=$run_dir/garbd.pid + +source /var/vcap/packages/common/utils.sh + +case $1 in + +start) + log "Starting Galera Arbitrator..." + pid_guard $pidfile "Galera Arbitrator" + + mkdir -p $run_dir + mkdir -p $log_dir + + echo $$ > $pidfile + chown vcap:vcap $pidfile + + exec chpst -u vcap:vcap $package_dir/bin/garbd \ + --cfg $job_dir/config/garbd_config \ + >>$log_dir/garbd.stdout.log \ + 2>>$log_dir/garbd.stderr.log + + log "Starting garbd... done" +;; + +stop) + log "Stopping garbd..." + kill_and_wait $pidfile + log "Stopping garbd... done" +;; + +*) + echo "Usage: garbd_ctl {start|stop}" +;; + +boesac diff --git a/templates/cf-infrastructure-warden.yml b/templates/cf-infrastructure-warden.yml index 7aa3848a..ab0136fc 100644 --- a/templates/cf-infrastructure-warden.yml +++ b/templates/cf-infrastructure-warden.yml @@ -23,7 +23,7 @@ resource_pools: network: mysql1 stemcell: *stemcell cloud_properties: {name: random} -- name: mysql_z3 +- name: arbitrator_z3 network: mysql1 stemcell: *stemcell cloud_properties: {name: random} diff --git a/templates/cf-mysql-template.yml b/templates/cf-mysql-template.yml index 9bc6a044..c733df2d 100644 --- a/templates/cf-mysql-template.yml +++ b/templates/cf-mysql-template.yml @@ -26,7 +26,7 @@ properties: default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] || nil )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) - - (( jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) + - (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) default_proxy_ips: - (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) - (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) @@ -67,7 +67,7 @@ resource_pools: network: mysql2 stemcell: (( merge )) cloud_properties: (( merge )) -- name: mysql_z3 +- name: arbitrator_z3 network: mysql3 stemcell: (( merge )) cloud_properties: (( merge )) @@ -149,11 +149,11 @@ jobs: skip_name_resolve: (( jobs.mysql_z1.properties.skip_name_resolve )) bootstrap_endpoint: (( jobs.mysql_z1.properties.bootstrap_endpoint )) -- name: mysql_z3 +- name: arbitrator_z3 release: cf-mysql - template: mysql + template: arbitrator instances: (( merge || 1 )) - resource_pool: mysql_z3 + resource_pool: arbitrator_z3 persistent_disk_pool: (( disk_pools.[0].name )) networks: - name: mysql3 @@ -162,15 +162,7 @@ jobs: <<: (( merge || nil )) admin_password: (( jobs.mysql_z1.properties.admin_password )) cluster_ips: (( jobs.mysql_z1.properties.cluster_ips )) - seeded_databases: ~ - database_startup_timeout: (( jobs.mysql_z1.properties.database_startup_timeout )) - max_heap_table_size: (( jobs.mysql_z1.properties.max_heap_table_size )) - tmp_table_size: (( jobs.mysql_z1.properties.tmp_table_size )) - wsrep_max_ws_rows: (( jobs.mysql_z1.properties.wsrep_max_ws_rows )) - wsrep_max_ws_size: (( jobs.mysql_z1.properties.wsrep_max_ws_size )) - syslog_aggregator: (( meta.syslog_aggregator )) network_name: mysql3 - skip_name_resolve: (( jobs.mysql_z1.properties.skip_name_resolve )) bootstrap_endpoint: (( jobs.mysql_z1.properties.bootstrap_endpoint )) - name: proxy_z1 From 197de6f1e17104f7e7348c6a4ffe11440e2ee7d2 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Wed, 9 Dec 2015 09:33:50 -0800 Subject: [PATCH 004/110] Fix typo in garbd_ctl template --- jobs/arbitrator/templates/garbd_ctl.erb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/arbitrator/templates/garbd_ctl.erb b/jobs/arbitrator/templates/garbd_ctl.erb index 463cec31..44501025 100644 --- a/jobs/arbitrator/templates/garbd_ctl.erb +++ b/jobs/arbitrator/templates/garbd_ctl.erb @@ -38,4 +38,4 @@ stop) echo "Usage: garbd_ctl {start|stop}" ;; -boesac +esac From d47bba2e08c6f8f6511d42ce164a8dceb97e9db4 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Wed, 9 Dec 2015 10:00:21 -0800 Subject: [PATCH 005/110] Switchboard does not monitor arbitrator vm [#109744232] --- jobs/proxy/spec | 2 ++ jobs/proxy/templates/switchboard.yml.erb | 4 +++- templates/cf-mysql-template.yml | 2 ++ 3 files changed, 7 insertions(+), 1 deletion(-) diff --git a/jobs/proxy/spec b/jobs/proxy/spec index 9c5b347e..d4ffefbb 100644 --- a/jobs/proxy/spec +++ b/jobs/proxy/spec @@ -61,6 +61,8 @@ properties: default: "tcp" cluster_ips: description: "List of nodes. Must have the same number of ips as there are nodes in the cluster" + arbitrator_ip: + description: "List of IP addresses for the arbitrator nodes of the MySQL cluster" standalone: description: "Standalone Mode: Are you deploying MySQL without a CloudFoundry deployment?" default: false diff --git a/jobs/proxy/templates/switchboard.yml.erb b/jobs/proxy/templates/switchboard.yml.erb index 548d58bb..0256ef90 100644 --- a/jobs/proxy/templates/switchboard.yml.erb +++ b/jobs/proxy/templates/switchboard.yml.erb @@ -7,7 +7,9 @@ "Port": <%= p('proxy.port') %> "HealthcheckTimeoutMillis": <%= p("proxy.healthcheck_timeout_millis") %> "Backends": - <% p('cluster_ips').each_with_index do |ip, n| %> + <% cluster_ips = p('cluster_ips') %> + <% cluster_ips.delete(p('arbitrator_ip')) %> + <% cluster_ips.each_with_index do |ip, n| %> - "Host": <%= ip %> "Port": 3306 "StatusPort": 9200 diff --git a/templates/cf-mysql-template.yml b/templates/cf-mysql-template.yml index c733df2d..a6a82052 100644 --- a/templates/cf-mysql-template.yml +++ b/templates/cf-mysql-template.yml @@ -188,6 +188,7 @@ jobs: nats: (( .properties.nats )) syslog_aggregator: (( meta.syslog_aggregator )) standalone: (( .properties.standalone )) + arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) - name: proxy_z2 release: cf-mysql @@ -212,6 +213,7 @@ jobs: nats: (( .properties.nats )) syslog_aggregator: (( meta.syslog_aggregator )) standalone: (( .properties.standalone )) + arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) - name: cf-mysql-broker_z1 release: cf-mysql From 044ebe004728609ea06f3837ceaba463cf17748e Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Wed, 9 Dec 2015 13:55:07 -0800 Subject: [PATCH 006/110] Galera-healthcheck now runs on arbitrator job [#109744234] --- jobs/arbitrator/monit | 6 +++ jobs/arbitrator/spec | 3 ++ .../templates/galera-healthcheck_ctl.erb | 42 +++++++++++++++++++ .../galera_healthcheck_config.yaml.erb | 18 ++++++++ 4 files changed, 69 insertions(+) create mode 100755 jobs/arbitrator/templates/galera-healthcheck_ctl.erb create mode 100644 jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb diff --git a/jobs/arbitrator/monit b/jobs/arbitrator/monit index 48a3e270..5732425b 100644 --- a/jobs/arbitrator/monit +++ b/jobs/arbitrator/monit @@ -3,3 +3,9 @@ check process garbd start program "/var/vcap/jobs/arbitrator/bin/garbd_ctl start" with timeout 60 seconds stop program "/var/vcap/jobs/arbitrator/bin/garbd_ctl stop" with timeout 10 seconds group vcap + +check process galera-healthcheck + with pidfile /var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid + start program "/var/vcap/jobs/arbitrator/bin/galera-healthcheck_ctl start" with timeout 60 seconds + stop program "/var/vcap/jobs/arbitrator/bin/galera-healthcheck_ctl stop" with timeout 10 seconds + group vcap diff --git a/jobs/arbitrator/spec b/jobs/arbitrator/spec index 147843c5..af2c94ad 100644 --- a/jobs/arbitrator/spec +++ b/jobs/arbitrator/spec @@ -4,9 +4,12 @@ name: arbitrator templates: garbd_ctl.erb: bin/garbd_ctl garbd_config.erb: config/garbd_config + galera-healthcheck_ctl.erb: bin/galera-healthcheck_ctl + galera_healthcheck_config.yaml.erb: config/galera_healthcheck_config.yaml packages: - mariadb +- galera-healthcheck - common properties: diff --git a/jobs/arbitrator/templates/galera-healthcheck_ctl.erb b/jobs/arbitrator/templates/galera-healthcheck_ctl.erb new file mode 100755 index 00000000..2bca3c15 --- /dev/null +++ b/jobs/arbitrator/templates/galera-healthcheck_ctl.erb @@ -0,0 +1,42 @@ +#!/bin/bash -e + +job_dir=/var/vcap/jobs/arbitrator +run_dir=/var/vcap/sys/run/galera-healthcheck +log_dir=/var/vcap/sys/log/galera-healthcheck +package_dir=/var/vcap/packages/galera-healthcheck +pidfile=$run_dir/galera-healthcheck.pid + +export GOROOT=/var/vcap/packages/golang +export PATH=$GOROOT/bin:$PWD/bin:$PATH + +source /var/vcap/packages/common/utils.sh + +case $1 in + + start) + log "Starting galera-healthcheck..." + pid_guard $pidfile "Galera Healthcheck" + + mkdir -p $run_dir + mkdir -p $log_dir + + cd $package_dir + + $package_dir/bin/galera-healthcheck \ + -configPath=$job_dir/config/galera_healthcheck_config.yaml \ + >>$log_dir/galera-healthcheck.stdout.log 2>>$log_dir/galera-healthcheck.stderr.log & + + log "Starting galera-healthcheck... done" + ;; + + stop) + log "Stopping galera-healthcheck..." + kill_and_wait $pidfile + log "Stopping galera-healthcheck... done" + ;; + + *) + echo "Usage: galera-healthcheck_ctl {start|stop}" + ;; + +esac diff --git a/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb new file mode 100644 index 00000000..6f1cffd5 --- /dev/null +++ b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb @@ -0,0 +1,18 @@ +--- +Port: <%= p('healthcheck_port') %> +PidFile: '/var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid' +MysqldPath: '/var/vcap/packages/mariadb/bin/mysqld' +DB: + User: <%= p('admin_username') %> + Password: <%= p('admin_password') %> +# This is the config that bosh sets up by default for monit. +Monit: + User: 'vcap' + Password: 'random-password' + Host: 'localhost' + Port: 2822 + MysqlStateFilePath: '/var/vcap/store/mysql/state.txt' + ServiceName: 'mariadb_ctrl' +BootstrapEndpoint: + Username: <%= p('bootstrap_endpoint.username') %> + Password: <%= p('bootstrap_endpoint.password') %> From 28da0cdab2213fa5ed0ada64078f2936e53ca0fd Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Wed, 9 Dec 2015 14:18:29 -0800 Subject: [PATCH 007/110] Bump src/github.com/cloudfoundry-incubator/galera-healthcheck Bump cloudfoundry-incubator/galera-healthcheck: Morgan Fine: /sequence_number returns helpful response on arbitrator node ( Travis Unknown ) Restructure api_test for upcoming arbitrator logic [#109744234] ( Travis Unknown ) [#109744234] --- jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb | 1 + jobs/mysql/templates/galera_healthcheck_config.yaml.erb | 1 + src/github.com/cloudfoundry-incubator/galera-healthcheck | 2 +- 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb index 6f1cffd5..80bc92ed 100644 --- a/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb +++ b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb @@ -2,6 +2,7 @@ Port: <%= p('healthcheck_port') %> PidFile: '/var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid' MysqldPath: '/var/vcap/packages/mariadb/bin/mysqld' +ArbitratorNode: true DB: User: <%= p('admin_username') %> Password: <%= p('admin_password') %> diff --git a/jobs/mysql/templates/galera_healthcheck_config.yaml.erb b/jobs/mysql/templates/galera_healthcheck_config.yaml.erb index 6f1cffd5..a3a48bb5 100644 --- a/jobs/mysql/templates/galera_healthcheck_config.yaml.erb +++ b/jobs/mysql/templates/galera_healthcheck_config.yaml.erb @@ -2,6 +2,7 @@ Port: <%= p('healthcheck_port') %> PidFile: '/var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid' MysqldPath: '/var/vcap/packages/mariadb/bin/mysqld' +ArbitratorNode: false DB: User: <%= p('admin_username') %> Password: <%= p('admin_password') %> diff --git a/src/github.com/cloudfoundry-incubator/galera-healthcheck b/src/github.com/cloudfoundry-incubator/galera-healthcheck index 3924873f..58f62ff9 160000 --- a/src/github.com/cloudfoundry-incubator/galera-healthcheck +++ b/src/github.com/cloudfoundry-incubator/galera-healthcheck @@ -1 +1 @@ -Subproject commit 3924873f2a44e8f23efc9aee0dc24f736294f093 +Subproject commit 58f62ff9a27b4db870bc7a10ea546bb5a1d49ae8 From 88eef1c0c1ca6b0cdfdb22330c373950d14c2a06 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Wed, 9 Dec 2015 16:42:25 -0800 Subject: [PATCH 008/110] Bump src/github.com/cloudfoundry-incubator/galera-healthcheck Bump cloudfoundry-incubator/galera-healthcheck: Morgan Fine: Use Monit.ServiceName config as node type source of truth ( Travis Unknown ) /stop_mysql, /start_mysql work with garbd ( Travis Unknown ) Restructure monit_client_test for upcoming arbitrator logic [#109744234] ( Travis Unknown ) /sequence_number returns helpful response on arbitrator node ( Travis Unknown ) [#109744234] --- jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb | 4 +--- jobs/mysql/templates/galera_healthcheck_config.yaml.erb | 1 - src/github.com/cloudfoundry-incubator/galera-healthcheck | 2 +- 3 files changed, 2 insertions(+), 5 deletions(-) diff --git a/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb index 80bc92ed..1fdd1ce6 100644 --- a/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb +++ b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb @@ -2,7 +2,6 @@ Port: <%= p('healthcheck_port') %> PidFile: '/var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid' MysqldPath: '/var/vcap/packages/mariadb/bin/mysqld' -ArbitratorNode: true DB: User: <%= p('admin_username') %> Password: <%= p('admin_password') %> @@ -12,8 +11,7 @@ Monit: Password: 'random-password' Host: 'localhost' Port: 2822 - MysqlStateFilePath: '/var/vcap/store/mysql/state.txt' - ServiceName: 'mariadb_ctrl' + ServiceName: 'garbd' BootstrapEndpoint: Username: <%= p('bootstrap_endpoint.username') %> Password: <%= p('bootstrap_endpoint.password') %> diff --git a/jobs/mysql/templates/galera_healthcheck_config.yaml.erb b/jobs/mysql/templates/galera_healthcheck_config.yaml.erb index a3a48bb5..6f1cffd5 100644 --- a/jobs/mysql/templates/galera_healthcheck_config.yaml.erb +++ b/jobs/mysql/templates/galera_healthcheck_config.yaml.erb @@ -2,7 +2,6 @@ Port: <%= p('healthcheck_port') %> PidFile: '/var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid' MysqldPath: '/var/vcap/packages/mariadb/bin/mysqld' -ArbitratorNode: false DB: User: <%= p('admin_username') %> Password: <%= p('admin_password') %> diff --git a/src/github.com/cloudfoundry-incubator/galera-healthcheck b/src/github.com/cloudfoundry-incubator/galera-healthcheck index 58f62ff9..564e8f91 160000 --- a/src/github.com/cloudfoundry-incubator/galera-healthcheck +++ b/src/github.com/cloudfoundry-incubator/galera-healthcheck @@ -1 +1 @@ -Subproject commit 58f62ff9a27b4db870bc7a10ea546bb5a1d49ae8 +Subproject commit 564e8f91b9ae996e7bf437523ebe5cb23442abbc From 9b1cca5739a6125db0e7edbad87794251a1c2b70 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Wed, 9 Dec 2015 18:03:01 -0800 Subject: [PATCH 009/110] Extended healthcheck endpoint on arbitrator node [#109744234] --- src/github.com/cloudfoundry-incubator/galera-healthcheck | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/galera-healthcheck b/src/github.com/cloudfoundry-incubator/galera-healthcheck index 564e8f91..a93e42aa 160000 --- a/src/github.com/cloudfoundry-incubator/galera-healthcheck +++ b/src/github.com/cloudfoundry-incubator/galera-healthcheck @@ -1 +1 @@ -Subproject commit 564e8f91b9ae996e7bf437523ebe5cb23442abbc +Subproject commit a93e42aa5743225df5ac2a88df7ee95c83d78c3c From 939d9e8eb66eb14640b3f97f325115e92c620f9d Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Thu, 10 Dec 2015 08:21:27 -0800 Subject: [PATCH 010/110] Bump src/github.com/cloudfoundry-incubator/galera-healthcheck Bump cloudfoundry-incubator/galera-healthcheck: Morgan Fine: Extended healthcheck endpoint running on arbitrator node to return "arbitrator node" [#109744234] --- src/github.com/cloudfoundry-incubator/galera-healthcheck | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/galera-healthcheck b/src/github.com/cloudfoundry-incubator/galera-healthcheck index a93e42aa..b9674336 160000 --- a/src/github.com/cloudfoundry-incubator/galera-healthcheck +++ b/src/github.com/cloudfoundry-incubator/galera-healthcheck @@ -1 +1 @@ -Subproject commit a93e42aa5743225df5ac2a88df7ee95c83d78c3c +Subproject commit b9674336d72f32f48c29aaf3a5bbbfdab0e2d5ae From fb7c9852bb06ce1874ea47c343d07c33a88c3941 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 10 Dec 2015 18:01:35 -0800 Subject: [PATCH 011/110] Bump src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap, src/github.com/cloudfoundry-incubator/galera-healthcheck Bump : Lyle Franklin: Update node_manager to handle an arbitrator node Test the bootstrapper interface to node_manager Extract bootstrap logic into collaborator object Morgan Fine: Simplify context setups through better use of test_helpers Backfill tests for node_manager from previous extraction Bump cloudfoundry-incubator/galera-healthcheck: Morgan Fine: Arbitrator node should return non-200 on root endpoint ( Travis Unknown ) Signed-off-by: Morgan Fine [#109744234] --- src/github.com/cloudfoundry-incubator/galera-healthcheck | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/galera-healthcheck b/src/github.com/cloudfoundry-incubator/galera-healthcheck index b9674336..88d78c23 160000 --- a/src/github.com/cloudfoundry-incubator/galera-healthcheck +++ b/src/github.com/cloudfoundry-incubator/galera-healthcheck @@ -1 +1 @@ -Subproject commit b9674336d72f32f48c29aaf3a5bbbfdab0e2d5ae +Subproject commit 88d78c23763a9e9edcd100e7f749dacc0ff58fa2 From 393a318697acb06db5da18525aef4b71ff46139c Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Fri, 11 Dec 2015 14:19:36 -0800 Subject: [PATCH 012/110] 3rd cluster node can be arbitrator or mysql node [#109744236] Signed-off-by: Aditya Anchuri --- bosh-lite/make_manifest | 13 ++++++++++ generate_deployment_manifest | 29 +++++++++++++++++---- jobs/proxy/spec | 1 + templates/cf-infrastructure-warden.yml | 2 +- templates/cf-mysql-node3-arbitrator.yml | 6 +++++ templates/cf-mysql-node3-mysql.yml | 6 +++++ templates/cf-mysql-template.yml | 34 +++++++++++++++++++++---- 7 files changed, 80 insertions(+), 11 deletions(-) create mode 100644 templates/cf-mysql-node3-arbitrator.yml create mode 100644 templates/cf-mysql-node3-mysql.yml diff --git a/bosh-lite/make_manifest b/bosh-lite/make_manifest index 5db64635..6d47a8bd 100755 --- a/bosh-lite/make_manifest +++ b/bosh-lite/make_manifest @@ -18,6 +18,18 @@ NO_COLOR='\033[0m' BOSH_LITE_USERNAME=${BOSH_LITE_USERNAME:-admin} BOSH_LITE_PASSWORD=${BOSH_LITE_PASSWORD:-admin} +ARBITRATOR=${ARBITRATOR:-"true"} + +if [ "${ARBITRATOR}" != "true" ] && \ + [ "${ARBITRATOR}" != "false" ]; then + echo "Error: ARBITRATOR Env var must be set to true or false" + exit 1 +elif [ "${ARBITRATOR}" == "true" ]; then + DEP_FLAG="arbitrator" +else + DEP_FLAG="no-arbitrator" +fi + # By default use 3 node stub mysql_stub="cf-mysql-stub-spiff-3-node.yml" # If any arguments were provided, @@ -56,6 +68,7 @@ perl -pi -e "s/PLACEHOLDER-DIRECTOR-UUID/$DIRECTOR_UUID/g" $RELEASE_DIR/bosh-lit ${RELEASE_DIR}/generate_deployment_manifest \ warden \ + ${DEP_FLAG} \ ${RELEASE_DIR}/templates/sample_stubs/sample_plans_stub.yml \ ${RELEASE_DIR}/bosh-lite/tmp/cf-mysql-stub-with-uuid.yml \ "$@" > ${RELEASE_DIR}/bosh-lite/manifests/cf-mysql-manifest.yml diff --git a/generate_deployment_manifest b/generate_deployment_manifest index 4cc89f84..9d6b6216 100755 --- a/generate_deployment_manifest +++ b/generate_deployment_manifest @@ -6,16 +6,35 @@ templates="${MY_DIR}/templates" infrastructure="$1" +if [ -z "$2" ]; then + arb="arbitrator" +elif [ "$2" != "arbitrator" ] && [ "$2" != "no-arbitrator" ]; then + arb="arbitrator" +elif [ "$2" == "arbitrator" ] || [ "$2" == "no-arbitrator" ]; then + arb=$2 + shift +fi + if [ "$infrastructure" != "aws" ] && \ [ "$infrastructure" != "warden" ] && \ [ "$infrastructure" != "vsphere" ] ; then - echo "usage: ./generate_deployment_manifest [stubs...]" + echo "usage: ./generate_deployment_manifest {no-arbitrator} [stubs...]" exit 1 fi shift -spiff merge \ - "$templates/cf-mysql-template.yml" \ - "$templates/cf-infrastructure-${infrastructure}.yml" \ - $* +if [ "${arb}" == "arbitrator" ]; then + spiff merge \ + "$templates/cf-mysql-template.yml" \ + "$templates/cf-mysql-node3-arbitrator.yml" \ + "$templates/cf-infrastructure-${infrastructure}.yml" \ + $* +else + spiff merge \ + "$templates/cf-mysql-template.yml" \ + "$templates/cf-mysql-node3-mysql.yml" \ + "$templates/cf-infrastructure-${infrastructure}.yml" \ + $* +fi + diff --git a/jobs/proxy/spec b/jobs/proxy/spec index d4ffefbb..8784437f 100644 --- a/jobs/proxy/spec +++ b/jobs/proxy/spec @@ -63,6 +63,7 @@ properties: description: "List of nodes. Must have the same number of ips as there are nodes in the cluster" arbitrator_ip: description: "List of IP addresses for the arbitrator nodes of the MySQL cluster" + default: "no-arbitrator-ip" standalone: description: "Standalone Mode: Are you deploying MySQL without a CloudFoundry deployment?" default: false diff --git a/templates/cf-infrastructure-warden.yml b/templates/cf-infrastructure-warden.yml index ab0136fc..7aa3848a 100644 --- a/templates/cf-infrastructure-warden.yml +++ b/templates/cf-infrastructure-warden.yml @@ -23,7 +23,7 @@ resource_pools: network: mysql1 stemcell: *stemcell cloud_properties: {name: random} -- name: arbitrator_z3 +- name: mysql_z3 network: mysql1 stemcell: *stemcell cloud_properties: {name: random} diff --git a/templates/cf-mysql-node3-arbitrator.yml b/templates/cf-mysql-node3-arbitrator.yml new file mode 100644 index 00000000..7df1f8bf --- /dev/null +++ b/templates/cf-mysql-node3-arbitrator.yml @@ -0,0 +1,6 @@ +jobs: +- name: arbitrator_z3 + instances: 1 + +- name: mysql_z3 + instances: 0 diff --git a/templates/cf-mysql-node3-mysql.yml b/templates/cf-mysql-node3-mysql.yml new file mode 100644 index 00000000..2783a46b --- /dev/null +++ b/templates/cf-mysql-node3-mysql.yml @@ -0,0 +1,6 @@ +jobs: +- name: arbitrator_z3 + instances: 0 + +- name: mysql_z3 + instances: 1 diff --git a/templates/cf-mysql-template.yml b/templates/cf-mysql-template.yml index a6a82052..8c071f58 100644 --- a/templates/cf-mysql-template.yml +++ b/templates/cf-mysql-template.yml @@ -67,7 +67,7 @@ resource_pools: network: mysql2 stemcell: (( merge )) cloud_properties: (( merge )) -- name: arbitrator_z3 +- name: mysql_z3 network: mysql3 stemcell: (( merge )) cloud_properties: (( merge )) @@ -149,11 +149,11 @@ jobs: skip_name_resolve: (( jobs.mysql_z1.properties.skip_name_resolve )) bootstrap_endpoint: (( jobs.mysql_z1.properties.bootstrap_endpoint )) -- name: arbitrator_z3 +- name: mysql_z3 release: cf-mysql - template: arbitrator - instances: (( merge || 1 )) - resource_pool: arbitrator_z3 + template: mysql + instances: (( merge || 0 )) + resource_pool: mysql_z3 persistent_disk_pool: (( disk_pools.[0].name )) networks: - name: mysql3 @@ -162,9 +162,33 @@ jobs: <<: (( merge || nil )) admin_password: (( jobs.mysql_z1.properties.admin_password )) cluster_ips: (( jobs.mysql_z1.properties.cluster_ips )) + seeded_databases: ~ + database_startup_timeout: (( jobs.mysql_z1.properties.database_startup_timeout )) + max_heap_table_size: (( jobs.mysql_z1.properties.max_heap_table_size )) + tmp_table_size: (( jobs.mysql_z1.properties.tmp_table_size )) + wsrep_max_ws_rows: (( jobs.mysql_z1.properties.wsrep_max_ws_rows )) + wsrep_max_ws_size: (( jobs.mysql_z1.properties.wsrep_max_ws_size )) + syslog_aggregator: (( meta.syslog_aggregator )) network_name: mysql3 + skip_name_resolve: (( jobs.mysql_z1.properties.skip_name_resolve )) bootstrap_endpoint: (( jobs.mysql_z1.properties.bootstrap_endpoint )) +- name: arbitrator_z3 + release: cf-mysql + template: arbitrator + instances: (( merge || 1 )) + resource_pool: mysql_z3 + persistent_disk_pool: (( merge || disk_pools.[0].name )) + networks: + - name: mysql3 + static_ips: (( merge || static_ips(0) )) + properties: + <<: (( merge || nil )) + admin_password: (( merge || jobs.mysql_z1.properties.admin_password )) + cluster_ips: (( merge || jobs.mysql_z1.properties.cluster_ips )) + network_name: mysql3 + bootstrap_endpoint: (( merge || jobs.mysql_z1.properties.bootstrap_endpoint )) + - name: proxy_z1 release: cf-mysql template: proxy From 65a19644a69b9b4344e9c63eb63bd82172cf6690 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Mon, 14 Dec 2015 11:53:39 -0800 Subject: [PATCH 013/110] Bump src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap Bump : Morgan Fine: Revert "Revert "Update node_manager to handle an arbitrator node"" Revert "Update node_manager to handle an arbitrator node" Aditya Anchuri: Fix flaky StopAllNodes failure in node_manager tests Signed-off-by: Aditya Anchuri --- src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap b/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap index 851bbc6d..e5fadb76 160000 --- a/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap +++ b/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap @@ -1 +1 @@ -Subproject commit 851bbc6d8ee934b5f71735e93719091f3bf6a6d9 +Subproject commit e5fadb76d3ce0d9ffdb1430bff9d82825ad8a0b7 From 301f399d855cf76c6ae263f176c4ac682ae411ef Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Thu, 17 Dec 2015 18:08:25 -0800 Subject: [PATCH 014/110] Add route-registrar unit tests to CI [#108793976] Signed-off-by: Lyle Franklin --- scripts/test-unit | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/test-unit b/scripts/test-unit index f5ff6da1..549880f4 100755 --- a/scripts/test-unit +++ b/scripts/test-unit @@ -27,3 +27,6 @@ ${RELEASE_DIR}/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer/bi echo -e "\nTesting Bootstrapper..." ${RELEASE_DIR}/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap/bin/test "$@" + +echo -e "\nTesting Route Registrar..." +${RELEASE_DIR}/src/github.com/cloudfoundry-incubator/route-registrar/bin/test "$@" From 8077ec5b4d646f8a2be8d5fb84bd74693efee73f Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Mon, 21 Dec 2015 10:24:12 -0800 Subject: [PATCH 015/110] Bump src/github.com/cloudfoundry-incubator/route-registrar Bump cloudfoundry-incubator/route-registrar: Lyle Franklin: Don't assume working directory when running ./bin/test ( Travis finished: https://travis-ci.org/cloudfoundry-incubator/route-registrar/builds/97589056 ) Avoid test pollution in script_checker_test ( Travis Unknown ) Ben Calegari: Do not run route-registrar tests in parallel ( Travis finished: https://travis-ci.org/cloudfoundry-incubator/route-registrar/builds/97588558 ) [#108793976] Signed-off-by: Ben Calegari --- src/github.com/cloudfoundry-incubator/route-registrar | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/route-registrar b/src/github.com/cloudfoundry-incubator/route-registrar index 952d993c..e821702b 160000 --- a/src/github.com/cloudfoundry-incubator/route-registrar +++ b/src/github.com/cloudfoundry-incubator/route-registrar @@ -1 +1 @@ -Subproject commit 952d993c4dd5db4a4e42307bc3498c970f1c0f9f +Subproject commit e821702b792ea806277dee5fd753001105bae550 From 24e1746a3dc00ebf74f982fcfd0ab55eef3dda0b Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Mon, 21 Dec 2015 10:43:00 -0800 Subject: [PATCH 016/110] Add godep to manage deps in tests [#108793976] Signed-off-by: Morgan Fine --- scripts/test-unit | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/test-unit b/scripts/test-unit index 549880f4..713b6249 100755 --- a/scripts/test-unit +++ b/scripts/test-unit @@ -13,6 +13,9 @@ source ${RELEASE_DIR}/.envrc rm -rf ${RELEASE_DIR}/bin/ginkgo go install -v github.com/onsi/ginkgo/ginkgo +rm -rf ${RELEASE_DIR}/bin/ginkgo +go install -v github.com/tools/godep + echo -e "\nTesting Switchboard..." ${RELEASE_DIR}/src/github.com/cloudfoundry-incubator/switchboard/bin/test "$@" From 3cdea69e810b4b78fa9181b4b91e939d671ce8ac Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Mon, 21 Dec 2015 10:50:19 -0800 Subject: [PATCH 017/110] Go get godeps before installing Signed-off-by: Ben Calegari --- scripts/test-unit | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/test-unit b/scripts/test-unit index 713b6249..0d7db9aa 100755 --- a/scripts/test-unit +++ b/scripts/test-unit @@ -13,7 +13,8 @@ source ${RELEASE_DIR}/.envrc rm -rf ${RELEASE_DIR}/bin/ginkgo go install -v github.com/onsi/ginkgo/ginkgo -rm -rf ${RELEASE_DIR}/bin/ginkgo +rm -rf ${RELEASE_DIR}/bin/godep +go get -v github.com/tools/godep go install -v github.com/tools/godep echo -e "\nTesting Switchboard..." From d21b9993ff84a86f91ee8610f4e711bd7d6439a5 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Mon, 21 Dec 2015 11:45:01 -0800 Subject: [PATCH 018/110] Bump src/github.com/cloudfoundry-incubator/route-registrar Bump cloudfoundry-incubator/route-registrar: Ben Calegari: Install gnatsd binary in bin/test ( Travis created: https://travis-ci.org/cloudfoundry-incubator/route-registrar/builds/98184847 ) Signed-off-by: Ben Calegari [#108793976] --- src/github.com/cloudfoundry-incubator/route-registrar | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/route-registrar b/src/github.com/cloudfoundry-incubator/route-registrar index e821702b..e8834cf6 160000 --- a/src/github.com/cloudfoundry-incubator/route-registrar +++ b/src/github.com/cloudfoundry-incubator/route-registrar @@ -1 +1 @@ -Subproject commit e821702b792ea806277dee5fd753001105bae550 +Subproject commit e8834cf6fd22131d5e943cbf129c9fcd9088e8ac From a9bb83d710cec02d249fc5a704e03ce1c214a72b Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 22 Dec 2015 11:21:23 -0800 Subject: [PATCH 019/110] arbitrator_z3 job uses static_ip at index 1 - This is because in the upgrade case where there was previously a mysql job deployed to the static ip at index 0, bosh is unable to swap jobs at a given IP during a single deploy. Thus mysql_z3 and arbitrator_z3 are deployed to separate IP addresses. [#109744236] Signed-off-by: Morgan Fine --- templates/cf-mysql-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/templates/cf-mysql-template.yml b/templates/cf-mysql-template.yml index 8c071f58..71a5ac18 100644 --- a/templates/cf-mysql-template.yml +++ b/templates/cf-mysql-template.yml @@ -181,7 +181,7 @@ jobs: persistent_disk_pool: (( merge || disk_pools.[0].name )) networks: - name: mysql3 - static_ips: (( merge || static_ips(0) )) + static_ips: (( merge || static_ips(1) )) properties: <<: (( merge || nil )) admin_password: (( merge || jobs.mysql_z1.properties.admin_password )) From 8adb62d95a6d083ceec4fb454d4a001cee4fcc93 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 29 Dec 2015 16:30:31 -0800 Subject: [PATCH 020/110] Add new stubs to conform with diego-style manifest generation - This approach is more prescriptive about what belongs in each stub - It pulls properties directly from a cf manifest to avoid copy-paste creds [#110775864] Signed-off-by: Lyle Franklin --- templates/bosh.yml | 19 + templates/bosh_lite_stubs/iaas_settings.yml | 1291 +++++++++++++++++ .../bosh_lite_stubs/property_overrides.yml | 25 + templates/config-from-cf-internal.yml | 37 + templates/config-from-cf.yml | 18 + templates/sample_stubs/aws_iaas_settings.yml | 117 ++ .../sample_stubs/instance_count_overrides.yml | 1 + templates/sample_stubs/property_overrides.yml | 48 + templates/sample_stubs/release_versions.yml | 1 + 9 files changed, 1557 insertions(+) create mode 100644 templates/bosh.yml create mode 100644 templates/bosh_lite_stubs/iaas_settings.yml create mode 100644 templates/bosh_lite_stubs/property_overrides.yml create mode 100644 templates/config-from-cf-internal.yml create mode 100644 templates/config-from-cf.yml create mode 100644 templates/sample_stubs/aws_iaas_settings.yml create mode 100644 templates/sample_stubs/instance_count_overrides.yml create mode 100644 templates/sample_stubs/property_overrides.yml create mode 100644 templates/sample_stubs/release_versions.yml diff --git a/templates/bosh.yml b/templates/bosh.yml new file mode 100644 index 00000000..b0d81704 --- /dev/null +++ b/templates/bosh.yml @@ -0,0 +1,19 @@ +director_uuid: (( merge )) + +name: (( merge )) + +releases: (( merge )) + +compilation: (( merge )) + +update: (( merge )) + +networks: (( merge )) + +disk_pools: (( merge || [] )) + +resource_pools: (( merge )) + +jobs: (( merge )) + +properties: (( merge )) diff --git a/templates/bosh_lite_stubs/iaas_settings.yml b/templates/bosh_lite_stubs/iaas_settings.yml new file mode 100644 index 00000000..89815b9a --- /dev/null +++ b/templates/bosh_lite_stubs/iaas_settings.yml @@ -0,0 +1,1291 @@ +iaas_settings: + + stemcell: + name: bosh-warden-boshlite-ubuntu-trusty-go_agent + version: latest + + compilation_cloud_properties: {} + + resource_pool_cloud_properties: + - name: mysql_z1 + cloud_properties: {} + - name: mysql_z2 + cloud_properties: {} + - name: mysql_z3 + cloud_properties: {} + - name: arbitrator_z3 + cloud_properties: {} + - name: proxy_z1 + cloud_properties: {} + - name: proxy_z2 + cloud_properties: {} + - name: cf-mysql-broker_z1 + cloud_properties: {} + - name: cf-mysql-broker_z2 + cloud_properties: {} + - name: errands_z1 + cloud_properties: {} + + disk_pools: + - name: mysql-persistent-disk + disk_size: 100000 + cloud_properties: {} + + subnet_configs: + - name: mysql1 + subnets: + - cloud_properties: + name: random + range: 10.244.7.0/30 + reserved: + - 10.244.7.1 + static: + - 10.244.7.2 + - cloud_properties: + name: random + range: 10.244.7.4/30 + reserved: + - 10.244.7.5 + static: + - 10.244.7.6 + - cloud_properties: + name: random + range: 10.244.7.8/30 + reserved: + - 10.244.7.9 + static: + - 10.244.7.10 + - cloud_properties: + name: random + range: 10.244.7.12/30 + reserved: + - 10.244.7.13 + static: + - 10.244.7.14 + - cloud_properties: + name: random + range: 10.244.7.16/30 + reserved: + - 10.244.7.17 + static: + - 10.244.7.18 + - cloud_properties: + name: random + range: 10.244.7.20/30 + reserved: + - 10.244.7.21 + static: + - 10.244.7.22 + - cloud_properties: + name: random + range: 10.244.7.24/30 + reserved: + - 10.244.7.25 + static: + - 10.244.7.26 + - cloud_properties: + name: random + range: 10.244.7.28/30 + reserved: + - 10.244.7.29 + static: + - 10.244.7.30 + - cloud_properties: + name: random + range: 10.244.7.32/30 + reserved: + - 10.244.7.33 + static: + - 10.244.7.34 + - cloud_properties: + name: random + range: 10.244.7.36/30 + reserved: + - 10.244.7.37 + static: + - 10.244.7.38 + - cloud_properties: + name: random + range: 10.244.7.40/30 + reserved: + - 10.244.7.41 + static: + - 10.244.7.42 + - cloud_properties: + name: random + range: 10.244.7.44/30 + reserved: + - 10.244.7.45 + static: + - 10.244.7.46 + - cloud_properties: + name: random + range: 10.244.7.48/30 + reserved: + - 10.244.7.49 + static: + - 10.244.7.50 + - cloud_properties: + name: random + range: 10.244.7.52/30 + reserved: + - 10.244.7.53 + static: + - 10.244.7.54 + - cloud_properties: + name: random + range: 10.244.7.56/30 + reserved: + - 10.244.7.57 + static: + - 10.244.7.58 + - cloud_properties: + name: random + range: 10.244.7.60/30 + reserved: + - 10.244.7.61 + static: + - 10.244.7.62 + - cloud_properties: + name: random + range: 10.244.7.64/30 + reserved: + - 10.244.7.65 + static: + - 10.244.7.66 + - cloud_properties: + name: random + range: 10.244.7.68/30 + reserved: + - 10.244.7.69 + static: + - 10.244.7.70 + - cloud_properties: + name: random + range: 10.244.7.72/30 + reserved: + - 10.244.7.73 + static: + - 10.244.7.74 + - cloud_properties: + name: random + range: 10.244.7.76/30 + reserved: + - 10.244.7.77 + static: + - 10.244.7.78 + - cloud_properties: + name: random + range: 10.244.7.80/30 + reserved: + - 10.244.7.81 + static: + - 10.244.7.82 + - cloud_properties: + name: random + range: 10.244.7.84/30 + reserved: + - 10.244.7.85 + static: + - 10.244.7.86 + - cloud_properties: + name: random + range: 10.244.7.88/30 + reserved: + - 10.244.7.89 + static: + - 10.244.7.90 + - cloud_properties: + name: random + range: 10.244.7.92/30 + reserved: + - 10.244.7.93 + static: + - 10.244.7.94 + - cloud_properties: + name: random + range: 10.244.7.96/30 + reserved: + - 10.244.7.97 + static: + - 10.244.7.98 + - cloud_properties: + name: random + range: 10.244.7.100/30 + reserved: + - 10.244.7.101 + static: + - 10.244.7.102 + - cloud_properties: + name: random + range: 10.244.7.104/30 + reserved: + - 10.244.7.105 + static: + - 10.244.7.106 + - cloud_properties: + name: random + range: 10.244.7.108/30 + reserved: + - 10.244.7.109 + static: + - 10.244.7.110 + - cloud_properties: + name: random + range: 10.244.7.112/30 + reserved: + - 10.244.7.113 + static: + - 10.244.7.114 + - cloud_properties: + name: random + range: 10.244.7.116/30 + reserved: + - 10.244.7.117 + static: + - 10.244.7.118 + - cloud_properties: + name: random + range: 10.244.7.120/30 + reserved: + - 10.244.7.121 + static: + - 10.244.7.122 + - cloud_properties: + name: random + range: 10.244.7.124/30 + reserved: + - 10.244.7.125 + static: + - 10.244.7.126 + - cloud_properties: + name: random + range: 10.244.7.128/30 + reserved: + - 10.244.7.129 + static: [] + - cloud_properties: + name: random + range: 10.244.7.132/30 + reserved: + - 10.244.7.133 + static: [] + - cloud_properties: + name: random + range: 10.244.7.136/30 + reserved: + - 10.244.7.137 + static: [] + - cloud_properties: + name: random + range: 10.244.7.140/30 + reserved: + - 10.244.7.141 + static: [] + - cloud_properties: + name: random + range: 10.244.7.144/30 + reserved: + - 10.244.7.145 + static: [] + - cloud_properties: + name: random + range: 10.244.7.148/30 + reserved: + - 10.244.7.149 + static: [] + - cloud_properties: + name: random + range: 10.244.7.152/30 + reserved: + - 10.244.7.153 + static: [] + - cloud_properties: + name: random + range: 10.244.7.156/30 + reserved: + - 10.244.7.157 + static: [] + - cloud_properties: + name: random + range: 10.244.7.160/30 + reserved: + - 10.244.7.161 + static: [] + - cloud_properties: + name: random + range: 10.244.7.164/30 + reserved: + - 10.244.7.165 + static: [] + - cloud_properties: + name: random + range: 10.244.7.168/30 + reserved: + - 10.244.7.169 + static: [] + - cloud_properties: + name: random + range: 10.244.7.172/30 + reserved: + - 10.244.7.173 + static: [] + - cloud_properties: + name: random + range: 10.244.7.176/30 + reserved: + - 10.244.7.177 + static: [] + - cloud_properties: + name: random + range: 10.244.7.180/30 + reserved: + - 10.244.7.181 + static: [] + - cloud_properties: + name: random + range: 10.244.7.184/30 + reserved: + - 10.244.7.185 + static: [] + - cloud_properties: + name: random + range: 10.244.7.188/30 + reserved: + - 10.244.7.189 + static: [] + - cloud_properties: + name: random + range: 10.244.7.192/30 + reserved: + - 10.244.7.193 + static: [] + - cloud_properties: + name: random + range: 10.244.7.196/30 + reserved: + - 10.244.7.197 + static: [] + - cloud_properties: + name: random + range: 10.244.7.200/30 + reserved: + - 10.244.7.201 + static: [] + - cloud_properties: + name: random + range: 10.244.7.204/30 + reserved: + - 10.244.7.205 + static: [] + - cloud_properties: + name: random + range: 10.244.7.208/30 + reserved: + - 10.244.7.209 + static: [] + - cloud_properties: + name: random + range: 10.244.7.212/30 + reserved: + - 10.244.7.213 + static: [] + - cloud_properties: + name: random + range: 10.244.7.216/30 + reserved: + - 10.244.7.217 + static: [] + - cloud_properties: + name: random + range: 10.244.7.220/30 + reserved: + - 10.244.7.221 + static: [] + - cloud_properties: + name: random + range: 10.244.7.224/30 + reserved: + - 10.244.7.225 + static: [] + - cloud_properties: + name: random + range: 10.244.7.228/30 + reserved: + - 10.244.7.229 + static: [] + - cloud_properties: + name: random + range: 10.244.7.232/30 + reserved: + - 10.244.7.233 + static: [] + - cloud_properties: + name: random + range: 10.244.7.236/30 + reserved: + - 10.244.7.237 + static: [] + - cloud_properties: + name: random + range: 10.244.7.240/30 + reserved: + - 10.244.7.241 + static: [] + - cloud_properties: + name: random + range: 10.244.7.244/30 + reserved: + - 10.244.7.245 + static: [] + - cloud_properties: + name: random + range: 10.244.7.248/30 + reserved: + - 10.244.7.249 + static: [] + - cloud_properties: + name: random + range: 10.244.7.252/30 + reserved: + - 10.244.7.253 + static: [] + + - name: mysql2 + subnets: + - cloud_properties: + name: random + range: 10.244.8.0/30 + reserved: + - 10.244.8.1 + static: + - 10.244.8.2 + - cloud_properties: + name: random + range: 10.244.8.4/30 + reserved: + - 10.244.8.5 + static: + - 10.244.8.6 + - cloud_properties: + name: random + range: 10.244.8.8/30 + reserved: + - 10.244.8.9 + static: + - 10.244.8.10 + - cloud_properties: + name: random + range: 10.244.8.12/30 + reserved: + - 10.244.8.13 + static: + - 10.244.8.14 + - cloud_properties: + name: random + range: 10.244.8.16/30 + reserved: + - 10.244.8.17 + static: + - 10.244.8.18 + - cloud_properties: + name: random + range: 10.244.8.20/30 + reserved: + - 10.244.8.21 + static: + - 10.244.8.22 + - cloud_properties: + name: random + range: 10.244.8.24/30 + reserved: + - 10.244.8.25 + static: + - 10.244.8.26 + - cloud_properties: + name: random + range: 10.244.8.28/30 + reserved: + - 10.244.8.29 + static: + - 10.244.8.30 + - cloud_properties: + name: random + range: 10.244.8.32/30 + reserved: + - 10.244.8.33 + static: + - 10.244.8.34 + - cloud_properties: + name: random + range: 10.244.8.36/30 + reserved: + - 10.244.8.37 + static: + - 10.244.8.38 + - cloud_properties: + name: random + range: 10.244.8.40/30 + reserved: + - 10.244.8.41 + static: + - 10.244.8.42 + - cloud_properties: + name: random + range: 10.244.8.44/30 + reserved: + - 10.244.8.45 + static: + - 10.244.8.46 + - cloud_properties: + name: random + range: 10.244.8.48/30 + reserved: + - 10.244.8.49 + static: + - 10.244.8.50 + - cloud_properties: + name: random + range: 10.244.8.52/30 + reserved: + - 10.244.8.53 + static: + - 10.244.8.54 + - cloud_properties: + name: random + range: 10.244.8.56/30 + reserved: + - 10.244.8.57 + static: + - 10.244.8.58 + - cloud_properties: + name: random + range: 10.244.8.60/30 + reserved: + - 10.244.8.61 + static: + - 10.244.8.62 + - cloud_properties: + name: random + range: 10.244.8.64/30 + reserved: + - 10.244.8.65 + static: + - 10.244.8.66 + - cloud_properties: + name: random + range: 10.244.8.68/30 + reserved: + - 10.244.8.69 + static: + - 10.244.8.70 + - cloud_properties: + name: random + range: 10.244.8.72/30 + reserved: + - 10.244.8.73 + static: + - 10.244.8.74 + - cloud_properties: + name: random + range: 10.244.8.76/30 + reserved: + - 10.244.8.77 + static: + - 10.244.8.78 + - cloud_properties: + name: random + range: 10.244.8.80/30 + reserved: + - 10.244.8.81 + static: + - 10.244.8.82 + - cloud_properties: + name: random + range: 10.244.8.84/30 + reserved: + - 10.244.8.85 + static: + - 10.244.8.86 + - cloud_properties: + name: random + range: 10.244.8.88/30 + reserved: + - 10.244.8.89 + static: + - 10.244.8.90 + - cloud_properties: + name: random + range: 10.244.8.92/30 + reserved: + - 10.244.8.93 + static: + - 10.244.8.94 + - cloud_properties: + name: random + range: 10.244.8.96/30 + reserved: + - 10.244.8.97 + static: + - 10.244.8.98 + - cloud_properties: + name: random + range: 10.244.8.100/30 + reserved: + - 10.244.8.101 + static: + - 10.244.8.102 + - cloud_properties: + name: random + range: 10.244.8.104/30 + reserved: + - 10.244.8.105 + static: + - 10.244.8.106 + - cloud_properties: + name: random + range: 10.244.8.108/30 + reserved: + - 10.244.8.109 + static: + - 10.244.8.110 + - cloud_properties: + name: random + range: 10.244.8.112/30 + reserved: + - 10.244.8.113 + static: + - 10.244.8.114 + - cloud_properties: + name: random + range: 10.244.8.116/30 + reserved: + - 10.244.8.117 + static: + - 10.244.8.118 + - cloud_properties: + name: random + range: 10.244.8.120/30 + reserved: + - 10.244.8.121 + static: + - 10.244.8.122 + - cloud_properties: + name: random + range: 10.244.8.124/30 + reserved: + - 10.244.8.125 + static: + - 10.244.8.126 + - cloud_properties: + name: random + range: 10.244.8.128/30 + reserved: + - 10.244.8.129 + static: [] + - cloud_properties: + name: random + range: 10.244.8.132/30 + reserved: + - 10.244.8.133 + static: [] + - cloud_properties: + name: random + range: 10.244.8.136/30 + reserved: + - 10.244.8.137 + static: [] + - cloud_properties: + name: random + range: 10.244.8.140/30 + reserved: + - 10.244.8.141 + static: [] + - cloud_properties: + name: random + range: 10.244.8.144/30 + reserved: + - 10.244.8.145 + static: [] + - cloud_properties: + name: random + range: 10.244.8.148/30 + reserved: + - 10.244.8.149 + static: [] + - cloud_properties: + name: random + range: 10.244.8.152/30 + reserved: + - 10.244.8.153 + static: [] + - cloud_properties: + name: random + range: 10.244.8.156/30 + reserved: + - 10.244.8.157 + static: [] + - cloud_properties: + name: random + range: 10.244.8.160/30 + reserved: + - 10.244.8.161 + static: [] + - cloud_properties: + name: random + range: 10.244.8.164/30 + reserved: + - 10.244.8.165 + static: [] + - cloud_properties: + name: random + range: 10.244.8.168/30 + reserved: + - 10.244.8.169 + static: [] + - cloud_properties: + name: random + range: 10.244.8.172/30 + reserved: + - 10.244.8.173 + static: [] + - cloud_properties: + name: random + range: 10.244.8.176/30 + reserved: + - 10.244.8.177 + static: [] + - cloud_properties: + name: random + range: 10.244.8.180/30 + reserved: + - 10.244.8.181 + static: [] + - cloud_properties: + name: random + range: 10.244.8.184/30 + reserved: + - 10.244.8.185 + static: [] + - cloud_properties: + name: random + range: 10.244.8.188/30 + reserved: + - 10.244.8.189 + static: [] + - cloud_properties: + name: random + range: 10.244.8.192/30 + reserved: + - 10.244.8.193 + static: [] + - cloud_properties: + name: random + range: 10.244.8.196/30 + reserved: + - 10.244.8.197 + static: [] + - cloud_properties: + name: random + range: 10.244.8.200/30 + reserved: + - 10.244.8.201 + static: [] + - cloud_properties: + name: random + range: 10.244.8.204/30 + reserved: + - 10.244.8.205 + static: [] + - cloud_properties: + name: random + range: 10.244.8.208/30 + reserved: + - 10.244.8.209 + static: [] + - cloud_properties: + name: random + range: 10.244.8.212/30 + reserved: + - 10.244.8.213 + static: [] + - cloud_properties: + name: random + range: 10.244.8.216/30 + reserved: + - 10.244.8.217 + static: [] + - cloud_properties: + name: random + range: 10.244.8.220/30 + reserved: + - 10.244.8.221 + static: [] + - cloud_properties: + name: random + range: 10.244.8.224/30 + reserved: + - 10.244.8.225 + static: [] + - cloud_properties: + name: random + range: 10.244.8.228/30 + reserved: + - 10.244.8.229 + static: [] + - cloud_properties: + name: random + range: 10.244.8.232/30 + reserved: + - 10.244.8.233 + static: [] + - cloud_properties: + name: random + range: 10.244.8.236/30 + reserved: + - 10.244.8.237 + static: [] + - cloud_properties: + name: random + range: 10.244.8.240/30 + reserved: + - 10.244.8.241 + static: [] + - cloud_properties: + name: random + range: 10.244.8.244/30 + reserved: + - 10.244.8.245 + static: [] + - cloud_properties: + name: random + range: 10.244.8.248/30 + reserved: + - 10.244.8.249 + static: [] + - cloud_properties: + name: random + range: 10.244.8.252/30 + reserved: + - 10.244.8.253 + static: [] + + - name: mysql3 + subnets: + - cloud_properties: + name: random + range: 10.244.9.0/30 + reserved: + - 10.244.9.1 + static: + - 10.244.9.2 + - cloud_properties: + name: random + range: 10.244.9.4/30 + reserved: + - 10.244.9.5 + static: + - 10.244.9.6 + - cloud_properties: + name: random + range: 10.244.9.8/30 + reserved: + - 10.244.9.9 + static: + - 10.244.9.10 + - cloud_properties: + name: random + range: 10.244.9.12/30 + reserved: + - 10.244.9.13 + static: + - 10.244.9.14 + - cloud_properties: + name: random + range: 10.244.9.16/30 + reserved: + - 10.244.9.17 + static: + - 10.244.9.18 + - cloud_properties: + name: random + range: 10.244.9.20/30 + reserved: + - 10.244.9.21 + static: + - 10.244.9.22 + - cloud_properties: + name: random + range: 10.244.9.24/30 + reserved: + - 10.244.9.25 + static: + - 10.244.9.26 + - cloud_properties: + name: random + range: 10.244.9.28/30 + reserved: + - 10.244.9.29 + static: + - 10.244.9.30 + - cloud_properties: + name: random + range: 10.244.9.32/30 + reserved: + - 10.244.9.33 + static: + - 10.244.9.34 + - cloud_properties: + name: random + range: 10.244.9.36/30 + reserved: + - 10.244.9.37 + static: + - 10.244.9.38 + - cloud_properties: + name: random + range: 10.244.9.40/30 + reserved: + - 10.244.9.41 + static: + - 10.244.9.42 + - cloud_properties: + name: random + range: 10.244.9.44/30 + reserved: + - 10.244.9.45 + static: + - 10.244.9.46 + - cloud_properties: + name: random + range: 10.244.9.48/30 + reserved: + - 10.244.9.49 + static: + - 10.244.9.50 + - cloud_properties: + name: random + range: 10.244.9.52/30 + reserved: + - 10.244.9.53 + static: + - 10.244.9.54 + - cloud_properties: + name: random + range: 10.244.9.56/30 + reserved: + - 10.244.9.57 + static: + - 10.244.9.58 + - cloud_properties: + name: random + range: 10.244.9.60/30 + reserved: + - 10.244.9.61 + static: + - 10.244.9.62 + - cloud_properties: + name: random + range: 10.244.9.64/30 + reserved: + - 10.244.9.65 + static: + - 10.244.9.66 + - cloud_properties: + name: random + range: 10.244.9.68/30 + reserved: + - 10.244.9.69 + static: + - 10.244.9.70 + - cloud_properties: + name: random + range: 10.244.9.72/30 + reserved: + - 10.244.9.73 + static: + - 10.244.9.74 + - cloud_properties: + name: random + range: 10.244.9.76/30 + reserved: + - 10.244.9.77 + static: + - 10.244.9.78 + - cloud_properties: + name: random + range: 10.244.9.80/30 + reserved: + - 10.244.9.81 + static: + - 10.244.9.82 + - cloud_properties: + name: random + range: 10.244.9.84/30 + reserved: + - 10.244.9.85 + static: + - 10.244.9.86 + - cloud_properties: + name: random + range: 10.244.9.88/30 + reserved: + - 10.244.9.89 + static: + - 10.244.9.90 + - cloud_properties: + name: random + range: 10.244.9.92/30 + reserved: + - 10.244.9.93 + static: + - 10.244.9.94 + - cloud_properties: + name: random + range: 10.244.9.96/30 + reserved: + - 10.244.9.97 + static: + - 10.244.9.98 + - cloud_properties: + name: random + range: 10.244.9.100/30 + reserved: + - 10.244.9.101 + static: + - 10.244.9.102 + - cloud_properties: + name: random + range: 10.244.9.104/30 + reserved: + - 10.244.9.105 + static: + - 10.244.9.106 + - cloud_properties: + name: random + range: 10.244.9.108/30 + reserved: + - 10.244.9.109 + static: + - 10.244.9.110 + - cloud_properties: + name: random + range: 10.244.9.112/30 + reserved: + - 10.244.9.113 + static: + - 10.244.9.114 + - cloud_properties: + name: random + range: 10.244.9.116/30 + reserved: + - 10.244.9.117 + static: + - 10.244.9.118 + - cloud_properties: + name: random + range: 10.244.9.120/30 + reserved: + - 10.244.9.121 + static: + - 10.244.9.122 + - cloud_properties: + name: random + range: 10.244.9.124/30 + reserved: + - 10.244.9.125 + static: + - 10.244.9.126 + - cloud_properties: + name: random + range: 10.244.9.128/30 + reserved: + - 10.244.9.129 + static: [] + - cloud_properties: + name: random + range: 10.244.9.132/30 + reserved: + - 10.244.9.133 + static: [] + - cloud_properties: + name: random + range: 10.244.9.136/30 + reserved: + - 10.244.9.137 + static: [] + - cloud_properties: + name: random + range: 10.244.9.140/30 + reserved: + - 10.244.9.141 + static: [] + - cloud_properties: + name: random + range: 10.244.9.144/30 + reserved: + - 10.244.9.145 + static: [] + - cloud_properties: + name: random + range: 10.244.9.148/30 + reserved: + - 10.244.9.149 + static: [] + - cloud_properties: + name: random + range: 10.244.9.152/30 + reserved: + - 10.244.9.153 + static: [] + - cloud_properties: + name: random + range: 10.244.9.156/30 + reserved: + - 10.244.9.157 + static: [] + - cloud_properties: + name: random + range: 10.244.9.160/30 + reserved: + - 10.244.9.161 + static: [] + - cloud_properties: + name: random + range: 10.244.9.164/30 + reserved: + - 10.244.9.165 + static: [] + - cloud_properties: + name: random + range: 10.244.9.168/30 + reserved: + - 10.244.9.169 + static: [] + - cloud_properties: + name: random + range: 10.244.9.172/30 + reserved: + - 10.244.9.173 + static: [] + - cloud_properties: + name: random + range: 10.244.9.176/30 + reserved: + - 10.244.9.177 + static: [] + - cloud_properties: + name: random + range: 10.244.9.180/30 + reserved: + - 10.244.9.181 + static: [] + - cloud_properties: + name: random + range: 10.244.9.184/30 + reserved: + - 10.244.9.185 + static: [] + - cloud_properties: + name: random + range: 10.244.9.188/30 + reserved: + - 10.244.9.189 + static: [] + - cloud_properties: + name: random + range: 10.244.9.192/30 + reserved: + - 10.244.9.193 + static: [] + - cloud_properties: + name: random + range: 10.244.9.196/30 + reserved: + - 10.244.9.197 + static: [] + - cloud_properties: + name: random + range: 10.244.9.200/30 + reserved: + - 10.244.9.201 + static: [] + - cloud_properties: + name: random + range: 10.244.9.204/30 + reserved: + - 10.244.9.205 + static: [] + - cloud_properties: + name: random + range: 10.244.9.208/30 + reserved: + - 10.244.9.209 + static: [] + - cloud_properties: + name: random + range: 10.244.9.212/30 + reserved: + - 10.244.9.213 + static: [] + - cloud_properties: + name: random + range: 10.244.9.216/30 + reserved: + - 10.244.9.217 + static: [] + - cloud_properties: + name: random + range: 10.244.9.220/30 + reserved: + - 10.244.9.221 + static: [] + - cloud_properties: + name: random + range: 10.244.9.224/30 + reserved: + - 10.244.9.225 + static: [] + - cloud_properties: + name: random + range: 10.244.9.228/30 + reserved: + - 10.244.9.229 + static: [] + - cloud_properties: + name: random + range: 10.244.9.232/30 + reserved: + - 10.244.9.233 + static: [] + - cloud_properties: + name: random + range: 10.244.9.236/30 + reserved: + - 10.244.9.237 + static: [] + - cloud_properties: + name: random + range: 10.244.9.240/30 + reserved: + - 10.244.9.241 + static: [] + - cloud_properties: + name: random + range: 10.244.9.244/30 + reserved: + - 10.244.9.245 + static: [] + - cloud_properties: + name: random + range: 10.244.9.248/30 + reserved: + - 10.244.9.249 + static: [] + - cloud_properties: + name: random + range: 10.244.9.252/30 + reserved: + - 10.244.9.253 + static: [] + diff --git a/templates/bosh_lite_stubs/property_overrides.yml b/templates/bosh_lite_stubs/property_overrides.yml new file mode 100644 index 00000000..1c85af59 --- /dev/null +++ b/templates/bosh_lite_stubs/property_overrides.yml @@ -0,0 +1,25 @@ +property_overrides: + mysql: + admin_password: password + bootstrap_endpoint: + username: username + password: password + proxy: + api_username: username + api_password: password + broker: + auth_username: admin + auth_password: password + cookie_secret: 94046872-2602-4ca6-8d07-8b0da9762477 + dashboard_secret: p-mysql-secret + plans: + - name: 100mb + id: ab08f1bc-e6fc-4b56-a767-ee0fea6e3f20 + description: Shared MySQL Server + max_storage_mb: 10 + max_user_connections: 20 + - name: 1gb + id: 11d0aa36-dcec-4021-85f5-ea4d9a5c8342 + description: Shared MySQL Server + max_storage_mb: 20 + max_user_connections: 40 diff --git a/templates/config-from-cf-internal.yml b/templates/config-from-cf-internal.yml new file mode 100644 index 00000000..6c34e246 --- /dev/null +++ b/templates/config-from-cf-internal.yml @@ -0,0 +1,37 @@ +config_from_cf: + cf_deployment_name: (( name )) + cf_director_uuid: (( director_uuid )) + nats: + user: (( properties.nats.user )) + password: (( properties.nats.password )) + port: (( properties.nats.port )) + machines: (( properties.nats.machines )) + domain: (( properties.domain )) + system_domain: (( properties.system_domain )) + app_domains: (( properties.app_domains )) + api_url: (( "https://api." properties.domain )) + skip_ssl_validation: (( properties.skip_ssl_validation )) + admin_username: (( properties.admin_username )) + admin_password: (( properties.admin_password )) + smoke_tests: + use_existing_org: (( properties.smoke_tests.use_existing_org )) + org: (( properties.smoke_tests.org )) + +# The keys below should not be included in the final stub +name: (( merge )) +director_uuid: (( merge )) +properties: + nats: + user: (( merge )) + password: (( merge )) + machines: (( merge )) + port: (( merge )) + domain: (( merge )) + system_domain: (( merge )) + app_domains: (( merge )) + admin_username: (( merge )) + admin_password: (( merge )) + skip_ssl_validation: (( merge || nil )) + smoke_tests: + use_existing_org: (( merge || nil )) + org: (( merge || nil )) diff --git a/templates/config-from-cf.yml b/templates/config-from-cf.yml new file mode 100644 index 00000000..a7414a53 --- /dev/null +++ b/templates/config-from-cf.yml @@ -0,0 +1,18 @@ +config_from_cf: + cf_deployment_name: (( merge )) + cf_director_uuid: (( merge )) + nats: + user: (( merge )) + password: (( merge )) + port: (( merge )) + machines: (( merge )) + domain: (( merge )) + system_domain: (( merge )) + app_domains: (( merge )) + api_url: (( merge )) + skip_ssl_validation: (( merge )) + admin_username: (( merge )) + admin_password: (( merge )) + smoke_tests: + use_existing_org: (( merge )) + org: (( merge )) diff --git a/templates/sample_stubs/aws_iaas_settings.yml b/templates/sample_stubs/aws_iaas_settings.yml new file mode 100644 index 00000000..3ff047bf --- /dev/null +++ b/templates/sample_stubs/aws_iaas_settings.yml @@ -0,0 +1,117 @@ +properties: + template_only: + aws: + availability_zone: REPLACE_WITH_AZ_1 # e.g. us-east-1a + availability_zone2: REPLACE_WITH_AZ_2 + availability_zone3: REPLACE_WITH_AZ_3 + +iaas_settings: + stemcell: &stemcell + name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent + version: latest + compilation_cloud_properties: + instance_type: c4.large + availability_zone: (( properties.template_only.aws.availability_zone )) + ephemeral_disk: + size: 30000 # in MB + type: gp2 + resource_pool_cloud_properties: + - name: mysql_z1 + cloud_properties: + instance_type: m3.large + availability_zone: (( properties.template_only.aws.availability_zone )) + + - name: mysql_z2 + cloud_properties: + instance_type: m3.large + availability_zone: (( properties.template_only.aws.availability_zone2 )) + + - name: mysql_z3 + cloud_properties: + instance_type: m3.large + availability_zone: (( properties.template_only.aws.availability_zone3 )) + + - name: arbitrator_z3 + cloud_properties: + instance_type: m3.medium + availability_zone: (( properties.template_only.aws.availability_zone3 )) + + - name: proxy_z1 + cloud_properties: + instance_type: m3.medium + availability_zone: (( properties.template_only.aws.availability_zone )) + elbs: (( merge || nil )) + + - name: proxy_z2 + cloud_properties: + instance_type: m3.medium + availability_zone: (( properties.template_only.aws.availability_zone2 )) + elbs: (( merge || nil )) + + - name: cf-mysql-broker_z1 + cloud_properties: + instance_type: m3.medium + availability_zone: (( properties.template_only.aws.availability_zone )) + + - name: cf-mysql-broker_z2 + cloud_properties: + instance_type: m3.medium + availability_zone: (( properties.template_only.aws.availability_zone2 )) + + - name: errands_z1 + cloud_properties: + instance_type: m3.medium + availability_zone: (( properties.template_only.aws.availability_zone )) + + disk_pools: + - name: mysql-persistent-disk + disk_size: 100000 + cloud_properties: + type: gp2 + +# NOTE: +# Replace 10.0.{0,1,2} part of the octet +# with your network configuration + subnet_configs: + - name: mysql1 + type: manual + subnets: + - range: 10.10.32.0/24 + reserved: + - 10.10.32.2 - 10.10.32.9 + - 10.10.32.130 - 10.10.32.254 + static: + - 10.10.32.10 - 10.10.32.14 + gateway: 10.10.32.1 + dns: + - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 + cloud_properties: + subnet: REPLACE_WITH_YOUR_SUBNET_ID_1 + - name: mysql2 + type: manual + subnets: + - range: 10.10.33.0/24 + reserved: + - 10.10.33.2 - 10.10.33.9 + - 10.10.33.130 - 10.10.33.254 + static: + - 10.10.33.10 - 10.10.33.14 + gateway: 10.10.33.1 + dns: + - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 + cloud_properties: + subnet: REPLACE_WITH_YOUR_SUBNET_ID_2 + - name: mysql3 + type: manual + subnets: + - range: 10.10.34.0/24 + reserved: + - 10.10.34.2 - 10.10.34.9 + - 10.10.34.130 - 10.10.34.254 + static: + - 10.10.34.10 - 10.10.34.14 + gateway: 10.10.34.1 + dns: + - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 + cloud_properties: + subnet: REPLACE_WITH_YOUR_SUBNET_ID_3 diff --git a/templates/sample_stubs/instance_count_overrides.yml b/templates/sample_stubs/instance_count_overrides.yml new file mode 100644 index 00000000..fbcfe630 --- /dev/null +++ b/templates/sample_stubs/instance_count_overrides.yml @@ -0,0 +1 @@ +instance_count_overrides: {} diff --git a/templates/sample_stubs/property_overrides.yml b/templates/sample_stubs/property_overrides.yml new file mode 100644 index 00000000..233147ad --- /dev/null +++ b/templates/sample_stubs/property_overrides.yml @@ -0,0 +1,48 @@ +property_overrides: + syslog_aggregator: ~ + standalone: false + host: YOUR_LOAD_BALANCER_ADDRESS # Optional, set to your Load Balancer address if configured + mysql: + admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process + bootstrap_endpoint: + username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP + password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP + proxy: + api_username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP + api_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP + broker: + auth_username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP # eg. cc + auth_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP + cookie_secret: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP + dashboard_secret: REPLACE_WITH_PASSWORD_YOU_MAKE_UP #Secret used by MySQL dashboard to fetch an OAuth token from UAA + plans: + - name: 100mb + id: ab08f1bc-e6fc-4b56-a767-ee0fea6e3f20 + description: Shared MySQL Server + max_storage_mb: 100 + max_user_connections: 20 + metadata: + costs: + - amount: + usd: 0.0 + unit: MONTH + bullets: + - Shared MySQL server + - 100 MB storage + - 20 concurrent connections + displayName: "100 MB" + - name: 1gb + id: 11d0aa36-dcec-4021-85f5-ea4d9a5c8342 + description: Shared MySQL Server + max_storage_mb: 1000 + max_user_connections: 40 + metadata: + costs: + - amount: + usd: 0.0 + unit: MONTH + bullets: + - Shared MySQL server + - 1000 MB storage + - 40 concurrent connections + displayName: "1 GB" diff --git a/templates/sample_stubs/release_versions.yml b/templates/sample_stubs/release_versions.yml new file mode 100644 index 00000000..c588da05 --- /dev/null +++ b/templates/sample_stubs/release_versions.yml @@ -0,0 +1 @@ +release_versions: {} From eafcbeb6837d48f585aae77c4e15cb49dd6a9116 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 31 Dec 2015 13:54:46 -0800 Subject: [PATCH 021/110] Overhaul manifest generation process to improve consistency with CF / Diego - Moved manifest generation scripts to scripts/{generate-bosh-lite-manifest,generate-deployment-manifest} - Added flags to manifest generation scripts - Pull shared properties from an existing CF manifest instead of copying creds into stubs - Add example stubs for different use cases, e.g. standalone, no-arbitrator - Move templates properties into top-level properties section Signed-off-by: Ben Calegari [#110775864] --- bosh-lite/cf-mysql-standalone-stub.yml | 9 - bosh-lite/cf-mysql-stub-spiff-1-node.yml | 71 - bosh-lite/cf-mysql-stub-spiff-3-node.yml | 52 - bosh-lite/make_manifest | 76 - bosh-lite/manifests/.gitkeep | 0 bosh-lite/tmp/.gitkeep | 0 generate_deployment_manifest | 40 - jobs/acceptance-tests/spec | 82 +- jobs/acceptance-tests/templates/errand.sh.erb | 62 +- jobs/arbitrator/spec | 17 +- .../galera_healthcheck_config.yaml.erb | 10 +- jobs/arbitrator/templates/garbd_config.erb | 4 +- jobs/bootstrap/spec | 10 +- jobs/bootstrap/templates/config.yml.erb | 10 +- jobs/broker-deregistrar/spec | 4 +- .../templates/errand.sh.erb | 2 +- jobs/broker-registrar/spec | 20 +- jobs/broker-registrar/templates/errand.sh.erb | 12 +- jobs/cf-mysql-broker/spec | 95 +- .../templates/database.yml.erb | 8 +- .../templates/quota_enforcer_config.yml.erb | 8 +- .../templates/registrar_settings.yml.erb | 2 +- .../templates/settings.yml.erb | 84 +- jobs/mysql/monit | 4 +- jobs/mysql/spec | 67 +- .../galera_healthcheck_config.yaml.erb | 10 +- .../templates/mariadb_ctl_config.yml.erb | 10 +- jobs/mysql/templates/mariadb_init.erb | 4 +- jobs/mysql/templates/my.cnf.erb | 26 +- jobs/proxy/monit | 4 +- jobs/proxy/spec | 66 +- jobs/proxy/templates/route-registrar.yml.erb | 6 +- jobs/proxy/templates/switchboard.yml.erb | 18 +- .../bosh-lite-stubs/iaas-settings.yml | 0 .../property-overrides-standalone.yml | 28 + .../bosh-lite-stubs/property-overrides.yml | 0 manifest-generation/cf-mysql-template.yml | 304 +++ .../examples/aws/iaas-settings.yml | 0 .../minimal/instance-count-overrides.yml | 11 + .../instance-count-overrides.yml | 5 + .../examples/property-overrides.yml | 0 .../examples/release-versions.yml | 0 .../standalone/instance-count-overrides.yml | 5 + .../standalone/property-overrides.yml | 24 + .../standalone/standalone-cf-manifest.yml | 16 + .../examples/vsphere/iaas-settings.yml | 161 ++ .../misc-templates}/bosh.yml | 0 .../config-from-cf-internal.yml | 0 .../misc-templates}/config-from-cf.yml | 0 scripts/generate-bosh-lite-manifest | 31 + scripts/generate-deployment-manifest | 104 + scripts/test_sample_stubs | 14 + templates/cf-infrastructure-aws.yml | 74 - templates/cf-infrastructure-vsphere.yml | 109 - templates/cf-infrastructure-warden.yml | 2141 ----------------- templates/cf-mysql-node3-arbitrator.yml | 6 - templates/cf-mysql-node3-mysql.yml | 6 - templates/cf-mysql-template.yml | 386 --- .../sample_stubs/instance_count_overrides.yml | 1 - templates/sample_stubs/sample_aws_stub.yml | 132 - templates/sample_stubs/sample_plans_stub.yml | 36 - .../sample_stubs/sample_vsphere_stub.yml | 135 -- 62 files changed, 1014 insertions(+), 3608 deletions(-) delete mode 100644 bosh-lite/cf-mysql-standalone-stub.yml delete mode 100644 bosh-lite/cf-mysql-stub-spiff-1-node.yml delete mode 100644 bosh-lite/cf-mysql-stub-spiff-3-node.yml delete mode 100755 bosh-lite/make_manifest delete mode 100644 bosh-lite/manifests/.gitkeep delete mode 100644 bosh-lite/tmp/.gitkeep delete mode 100755 generate_deployment_manifest rename templates/bosh_lite_stubs/iaas_settings.yml => manifest-generation/bosh-lite-stubs/iaas-settings.yml (100%) create mode 100644 manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml rename templates/bosh_lite_stubs/property_overrides.yml => manifest-generation/bosh-lite-stubs/property-overrides.yml (100%) create mode 100644 manifest-generation/cf-mysql-template.yml rename templates/sample_stubs/aws_iaas_settings.yml => manifest-generation/examples/aws/iaas-settings.yml (100%) create mode 100644 manifest-generation/examples/minimal/instance-count-overrides.yml create mode 100644 manifest-generation/examples/no-arbitrator/instance-count-overrides.yml rename templates/sample_stubs/property_overrides.yml => manifest-generation/examples/property-overrides.yml (100%) rename templates/sample_stubs/release_versions.yml => manifest-generation/examples/release-versions.yml (100%) create mode 100644 manifest-generation/examples/standalone/instance-count-overrides.yml create mode 100644 manifest-generation/examples/standalone/property-overrides.yml create mode 100644 manifest-generation/examples/standalone/standalone-cf-manifest.yml create mode 100644 manifest-generation/examples/vsphere/iaas-settings.yml rename {templates => manifest-generation/misc-templates}/bosh.yml (100%) rename {templates => manifest-generation/misc-templates}/config-from-cf-internal.yml (100%) rename {templates => manifest-generation/misc-templates}/config-from-cf.yml (100%) create mode 100755 scripts/generate-bosh-lite-manifest create mode 100755 scripts/generate-deployment-manifest delete mode 100644 templates/cf-infrastructure-aws.yml delete mode 100644 templates/cf-infrastructure-vsphere.yml delete mode 100644 templates/cf-infrastructure-warden.yml delete mode 100644 templates/cf-mysql-node3-arbitrator.yml delete mode 100644 templates/cf-mysql-node3-mysql.yml delete mode 100644 templates/cf-mysql-template.yml delete mode 100644 templates/sample_stubs/instance_count_overrides.yml delete mode 100644 templates/sample_stubs/sample_aws_stub.yml delete mode 100644 templates/sample_stubs/sample_plans_stub.yml delete mode 100644 templates/sample_stubs/sample_vsphere_stub.yml diff --git a/bosh-lite/cf-mysql-standalone-stub.yml b/bosh-lite/cf-mysql-standalone-stub.yml deleted file mode 100644 index 9906fe78..00000000 --- a/bosh-lite/cf-mysql-standalone-stub.yml +++ /dev/null @@ -1,9 +0,0 @@ ---- -properties: - standalone: true - -jobs: -- name: cf-mysql-broker_z1 - instances: 0 -- name: cf-mysql-broker_z2 - instances: 0 \ No newline at end of file diff --git a/bosh-lite/cf-mysql-stub-spiff-1-node.yml b/bosh-lite/cf-mysql-stub-spiff-1-node.yml deleted file mode 100644 index a89eda86..00000000 --- a/bosh-lite/cf-mysql-stub-spiff-1-node.yml +++ /dev/null @@ -1,71 +0,0 @@ ---- -name: cf-warden-mysql -director_uuid: PLACEHOLDER-DIRECTOR-UUID -releases: - - name: cf-mysql - version: latest - -properties: - app_domains: (( merge || ["bosh-lite.com"] )) - domain: (( merge || "bosh-lite.com" )) - nats: - machines: [10.244.0.6] - password: nats - port: 4222 - user: nats - cf: - admin_username: admin - admin_password: admin - skip_ssl_validation: true - -jobs: - - name: mysql_z1 - instances: 1 - networks: - - name: mysql1 - static_ips: - - 10.244.7.2 - properties: - <<: (( merge || nil )) - admin_password: password - cluster_ips: (( jobs.mysql_z1.networks.[0].static_ips )) - database_startup_timeout: 600 - bootstrap_endpoint: - username: username - password: password - - name: mysql_z2 - instances: 0 - - name: mysql_z3 - instances: 0 - - name: cf-mysql-broker_z1 - instances: 1 - properties: - <<: (( merge || nil )) - auth_password: password - auth_username: admin - cookie_secret: 94046872-2602-4ca6-8d07-8b0da9762477 - skip_ssl_validation: true - services: - - name: p-mysql - dashboard_client: - secret: p-mysql-secret - plans: - - name: 100mb - max_storage_mb: 10 - - name: 1gb - max_storage_mb: 20 - - name: cf-mysql-broker_z2 - instances: 0 - - name: proxy_z1 - properties: - <<: (( merge || nil )) - proxy: - <<: (( merge || nil )) - api_username: username - api_password: password - - name: proxy_z2 - instances: 0 - - name: acceptance-tests - properties: - proxy: - proxy_count: 1 diff --git a/bosh-lite/cf-mysql-stub-spiff-3-node.yml b/bosh-lite/cf-mysql-stub-spiff-3-node.yml deleted file mode 100644 index f6d9c80e..00000000 --- a/bosh-lite/cf-mysql-stub-spiff-3-node.yml +++ /dev/null @@ -1,52 +0,0 @@ ---- -name: cf-warden-mysql -director_uuid: PLACEHOLDER-DIRECTOR-UUID -releases: - - name: cf-mysql - version: latest - -properties: - app_domains: (( merge || ["bosh-lite.com"] )) - domain: (( merge || "bosh-lite.com" )) - nats: - machines: [10.244.0.6] - password: nats - port: 4222 - user: nats - cf: - admin_username: admin - admin_password: admin - skip_ssl_validation: true - -jobs: - - name: mysql_z1 - properties: - <<: (( merge || nil )) - admin_password: password - database_startup_timeout: 600 - bootstrap_endpoint: - username: username - password: password - - name: cf-mysql-broker_z1 - properties: - <<: (( merge || nil )) - auth_password: password - auth_username: admin - cookie_secret: 94046872-2602-4ca6-8d07-8b0da9762477 - skip_ssl_validation: true - services: - - name: p-mysql - dashboard_client: - secret: p-mysql-secret - plans: - - name: 100mb - max_storage_mb: 10 - - name: 1gb - max_storage_mb: 20 - - name: proxy_z1 - properties: - <<: (( merge || nil )) - proxy: - <<: (( merge || nil )) - api_username: username - api_password: password diff --git a/bosh-lite/make_manifest b/bosh-lite/make_manifest deleted file mode 100755 index 6d47a8bd..00000000 --- a/bosh-lite/make_manifest +++ /dev/null @@ -1,76 +0,0 @@ -#!/bin/bash - -set -eu - -MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" - -# This script expects to live one directory below the release directory. -RELEASE_DIR=$( cd ${MY_DIR}/.. && pwd ) - -# And that the release directory lives one directory below the workspace directory. -WORKSPACE_DIR="$( cd ${RELEASE_DIR}/.. && pwd )" - -OK_COLOR='\033[1;32m' -ACTION_COLOR='\033[1;33m' -INFO_COLOR='\033[1;36m' -NO_COLOR='\033[0m' - -BOSH_LITE_USERNAME=${BOSH_LITE_USERNAME:-admin} -BOSH_LITE_PASSWORD=${BOSH_LITE_PASSWORD:-admin} - -ARBITRATOR=${ARBITRATOR:-"true"} - -if [ "${ARBITRATOR}" != "true" ] && \ - [ "${ARBITRATOR}" != "false" ]; then - echo "Error: ARBITRATOR Env var must be set to true or false" - exit 1 -elif [ "${ARBITRATOR}" == "true" ]; then - DEP_FLAG="arbitrator" -else - DEP_FLAG="no-arbitrator" -fi - -# By default use 3 node stub -mysql_stub="cf-mysql-stub-spiff-3-node.yml" -# If any arguments were provided, -# see if there is a stub with the number of nodes provided. -# If not, assume the argument is not the number of nodes and continue. -if [[ $# -ne 0 ]] && [[ -f "${RELEASE_DIR}/bosh-lite/cf-mysql-stub-spiff-$1-node.yml" ]] ; then - mysql_stub="cf-mysql-stub-spiff-$1-node.yml" - echo -e "Using stub ${INFO_COLOR}${RELEASE_DIR}/bosh-lite/${mysql_stub}${NO_COLOR}" - shift -fi - -mkdir -p ${RELEASE_DIR}/bosh-lite/tmp -mkdir -p ${RELEASE_DIR}/bosh-lite/manifest - -cp \ - ${RELEASE_DIR}/bosh-lite/$mysql_stub \ - ${RELEASE_DIR}/bosh-lite/tmp/cf-mysql-stub-with-uuid.yml - -DIRECTOR_IP=${DIRECTOR_IP:-192.168.50.4} - -# If a file exists with an api-address, use that instead -if [ -f ${WORKSPACE_DIR}/api-address ]; then - API_ADDRESS=$( cat ${WORKSPACE_DIR}/api-address ) - echo -e "WARNING: Found API address file at ${INFO_COLOR}${WORKSPACE_DIR}/api-address${NO_COLOR} - - using ${INFO_COLOR}${API_ADDRESS}${NO_COLOR} instead of ${INFO_COLOR}${DIRECTOR_IP}${NO_COLOR} for director IP - - and ${INFO_COLOR}${API_ADDRESS}.xip.io${NO_COLOR} instead of ${INFO_COLOR}bosh-lite.com${NO_COLOR} for API address" - DIRECTOR_IP=${API_ADDRESS} - perl -pi -e "s/bosh-lite.com/${API_ADDRESS}.xip.io/g" $RELEASE_DIR/bosh-lite/tmp/cf-mysql-stub-with-uuid.yml -fi - -bosh -n target ${DIRECTOR_IP} -bosh -n login "${BOSH_LITE_USERNAME}" "${BOSH_LITE_PASSWORD}" -DIRECTOR_UUID=$(bosh status | grep UUID | awk '{print $2}') -echo $DIRECTOR_UUID -perl -pi -e "s/PLACEHOLDER-DIRECTOR-UUID/$DIRECTOR_UUID/g" $RELEASE_DIR/bosh-lite/tmp/cf-mysql-stub-with-uuid.yml - -${RELEASE_DIR}/generate_deployment_manifest \ - warden \ - ${DEP_FLAG} \ - ${RELEASE_DIR}/templates/sample_stubs/sample_plans_stub.yml \ - ${RELEASE_DIR}/bosh-lite/tmp/cf-mysql-stub-with-uuid.yml \ - "$@" > ${RELEASE_DIR}/bosh-lite/manifests/cf-mysql-manifest.yml - -bosh deployment $RELEASE_DIR/bosh-lite/manifests/cf-mysql-manifest.yml diff --git a/bosh-lite/manifests/.gitkeep b/bosh-lite/manifests/.gitkeep deleted file mode 100644 index e69de29b..00000000 diff --git a/bosh-lite/tmp/.gitkeep b/bosh-lite/tmp/.gitkeep deleted file mode 100644 index e69de29b..00000000 diff --git a/generate_deployment_manifest b/generate_deployment_manifest deleted file mode 100755 index 9d6b6216..00000000 --- a/generate_deployment_manifest +++ /dev/null @@ -1,40 +0,0 @@ -#!/bin/bash - -MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" - -templates="${MY_DIR}/templates" - -infrastructure="$1" - -if [ -z "$2" ]; then - arb="arbitrator" -elif [ "$2" != "arbitrator" ] && [ "$2" != "no-arbitrator" ]; then - arb="arbitrator" -elif [ "$2" == "arbitrator" ] || [ "$2" == "no-arbitrator" ]; then - arb=$2 - shift -fi - -if [ "$infrastructure" != "aws" ] && \ - [ "$infrastructure" != "warden" ] && \ - [ "$infrastructure" != "vsphere" ] ; then - echo "usage: ./generate_deployment_manifest {no-arbitrator} [stubs...]" - exit 1 -fi - -shift - -if [ "${arb}" == "arbitrator" ]; then - spiff merge \ - "$templates/cf-mysql-template.yml" \ - "$templates/cf-mysql-node3-arbitrator.yml" \ - "$templates/cf-infrastructure-${infrastructure}.yml" \ - $* -else - spiff merge \ - "$templates/cf-mysql-template.yml" \ - "$templates/cf-mysql-node3-mysql.yml" \ - "$templates/cf-infrastructure-${infrastructure}.yml" \ - $* -fi - diff --git a/jobs/acceptance-tests/spec b/jobs/acceptance-tests/spec index fec8c560..c1c3a39e 100644 --- a/jobs/acceptance-tests/spec +++ b/jobs/acceptance-tests/spec @@ -15,63 +15,51 @@ properties: description: 'Username of the admin user' cf.admin_password: description: 'Password of the admin user' - cf.apps_domain: - description: 'Shared domain for pushed apps' + cf.app_domains: + description: 'List of shared domains for pushed apps (tests will use the first domain)' cf.skip_ssl_validation: description: 'Whether to add --skip-ssl-validation for cf cli' default: false - - broker.host: - description: 'Host for the broker. Should be the same as jobs.cf-mysql-broker.properties.external_host' - - service.name: - description: 'Name of the MySQL service' - service.plans: - description: 'Array of MySQL plans' - - service.max_user_connections_default: - description: 'number of user connections to allow in a plan if not specified' - default: 40 - - org_name: + cf.smoke_tests.use_existing_org: + description: 'Runs smoke test errand as an existing org. Creates a new org if false' + default: false + cf.smoke_tests.org: description: 'The name of the Org to run acceptance tests against' default: '' - smoke_tests_only: + cf_mysql.acceptance_tests.smoke_tests_only: description: 'Instead of running the full acceptance test suite, only run a shorter smoke test' default: true - smoke_test_password: - description: 'Password for smoke tests to comply with CF password policy, if exists.' - default: '@Ma8rtj2vu' - - proxy.external_host: - description: 'Proxy external host (e.g. p-mysql.example.com => proxy-0-p-mysql.example.com)' - proxy.api_username: - description: 'Proxy API username' - proxy.api_password: - description: 'Proxy API password' - proxy.api_force_https: - description: 'Expect proxy to force redirect to HTTPS' - default: true - proxy.skip_ssl_validation: - description: 'Tests will skip validation of SSL certificates' - default: true - proxy.proxy_count: - description: 'Number of proxy instances. Use to construct an array of proxy dashboard url (e.g. https://proxy-INDEX-EXTERNAL_HOST)' - standalone_tests_only: + cf_mysql.acceptance_tests.standalone_tests_only: description: 'Instead of running the full acceptance test suite, only run a minimal set of tests that do not require a running CF deployment' default: false - standalone.host: - description: 'Hostname of proxy or load balancer used to communicate with MySQL nodes in a standalone deployment' - standalone.port: - description: 'Port used to communicate with MySQL nodes' + cf_mysql.acceptance_tests.timeout_scale: + description: 'scaling multiplier used to determine command timeouts' + default: 1 + + cf_mysql.host: + description: 'Hostname given to apps by the broker, usually a Load Balancer address or the IP of the first proxy' + cf_mysql.external_host: + description: 'Hostname given for dashboard URLs (e.g. p-mysql.example.com)' + + cf_mysql.mysql.port: + description: 'Port used to access MySQL' default: 3306 - standalone.username: - description: 'MySQL admin username' + cf_mysql.mysql.admin_username: + description: 'Admin username used to access MySQL' default: 'root' - standalone.password: - description: 'MySQL admin password' + cf_mysql.mysql.admin_password: + description: 'Admin password used to access MySQL' - timeout_scale: - description: 'scaling multiplier used to determine command timeouts' - default: 1 + cf_mysql.broker.services: + description: 'Array of Services the broker provides' + + cf_mysql.proxy.api_username: + description: 'Proxy API username' + cf_mysql.proxy.api_password: + description: 'Proxy API password' + cf_mysql.proxy.api_force_https: + description: 'Expect proxy to force redirect to HTTPS' + default: true + cf_mysql.proxy.proxy_ips: + description: 'Use to construct an array of proxy dashboard url (e.g. https://proxy-INDEX-EXTERNAL_HOST)' diff --git a/jobs/acceptance-tests/templates/errand.sh.erb b/jobs/acceptance-tests/templates/errand.sh.erb index 3c1ce8fe..810433f6 100644 --- a/jobs/acceptance-tests/templates/errand.sh.erb +++ b/jobs/acceptance-tests/templates/errand.sh.erb @@ -3,25 +3,29 @@ set -e -x cd /var/vcap/packages/acceptance-tests/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests +<% mysql_service = p('cf_mysql.broker.services').first %> +<% default_max_user_connections = mysql_service['max_user_connections_default'] || 40 %> + cat > integration_config.json <", - "apps_domain": "<%= p('cf.apps_domain') %>", + "apps_domain": "<%= p('cf.app_domains').first %>", "admin_user": "<%= p('cf.admin_username') %>", - "broker_host": "<%= p('broker.host') %>", - "service_name": "<%= p('service.name') %>", - "test_password": "<%= p('smoke_test_password') %>", - <% if p('org_name') %> - "org_name": "<%= p('org_name') %>", + "admin_password": "<%= p('cf.admin_password') %>", + "test_password": "@Ma8rtj2vu", + "broker_host": "<%= p('cf_mysql.external_host') %>", + "service_name": "<%= mysql_service['name'] %>", + <% if p('cf.smoke_tests.use_existing_org') %> + "org_name": "<%= p('cf.smoke_tests.org') %>", <% end %> "plans": [ - <% plans = p('service.plans') + <% plans = mysql_service['plans'] plans.each do |plan| %> { "plan_name": "<%= plan['name'] %>", "private": <%= !plan.has_key?('private') ? false : plan['private'] %>, "max_storage_mb": <%= plan['max_storage_mb'] %>, - "max_user_connections": <%= plan['max_user_connections'] || p('service.max_user_connections_default')%> + "max_user_connections": <%= plan['max_user_connections'] || default_max_user_connections %> <% if plan != plans.last %> }, <% else %> @@ -31,52 +35,42 @@ cat > integration_config.json <, "proxy": { - "dashboard_urls": [ - <% p('proxy.proxy_count').times do |index| %> - <% proxy_url = "\"https://proxy-#{index}-#{p('proxy.external_host')}\"" %> - <% proxy_url += "," unless index == p('proxy.proxy_count') - 1 %> - <%= proxy_url %> - <% end %> - ], - "api_username": "<%= p('proxy.api_username') %>", - "api_password": "<%= p('proxy.api_password') %>", - "api_force_https": <%= p('proxy.api_force_https') %>, - "skip_ssl_validation": <%= p('proxy.skip_ssl_validation') %> + <% urls = p('cf_mysql.proxy.proxy_ips').each_with_index.map do |_, index| + "\"https://proxy-#{index}-#{p('cf_mysql.external_host')}\"" + end %> + "dashboard_urls": [<%= urls.join(",") %>], + "api_username": "<%= p('cf_mysql.proxy.api_username') %>", + "api_password": "<%= p('cf_mysql.proxy.api_password') %>", + "api_force_https": <%= p('cf_mysql.proxy.api_force_https') %>, + "skip_ssl_validation": <%= p('cf.skip_ssl_validation') %> }, - "timeout_scale": <%= p('timeout_scale') %>, EOF -<% if p('standalone_tests_only') %> +<% if p('cf_mysql.acceptance_tests.standalone_tests_only') %> cat >> integration_config.json <", - "port": <%= p('standalone.port') %>, - "username": "<%= p('standalone.username') %>", - "password": "<%= p('standalone.password') %>" + "host": "<%= p('cf_mysql.host') %>", + "port": <%= p('cf_mysql.mysql.port') %>, + "username": "<%= p('cf_mysql.mysql.admin_username') %>", + "password": "<%= p('cf_mysql.mysql.admin_password') %>" }, EOF <% end %> -# don't expose passwords -set +x - cat >> integration_config.json <" + "timeout_scale": <%= p('cf_mysql.acceptance_tests.timeout_scale') %> } EOF -# set -x - export GOPATH=/var/vcap/packages/acceptance-tests export GOROOT=/var/vcap/packages/golang export PATH=/var/vcap/packages/cli/bin:$GOPATH/bin:$GOROOT/bin:$PWD/bin:$PATH export CONFIG=$PWD/integration_config.json - -<% if p('standalone_tests_only') %> +<% if p('cf_mysql.acceptance_tests.standalone_tests_only') %> bin/test-standalone --noColor -<% elsif p('smoke_tests_only') %> +<% elsif p('cf_mysql.acceptance_tests.smoke_tests_only') %> bin/test-smoke --noColor <% else %> bin/test-acceptance --noColor diff --git a/jobs/arbitrator/spec b/jobs/arbitrator/spec index af2c94ad..93ca6f85 100644 --- a/jobs/arbitrator/spec +++ b/jobs/arbitrator/spec @@ -13,24 +13,23 @@ packages: - common properties: - admin_username: + cf_mysql.mysql.admin_username: description: 'Username for the MySQL server admin user' default: 'root' - admin_password: + cf_mysql.mysql.admin_password: description: 'Password for the MySQL server admin user' - port: + cf_mysql.mysql.port: description: 'Port the mysql server should bind to' default: 3306 - galera_port: + cf_mysql.mysql.galera_port: description: 'Port which garbd listens on' default: 4567 - healthcheck_port: + cf_mysql.mysql.healthcheck_port: description: 'Port used by healthcheck process to listen on' default: 9200 - cluster_ips: + cf_mysql.mysql.cluster_ips: description: 'List of nodes. Must have the same number of ips as there are nodes in the cluster' - - bootstrap_endpoint.username: + cf_mysql.mysql.bootstrap_endpoint.username: description: 'Username used by the bootstrap endpoints for Basic Auth' - bootstrap_endpoint.password: + cf_mysql.mysql.bootstrap_endpoint.password: description: 'Password used by the bootstrap endpoints for Basic Auth' diff --git a/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb index 1fdd1ce6..690dc9f4 100644 --- a/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb +++ b/jobs/arbitrator/templates/galera_healthcheck_config.yaml.erb @@ -1,10 +1,10 @@ --- -Port: <%= p('healthcheck_port') %> +Port: <%= p('cf_mysql.mysql.healthcheck_port') %> PidFile: '/var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid' MysqldPath: '/var/vcap/packages/mariadb/bin/mysqld' DB: - User: <%= p('admin_username') %> - Password: <%= p('admin_password') %> + User: <%= p('cf_mysql.mysql.admin_username') %> + Password: <%= p('cf_mysql.mysql.admin_password') %> # This is the config that bosh sets up by default for monit. Monit: User: 'vcap' @@ -13,5 +13,5 @@ Monit: Port: 2822 ServiceName: 'garbd' BootstrapEndpoint: - Username: <%= p('bootstrap_endpoint.username') %> - Password: <%= p('bootstrap_endpoint.password') %> + Username: <%= p('cf_mysql.mysql.bootstrap_endpoint.username') %> + Password: <%= p('cf_mysql.mysql.bootstrap_endpoint.password') %> diff --git a/jobs/arbitrator/templates/garbd_config.erb b/jobs/arbitrator/templates/garbd_config.erb index cd993740..2dad092f 100644 --- a/jobs/arbitrator/templates/garbd_config.erb +++ b/jobs/arbitrator/templates/garbd_config.erb @@ -1,4 +1,4 @@ group = cf-mariadb-galera-cluster -<% galera_port = p('galera_port') %> -<% node_addresses = p('cluster_ips').map { |ip| "#{ip}:#{galera_port}" } %> +<% galera_port = p('cf_mysql.mysql.galera_port') %> +<% node_addresses = p('cf_mysql.mysql.cluster_ips').map { |ip| "#{ip}:#{galera_port}" } %> address = gcomm://<%= node_addresses.join(',') %> diff --git a/jobs/bootstrap/spec b/jobs/bootstrap/spec index f3380dff..8fad76c7 100644 --- a/jobs/bootstrap/spec +++ b/jobs/bootstrap/spec @@ -6,14 +6,14 @@ templates: errand.sh.erb: bin/run config.yml.erb: config/config.yml properties: - healthcheck_port: + cf_mysql.mysql.healthcheck_port: description: 'Port used by healthcheck process to listen on' default: 9200 - cluster_ips: + cf_mysql.mysql.cluster_ips: description: 'IP addresses of all MySQL nodes' - database_startup_timeout: + cf_mysql.mysql.database_startup_timeout: description: 'Maximum time to transition to or from a stopped or started state before we timeout.' - bootstrap_endpoint.username: + cf_mysql.mysql.bootstrap_endpoint.username: description: 'Username used to contact the bootstrap endpoints via Basic Auth' - bootstrap_endpoint.password: + cf_mysql.mysql.bootstrap_endpoint.password: description: 'Password used to contact the bootstrap endpoints via Basic Auth' diff --git a/jobs/bootstrap/templates/config.yml.erb b/jobs/bootstrap/templates/config.yml.erb index 2898401c..ca4c3df4 100644 --- a/jobs/bootstrap/templates/config.yml.erb +++ b/jobs/bootstrap/templates/config.yml.erb @@ -1,7 +1,7 @@ HealthcheckURLs: - <% p('cluster_ips').each do |ip| %> - - http://<%= ip %>:<%= p('healthcheck_port') %> + <% p('cf_mysql.mysql.cluster_ips').each do |ip| %> + - http://<%= ip %>:<%= p('cf_mysql.mysql.healthcheck_port') %> <% end %> -DatabaseStartupTimeout: <%= p('database_startup_timeout') %> -Username: <%= p('bootstrap_endpoint.username') %> -Password: <%= p('bootstrap_endpoint.password') %> +DatabaseStartupTimeout: <%= p('cf_mysql.mysql.database_startup_timeout') %> +Username: <%= p('cf_mysql.mysql.bootstrap_endpoint.username') %> +Password: <%= p('cf_mysql.mysql.bootstrap_endpoint.password') %> diff --git a/jobs/broker-deregistrar/spec b/jobs/broker-deregistrar/spec index 6b8bf6be..d89f2545 100644 --- a/jobs/broker-deregistrar/spec +++ b/jobs/broker-deregistrar/spec @@ -14,5 +14,5 @@ properties: description: 'Username of the admin user' cf.admin_password: description: 'Password of the admin user' - broker.name: - description: 'Name of the service broker' + cf_mysql.broker.services: + description: 'List of services the broker provides' diff --git a/jobs/broker-deregistrar/templates/errand.sh.erb b/jobs/broker-deregistrar/templates/errand.sh.erb index 41ab0b12..3bb8dde5 100644 --- a/jobs/broker-deregistrar/templates/errand.sh.erb +++ b/jobs/broker-deregistrar/templates/errand.sh.erb @@ -6,7 +6,7 @@ CF_API_URL='<%= p("cf.api_url") %>' CF_ADMIN_USERNAME='<%= p("cf.admin_username") %>' CF_ADMIN_PASSWORD='<%= p("cf.admin_password") %>' -BROKER_NAME='<%= p("broker.name") %>' +BROKER_NAME='<%= p("cf_mysql.broker.services").first["name"] %>' export PATH=/var/vcap/packages/cli/bin diff --git a/jobs/broker-registrar/spec b/jobs/broker-registrar/spec index ded099fd..0370a065 100644 --- a/jobs/broker-registrar/spec +++ b/jobs/broker-registrar/spec @@ -14,19 +14,19 @@ properties: description: 'Username of the admin user' cf.admin_password: description: 'Password of the admin user' - broker.name: - description: 'Name of the service broker' - broker.protocol: + + cf_mysql.external_host: + description: 'Host address of the service broker' + + cf_mysql.broker.services: + description: 'List of services the broker provides' + cf_mysql.broker.protocol: description: 'Protocol (http/https) provided to the Cloud Controller when registering/deregistering the broker' default: https - broker.host: - description: 'Host address of the service broker' - broker.port: + cf_mysql.broker.port: description: 'Port for the service broker' default: 443 - broker.username: + cf_mysql.broker.auth_username: description: 'Basic Auth username for the service broker' - broker.password: + cf_mysql.broker.auth_password: description: 'Basic Auth password for the service broker' - broker.services: - description: 'Basic plans object for registering' \ No newline at end of file diff --git a/jobs/broker-registrar/templates/errand.sh.erb b/jobs/broker-registrar/templates/errand.sh.erb index e3bfe0ea..41977089 100644 --- a/jobs/broker-registrar/templates/errand.sh.erb +++ b/jobs/broker-registrar/templates/errand.sh.erb @@ -7,10 +7,10 @@ CF_ADMIN_USERNAME='<%= p("cf.admin_username") %>' CF_ADMIN_PASSWORD='<%= p("cf.admin_password") %>' CF_SKIP_SSL_VALIDATION='<%= p("cf.skip_ssl_validation") %>' -BROKER_NAME='<%= p("broker.name") %>' -BROKER_URL='<%= p("broker.protocol") %>://<%= p("broker.host") %>:<%= p("broker.port") %>' -BROKER_USERNAME='<%= p("broker.username") %>' -BROKER_PASSWORD='<%= p("broker.password") %>' +BROKER_NAME='<%= p("cf_mysql.broker.services").first["name"] %>' +BROKER_URL='<%= p("cf_mysql.broker.protocol") %>://<%= p("cf_mysql.external_host") %>:<%= p("cf_mysql.broker.port") %>' +BROKER_USERNAME='<%= p("cf_mysql.broker.auth_username") %>' +BROKER_PASSWORD='<%= p("cf_mysql.broker.auth_password") %>' function updateServiceBroker() { echo "Service broker already exists - updating broker" @@ -45,11 +45,11 @@ cf auth \ createServiceBroker || updateServiceBroker # Loop over plans for p-mysql broker and enable if is public -<% service = p('broker.services').first %> +<% service = p('cf_mysql.broker.services').first %> <% service['plans'].each do |plan| %> # Default plan to public if not specified in manifest <% is_private = ( !plan.has_key?('private') ? false : plan['private'] ) %> <% unless is_private %> cf enable-service-access ${BROKER_NAME} -p '<%= plan['name'] %>' <% end %> -<% end %> \ No newline at end of file +<% end %> diff --git a/jobs/cf-mysql-broker/spec b/jobs/cf-mysql-broker/spec index b613bf08..20e39da0 100644 --- a/jobs/cf-mysql-broker/spec +++ b/jobs/cf-mysql-broker/spec @@ -24,75 +24,68 @@ packages: - quota-enforcer properties: - auth_username: - description: "Broker's basic auth username" - auth_password: - description: "Broker's basic auth password" + nats.user: + description: 'Username for broker to register a route with NATS' + nats.password: + description: 'Password for broker to register a route with NATS' + nats.port: + description: 'IP port of Cloud Foundry NATS server' + nats.machines: + description: 'IP of each NATS cluster member.' - cookie_secret: - description: 'A unique secret key, used to sign sessions' + cf.api_url: + description: 'URL of the CloudFoundry Cloud Controller' + cf.skip_ssl_validation: + description: 'Determines whether dashboard verifies SSL certificates when communicating with Cloud Controller and UAA' + default: false + + syslog_aggregator.address: + description: "IP address for syslog aggregator" + syslog_aggregator.port: + description: "TCP port of syslog aggregator" + syslog_aggregator.all: + description: "Define whether forwarders should also send non-mysql syslog activity to the aggregator." + default: false + syslog_aggregator.transport: + description: "Transport to be used when forwarding logs (tcp|udp|relp)." + default: "tcp" - mysql_node.host: + cf_mysql.host: description: 'Host (DNS) or IP address used by the broker and bound applications to reach the service' - mysql_node.port: + cf_mysql.external_host: + description: 'Host used to register a route for the broker with the router in cf-release via NATS. By registering the broker with Cloud Foundry using this route, and using the router for load balancing, the broker can be horizontally scaled. Domain is typically the same as system domain in cf-release.' + + cf_mysql.mysql.port: description: 'Port used by the broker and bound applications to reach the service' default: 3306 - mysql_node.admin_username: + cf_mysql.mysql.admin_username: description: 'Username for the admin MySQL account' default: 'root' - mysql_node.admin_password: + cf_mysql.mysql.admin_password: description: 'Password for the admin MySQL account' - mysql_node.persistent_disk: + cf_mysql.mysql.persistent_disk: description: 'Size of the persistent disk allocated to the MySQL node for storage' - mysql_node.gcache_size: + cf_mysql.mysql.gcache_size: description: 'Cache size used by galera (maximum amount of data possible in an IST), in MB' default: 512 - mysql_node.ib_log_file_size: + cf_mysql.mysql.ib_log_file_size: description: 'Size of the ib_log_file used by innodb, in MB' default: 1024 - max_user_connections_default: + cf_mysql.broker.auth_username: + description: "Broker's basic auth username" + cf_mysql.broker.auth_password: + description: "Broker's basic auth password" + cf_mysql.broker.cookie_secret: + description: 'A unique secret key, used to sign sessions' + cf_mysql.broker.max_user_connections_default: description: 'number of user connections to allow in a plan if not specified' default: 40 - - services: + cf_mysql.broker.services: description: 'Services and plans offered by the broker' - - cc_api_uri: - description: 'URL of the CloudFoundry Cloud Controller' - - external_host: - description: 'Host used to register a route for the broker with the router in cf-release via NATS. By registering the broker with Cloud Foundry using this route, and using the router for load balancing, the broker can be horizontally scaled. Domain is typically the same as system domain in cf-release.' - - ssl_enabled: + cf_mysql.broker.ssl_enabled: description: 'Determines use of https in dashboard url and in callback uri for calls to UAA' default: true - - skip_ssl_validation: - description: 'Determines whether dashboard verifies SSL certificates when communicating with Cloud Controller and UAA' - default: false - - nats.user: - description: 'Username for broker to register a route with NATS' - nats.password: - description: 'Password for broker to register a route with NATS' - nats.port: - description: 'IP port of Cloud Foundry NATS server' - nats.machines: - description: 'IP of each NATS cluster member.' - - networks.broker_network: - description: 'Name of the network the broker will get its IP address from' - + network_name: description: "The name of the network (needed for the syslog aggregator)" - syslog_aggregator.address: - description: "IP address for syslog aggregator" - syslog_aggregator.port: - description: "TCP port of syslog aggregator" - syslog_aggregator.all: - description: "Define whether forwarders should also send non-mysql syslog activity to the aggregator." - default: false - syslog_aggregator.transport: - description: "Transport to be used when forwarding logs (tcp|udp|relp)." - default: "tcp" diff --git a/jobs/cf-mysql-broker/templates/database.yml.erb b/jobs/cf-mysql-broker/templates/database.yml.erb index 7faecd83..38092ba4 100644 --- a/jobs/cf-mysql-broker/templates/database.yml.erb +++ b/jobs/cf-mysql-broker/templates/database.yml.erb @@ -17,7 +17,7 @@ production: connect_timeout: 20 read_timeout: 20 reconnect: true - host: <%= p('mysql_node.host') %> - port: <%= p('mysql_node.port') %> - username: <%= p('mysql_node.admin_username') %> - password: <%= p('mysql_node.admin_password') %> + host: <%= p('cf_mysql.host') %> + port: <%= p('cf_mysql.mysql.port') %> + username: <%= p('cf_mysql.mysql.admin_username') %> + password: <%= p('cf_mysql.mysql.admin_password') %> diff --git a/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb b/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb index e3c7cf0d..f9d19ba5 100644 --- a/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb +++ b/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb @@ -1,6 +1,6 @@ --- -Host: <%= p('mysql_node.host') %> -Port: <%= p('mysql_node.port') %> -User: <%= p('mysql_node.admin_username') %> -Password: <%= p('mysql_node.admin_password') %> +Host: <%= p('cf_mysql.host') %> +Port: <%= p('cf_mysql.mysql.port') %> +User: <%= p('cf_mysql.mysql.admin_username') %> +Password: <%= p('cf_mysql.mysql.admin_password') %> DBName: mysql_broker diff --git a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb index 877e3eb9..9ef187d8 100644 --- a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb +++ b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb @@ -4,7 +4,7 @@ message_bus_servers: user: <%= p('nats.user') %> password: <%= p('nats.password') %> <% end %> -external_host: <%= p('external_host') %> +external_host: <%= p('cf_mysql.external_host') %> external_ip: <%= spec.networks.send(p('network_name')).ip %> port: 80 health_checker: diff --git a/jobs/cf-mysql-broker/templates/settings.yml.erb b/jobs/cf-mysql-broker/templates/settings.yml.erb index 6c0e101c..afcd98cb 100644 --- a/jobs/cf-mysql-broker/templates/settings.yml.erb +++ b/jobs/cf-mysql-broker/templates/settings.yml.erb @@ -1,65 +1,65 @@ production: database_config_path: '/var/vcap/jobs/cf-mysql-broker/config/database.yml' - auth_username: '<%= p('auth_username') %>' - auth_password: '<%= p('auth_password') %>' - cookie_secret: <%= p('cookie_secret') %> + auth_username: '<%= p('cf_mysql.broker.auth_username') %>' + auth_password: '<%= p('cf_mysql.broker.auth_password') %>' + cookie_secret: <%= p('cf_mysql.broker.cookie_secret') %> session_expiry: 86400 - external_host: <%= p('external_host') %> + external_host: <%= p('cf_mysql.external_host') %> - ssl_enabled: <%= p('ssl_enabled') %> - skip_ssl_validation: <%= p('skip_ssl_validation') %> + ssl_enabled: <%= p('cf_mysql.broker.ssl_enabled') %> + skip_ssl_validation: <%= p('cf.skip_ssl_validation') %> - cc_api_uri: <%= p('cc_api_uri') %> + cc_api_uri: <%= p('cf.api_url') %> - persistent_disk: <%= p('mysql_node.persistent_disk') %> - ib_log_file_size: <%= p('mysql_node.ib_log_file_size')%> - gcache_size: <%= p('mysql_node.gcache_size')%> + persistent_disk: <%= p('cf_mysql.mysql.persistent_disk') %> + ib_log_file_size: <%= p('cf_mysql.mysql.ib_log_file_size')%> + gcache_size: <%= p('cf_mysql.mysql.gcache_size')%> services: - <% (properties.services || []).each do |service| %> - - id: <%= service.id %> - name: <%= service.name %> - plan_updateable: <%= service.plan_updateable %> - description: <%= service.description %> + <% p('cf_mysql.broker.services').each do |service| %> + - id: <%= service['id'] %> + name: <%= service['name'] %> + plan_updateable: <%= service['plan_updateable'] %> + description: <%= service['description'] %> tags: - <% (service.tags || []).each do |tag| %> + <% (service['tags'] || []).each do |tag| %> - <%= tag %> <% end %> metadata: - displayName: <%= service.metadata.displayName %> - imageUrl: <%= service.metadata.imageUrl %> - longDescription: <%= service.metadata.longDescription %> - providerDisplayName: <%= service.metadata.providerDisplayName %> - documentationUrl: <%= service.metadata.documentationUrl %> - supportUrl: <%= service.metadata.supportUrl %> + displayName: <%= service['metadata']['displayName'] %> + imageUrl: <%= service['metadata']['imageUrl'] %> + longDescription: <%= service['metadata']['longDescription'] %> + providerDisplayName: <%= service['metadata']['providerDisplayName'] %> + documentationUrl: <%= service['metadata']['documentationUrl'] %> + supportUrl: <%= service['metadata']['supportUrl'] %> plans: - <% (service.plans || []).each do |plan| %> - - id: <%= plan.id || plan.guid %> + <% (service['plans'] || []).each do |plan| %> + - id: <%= plan['id'] || plan['guid'] %> <% - if plan.name !~ /^[a-z0-9\_\-]+$/ - raise "Plan name '#{plan.name}' must only contain lowercase letters, numbers, hyphen(-), or underscore(_)." + if plan['name'] !~ /^[a-z0-9\_\-]+$/ + raise "Plan name '#{plan['name']}' must only contain lowercase letters, numbers, hyphen(-), or underscore(_)." end %> - name: <%= plan.name %> - description: <%= plan.description %> - max_storage_mb: <%= plan.max_storage_mb %> + name: <%= plan['name'] %> + description: <%= plan['description'] %> + max_storage_mb: <%= plan['max_storage_mb'] %> - <% max_user_connections = plan.max_user_connections || p('max_user_connections_default') %> + <% max_user_connections = plan['max_user_connections'] || p('cf_mysql.broker.max_user_connections_default') %> max_user_connections: <%= max_user_connections %> <% - if !plan.metadata + if !plan['metadata'] costs = [] bullets = [ - "#{plan.description}", - "#{plan.max_storage_mb} MB storage", + "#{plan['description']}", + "#{plan['max_storage_mb']} MB storage", "#{max_user_connections} concurrent connections" ] - display_name = plan.name + display_name = plan['name'] else - costs = plan.metadata.costs || [] - bullets = plan.metadata.bullets || [] - display_name = plan.metadata.displayName + costs = plan['metadata']['costs'] || [] + bullets = plan['metadata']['bullets'] || [] + display_name = plan['metadata']['displayName'] end %> @@ -73,8 +73,8 @@ production: costs: <% costs.each do |cost| %> - amount: - usd: <%= cost.amount.usd %> - unit: <%= cost.unit %> + usd: <%= cost['amount']['usd'] %> + unit: <%= cost['unit'] %> <% end %> <% end %> @@ -90,7 +90,7 @@ production: displayName: <%= display_name %> <% end %> dashboard_client: - id: <%= service.dashboard_client.id %> - secret: <%= service.dashboard_client.secret %> - redirect_uri: <%= "#{p('ssl_enabled') ? 'https' : 'http'}://#{p('external_host')}/" %> + id: <%= service['dashboard_client']['id'] %> + secret: <%= service['dashboard_client']['secret'] %> + redirect_uri: <%= "#{p('cf_mysql.broker.ssl_enabled') ? 'https' : 'http'}://#{p('cf_mysql.external_host')}/" %> <% end %> diff --git a/jobs/mysql/monit b/jobs/mysql/monit index 0dbc67de..0de1e944 100644 --- a/jobs/mysql/monit +++ b/jobs/mysql/monit @@ -1,4 +1,4 @@ -<% monit_db_timeout = (p('database_startup_timeout') * 0.9 ).round %> +<% monit_db_timeout = ( p('cf_mysql.mysql.database_startup_timeout') * 0.9 ).round %> check process mariadb_ctrl with pidfile /var/vcap/sys/run/mariadb_ctl/mariadb_ctl.pid start program "/var/vcap/jobs/mysql/bin/mariadb_ctl start" with timeout <%= monit_db_timeout %> seconds @@ -18,5 +18,3 @@ check process gra-log-purger-executable stop program "/var/vcap/jobs/mysql/bin/gra-log-purger_ctl stop" with timeout 10 seconds group vcap - - diff --git a/jobs/mysql/spec b/jobs/mysql/spec index a4104106..9e9b4a19 100644 --- a/jobs/mysql/spec +++ b/jobs/mysql/spec @@ -24,68 +24,67 @@ packages: - syslog_aggregator properties: - admin_username: + syslog_aggregator.address: + description: "IP address for syslog aggregator" + syslog_aggregator.port: + description: "TCP port of syslog aggregator" + syslog_aggregator.all: + description: "Define whether forwarders should also send non-mysql syslog activity to the aggregator." + default: false + syslog_aggregator.transport: + description: "Transport to be used when forwarding logs (tcp|udp|relp)." + default: "tcp" + + cf_mysql.mysql.admin_username: description: 'Username for the MySQL server admin user' default: 'root' - admin_password: + cf_mysql.mysql.admin_password: description: 'Password for the MySQL server admin user' - port: + cf_mysql.mysql.port: description: 'Port the mysql server should bind to' default: 3306 - healthcheck_port: + cf_mysql.mysql.healthcheck_port: description: 'Port used by healthcheck process to listen on' default: 9200 - max_connections: + cf_mysql.mysql.max_connections: description: 'Maximum total number of database connections for the node' default: 1500 - innodb_buffer_pool_size: - description: 'The size in bytes of the memory buffer InnoDB uses to cache data and indexes of its tables' - cluster_ips: + cf_mysql.mysql.innodb_buffer_pool_size: + description: 'Optional, the size in bytes of the memory buffer InnoDB uses to cache data and indexes of its tables' + cf_mysql.mysql.cluster_ips: description: 'List of nodes. Must have the same number of ips as there are nodes in the cluster' - max_heap_table_size: + cf_mysql.mysql.max_heap_table_size: description: 'The maximum size (in rows) to which user-created MEMORY tables are permitted to grow' default: 16777216 - tmp_table_size: + cf_mysql.mysql.tmp_table_size: description: 'The maximum size (in bytes) of internal in-memory temporary tables' default: 33554432 - wsrep_max_ws_rows: + cf_mysql.mysql.wsrep_max_ws_rows: description: 'Maximum permitted number of rows per writeset.' default: 131072 - wsrep_max_ws_size: + cf_mysql.mysql.wsrep_max_ws_size: description: 'Maximum permitted size in bytes per writeset.' default: 1073741824 - skip_name_resolve: + cf_mysql.mysql.skip_name_resolve: description: 'Do not restrict connections to database based on hostname' default: false - #these two properties are also used by the Broker - gcache_size: + cf_mysql.mysql.gcache_size: description: 'Cache size used by galera (maximum amount of data possible in an IST), in MB' default: 512 - ib_log_file_size: + cf_mysql.mysql.ib_log_file_size: description: 'Size of the ib_log_file used by innodb, in MB' default: 1024 - seeded_databases: + cf_mysql.mysql.seeded_databases: description: 'Set of databases to seed' default: {} + cf_mysql.mysql.database_startup_timeout: + description: 'How long the startup script waits for the database to come online (in seconds)' + cf_mysql.mysql.bootstrap_endpoint.username: + description: 'Username used by the bootstrap endpoints for Basic Auth' + cf_mysql.mysql.bootstrap_endpoint.password: + description: 'Password used by the bootstrap endpoints for Basic Auth' network_name: description: "The name of the network (needed for the syslog aggregator)" - syslog_aggregator.address: - description: "IP address for syslog aggregator" - syslog_aggregator.port: - description: "TCP port of syslog aggregator" - syslog_aggregator.all: - description: "Define whether forwarders should also send non-mysql syslog activity to the aggregator." - default: false - syslog_aggregator.transport: - description: "Transport to be used when forwarding logs (tcp|udp|relp)." - default: "tcp" - database_startup_timeout: - description: 'How long the startup script waits for the database to come online (in seconds)' - - bootstrap_endpoint.username: - description: 'Username used by the bootstrap endpoints for Basic Auth' - bootstrap_endpoint.password: - description: 'Password used by the bootstrap endpoints for Basic Auth' diff --git a/jobs/mysql/templates/galera_healthcheck_config.yaml.erb b/jobs/mysql/templates/galera_healthcheck_config.yaml.erb index 6f1cffd5..654e7b31 100644 --- a/jobs/mysql/templates/galera_healthcheck_config.yaml.erb +++ b/jobs/mysql/templates/galera_healthcheck_config.yaml.erb @@ -1,10 +1,10 @@ --- -Port: <%= p('healthcheck_port') %> +Port: <%= p('cf_mysql.mysql.healthcheck_port') %> PidFile: '/var/vcap/sys/run/galera-healthcheck/galera-healthcheck.pid' MysqldPath: '/var/vcap/packages/mariadb/bin/mysqld' DB: - User: <%= p('admin_username') %> - Password: <%= p('admin_password') %> + User: <%= p('cf_mysql.mysql.admin_username') %> + Password: <%= p('cf_mysql.mysql.admin_password') %> # This is the config that bosh sets up by default for monit. Monit: User: 'vcap' @@ -14,5 +14,5 @@ Monit: MysqlStateFilePath: '/var/vcap/store/mysql/state.txt' ServiceName: 'mariadb_ctrl' BootstrapEndpoint: - Username: <%= p('bootstrap_endpoint.username') %> - Password: <%= p('bootstrap_endpoint.password') %> + Username: <%= p('cf_mysql.mysql.bootstrap_endpoint.username') %> + Password: <%= p('cf_mysql.mysql.bootstrap_endpoint.password') %> diff --git a/jobs/mysql/templates/mariadb_ctl_config.yml.erb b/jobs/mysql/templates/mariadb_ctl_config.yml.erb index c33a21eb..3be2767c 100644 --- a/jobs/mysql/templates/mariadb_ctl_config.yml.erb +++ b/jobs/mysql/templates/mariadb_ctl_config.yml.erb @@ -4,10 +4,10 @@ PidFile: /var/vcap/sys/run/mariadb_ctl/mariadb_ctl.pid Db: DaemonPath: /var/vcap/packages/mariadb_ctrl/bin/mysql_daemon.sh UpgradePath: /var/vcap/packages/mariadb/bin/mysql_upgrade - User: <%= p('admin_username')%> - Password: <%= p('admin_password')%> + User: <%= p('cf_mysql.mysql.admin_username')%> + Password: <%= p('cf_mysql.mysql.admin_password')%> PreseededDatabases: - <% p("seeded_databases").each do |seed| %> + <% p("cf_mysql.mysql.seeded_databases").each do |seed| %> - DBName: <%= seed["name"] %> User: <%= seed["username"]%> Password: <%= seed["password"] %> @@ -18,8 +18,8 @@ Upgrader: Manager: StateFileLocation: /var/vcap/store/mysql/state.txt ClusterIps: - <% p('cluster_ips').each do |ip| %> + <% p('cf_mysql.mysql.cluster_ips').each do |ip| %> - <%= ip %> <% end %> MyIP: <%= spec.networks.send(p('network_name')).ip %> - DatabaseStartupTimeout: <%= (p('database_startup_timeout') * 0.8).round %> + DatabaseStartupTimeout: <%= (p('cf_mysql.mysql.database_startup_timeout') * 0.8).round %> diff --git a/jobs/mysql/templates/mariadb_init.erb b/jobs/mysql/templates/mariadb_init.erb index 08a18996..a5a4759e 100644 --- a/jobs/mysql/templates/mariadb_init.erb +++ b/jobs/mysql/templates/mariadb_init.erb @@ -1,5 +1,5 @@ -GRANT ALL PRIVILEGES ON *.* TO '<%= properties.admin_username %>'@'%' WITH GRANT OPTION; -UPDATE mysql.user SET Password=PASSWORD('<%= properties.admin_password %>') WHERE User='<%= properties.admin_username %>'; +GRANT ALL PRIVILEGES ON *.* TO '<%= p('cf_mysql.mysql.admin_username') %>'@'%' WITH GRANT OPTION; +UPDATE mysql.user SET Password=PASSWORD('<%= p('cf_mysql.mysql.admin_password') %>') WHERE User='<%= p('cf_mysql.mysql.admin_username') %>'; DELETE FROM mysql.user WHERE User=''; FLUSH PRIVILEGES; DROP DATABASE IF EXISTS test; diff --git a/jobs/mysql/templates/my.cnf.erb b/jobs/mysql/templates/my.cnf.erb index d30e37e2..d38dc7ae 100644 --- a/jobs/mysql/templates/my.cnf.erb +++ b/jobs/mysql/templates/my.cnf.erb @@ -13,7 +13,7 @@ base_folder = '/var/vcap/sys/run/mysql' %> [client] -port = <%= p('port') %> +port = <%= p('cf_mysql.mysql.port') %> socket = <%= "#{base_folder}/mysqld.sock" %> [mysqld_safe] @@ -23,15 +23,15 @@ nice = 0 [mysqld] # GALERA options: wsrep_provider=/var/vcap/packages/mariadb/lib/plugin/libgalera_smm.so -wsrep_provider_options="gcache.size=<%= p('gcache_size') %>M;pc.recovery=TRUE" -wsrep_cluster_address="gcomm://<%= p('cluster_ips').join(",") %>" +wsrep_provider_options="gcache.size=<%= p('cf_mysql.mysql.gcache_size') %>M;pc.recovery=TRUE" +wsrep_cluster_address="gcomm://<%= p('cf_mysql.mysql.cluster_ips').join(",") %>" wsrep_node_address='<%= spec.networks.send(p('network_name')).ip %>' wsrep_node_name='mysql/<%= index %>' wsrep_cluster_name='cf-mariadb-galera-cluster' wsrep_sst_method=xtrabackup-v2 -wsrep_sst_auth=<%= p('admin_username')%>:<%= p('admin_password')%> -wsrep_max_ws_rows=<%= p('wsrep_max_ws_rows') %> -wsrep_max_ws_size=<%= p('wsrep_max_ws_size') %> +wsrep_sst_auth=<%= p('cf_mysql.mysql.admin_username')%>:<%= p('cf_mysql.mysql.admin_password')%> +wsrep_max_ws_rows=<%= p('cf_mysql.mysql.wsrep_max_ws_rows') %> +wsrep_max_ws_size=<%= p('cf_mysql.mysql.wsrep_max_ws_size') %> #wsrep_on='OFF' #wsrep_desync='ON' #wsrep_OSU_method='RSU' @@ -41,7 +41,7 @@ collation_server = utf8_unicode_ci character_set_server = utf8 user = vcap socket = <%= "#{base_folder}/mysqld.sock" %> -port = <%= p('port') %> +port = <%= p('cf_mysql.mysql.port') %> basedir = /var/vcap/packages/mariadb datadir = /var/vcap/store/mysql tmpdir = /var/vcap/data/mysql/tmp @@ -51,7 +51,7 @@ pid-file = /var/vcap/sys/run/mysql/mysql.pid init-file = /var/vcap/jobs/mysql/config/mariadb_init skip-external-locking = TRUE max_allowed_packet = 256M -<% if p('skip_name_resolve') %> +<% if p('cf_mysql.mysql.skip_name_resolve') %> skip_name_resolve = 1 <% else %> skip_name_resolve = 0 @@ -59,10 +59,10 @@ skip_name_resolve = 0 innodb_file_per_table = ON innodb_file_format = Barracuda -innodb_log_file_size = "<%= p('ib_log_file_size')%>MB" +innodb_log_file_size = "<%= p('cf_mysql.mysql.ib_log_file_size')%>MB" -max_heap_table_size = <%= p('max_heap_table_size') %> -tmp_table_size = <%= p('tmp_table_size') %> +max_heap_table_size = <%= p('cf_mysql.mysql.max_heap_table_size') %> +tmp_table_size = <%= p('cf_mysql.mysql.tmp_table_size') %> # These are mandatory MySQL settings for MariaDB Galera to work binlog_format = ROW @@ -75,11 +75,11 @@ query_cache_size = 0 innodb_stats_on_metadata = ON innodb_stats_persistent = OFF -<% if_p('innodb_buffer_pool_size') do |innodb_buffer_pool_size| %> +<% if_p('cf_mysql.mysql.innodb_buffer_pool_size') do |innodb_buffer_pool_size| %> innodb_buffer_pool_size = <%= innodb_buffer_pool_size %> <% end %> -<% if_p('max_connections') do |max_connections| %> +<% if_p('cf_mysql.mysql.max_connections') do |max_connections| %> max_connections = <%= max_connections %> <% end %> diff --git a/jobs/proxy/monit b/jobs/proxy/monit index 92b74b86..6c748bc9 100644 --- a/jobs/proxy/monit +++ b/jobs/proxy/monit @@ -4,10 +4,10 @@ check process switchboard stop program "/var/vcap/jobs/proxy/bin/switchboard_ctl stop" with timeout 10 seconds group vcap -<% if p("standalone") == false %> +<% if p("cf_mysql.standalone") == false %> check process route-registrar with pidfile /var/vcap/sys/run/route-registrar/route-registrar.pid start program "/var/vcap/jobs/proxy/bin/route-registrar_ctl start" stop program "/var/vcap/jobs/proxy/bin/route-registrar_ctl stop" group vcap -<% end %> \ No newline at end of file +<% end %> diff --git a/jobs/proxy/spec b/jobs/proxy/spec index 8784437f..1910e8d4 100644 --- a/jobs/proxy/spec +++ b/jobs/proxy/spec @@ -16,29 +16,10 @@ packages: - route-registrar properties: - proxy.port: - description: "Port for the proxy to listen on" - default: 3306 - proxy.healthcheck_timeout_millis: - description: "Timeout (milliseconds) before assuming a backend is unhealthy" - default: 5000 - proxy.api_force_https: - description: "Redirect all HTTP requests to the API to HTTPS" - default: true - proxy.api_port: - description: "Port for the proxy API to listen on" - default: 80 - proxy.api_username: - description: "Username for Basic Auth used to secure API" - proxy.api_password: - description: "Password for Basic Auth used to secure API" - proxy.health_port: - description: "Port for checking the health of the proxy process" - default: 1936 - proxy.proxy_ips: - description: "List of IP addresses for all proxy jobs" - external_host: - description: 'Domain of the route registered for the UI via NATS (with the router in cf-release)' + + network_name: + description: "The name of the network (needed for the syslog aggregator)" + nats.user: description: 'Username to register a route via NATS' nats.password: @@ -47,8 +28,7 @@ properties: description: 'IP port of Cloud Foundry NATS server' nats.machines: description: 'IP of each NATS cluster member.' - network_name: - description: "The name of the network (needed for the syslog aggregator)" + syslog_aggregator.address: description: "IP address for syslog aggregator" syslog_aggregator.port: @@ -59,11 +39,37 @@ properties: syslog_aggregator.transport: description: "Transport to be used when forwarding logs (tcp|udp|relp)." default: "tcp" - cluster_ips: + + cf_mysql.external_host: + description: 'Domain of the route registered for the UI via NATS (with the router in cf-release)' + + cf_mysql.standalone: + description: "Standalone Mode: Are you deploying MySQL without a CloudFoundry deployment?" + default: false + + cf_mysql.mysql.cluster_ips: description: "List of nodes. Must have the same number of ips as there are nodes in the cluster" - arbitrator_ip: + cf_mysql.proxy.port: + description: "Port for the proxy to listen on" + default: 3306 + cf_mysql.proxy.healthcheck_timeout_millis: + description: "Timeout (milliseconds) before assuming a backend is unhealthy" + default: 5000 + cf_mysql.proxy.api_port: + description: "Port for the proxy API to listen on" + default: 80 + cf_mysql.proxy.api_force_https: + description: "Redirect all HTTP requests to the API to HTTPS" + default: true + cf_mysql.proxy.api_username: + description: "Username for Basic Auth used to secure API" + cf_mysql.proxy.api_password: + description: "Password for Basic Auth used to secure API" + cf_mysql.proxy.health_port: + description: "Port for checking the health of the proxy process" + default: 1936 + cf_mysql.proxy.proxy_ips: + description: "List of IP addresses for all proxy jobs" + cf_mysql.proxy.arbitrator_ip: description: "List of IP addresses for the arbitrator nodes of the MySQL cluster" default: "no-arbitrator-ip" - standalone: - description: "Standalone Mode: Are you deploying MySQL without a CloudFoundry deployment?" - default: false diff --git a/jobs/proxy/templates/route-registrar.yml.erb b/jobs/proxy/templates/route-registrar.yml.erb index e97f0fcd..4a30bf36 100644 --- a/jobs/proxy/templates/route-registrar.yml.erb +++ b/jobs/proxy/templates/route-registrar.yml.erb @@ -5,7 +5,7 @@ message_bus_servers: password: <%= p('nats.password') %> <% end %> <% my_ip = spec.networks.send(p('network_name')).ip %> -<% proxy_index = p('proxy.proxy_ips').index(my_ip) %> -external_host: proxy-<%= proxy_index %>-<%= p('external_host') %> +<% proxy_index = p('cf_mysql.proxy.proxy_ips').index(my_ip) %> +external_host: proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> external_ip: <%= my_ip %> -port: <%= p('proxy.api_port') %> +port: <%= p('cf_mysql.proxy.api_port') %> diff --git a/jobs/proxy/templates/switchboard.yml.erb b/jobs/proxy/templates/switchboard.yml.erb index 0256ef90..e3a77f0f 100644 --- a/jobs/proxy/templates/switchboard.yml.erb +++ b/jobs/proxy/templates/switchboard.yml.erb @@ -1,14 +1,14 @@ "API": - "Port": <%= p('proxy.api_port') %> - "ForceHttps": <%= p('proxy.api_force_https') %> - "Username": <%= p('proxy.api_username') %> - "Password": <%= p('proxy.api_password') %> + "Port": <%= p('cf_mysql.proxy.api_port') %> + "ForceHttps": <%= p('cf_mysql.proxy.api_force_https') %> + "Username": <%= p('cf_mysql.proxy.api_username') %> + "Password": <%= p('cf_mysql.proxy.api_password') %> "Proxy": - "Port": <%= p('proxy.port') %> - "HealthcheckTimeoutMillis": <%= p("proxy.healthcheck_timeout_millis") %> + "Port": <%= p('cf_mysql.proxy.port') %> + "HealthcheckTimeoutMillis": <%= p("cf_mysql.proxy.healthcheck_timeout_millis") %> "Backends": - <% cluster_ips = p('cluster_ips') %> - <% cluster_ips.delete(p('arbitrator_ip')) %> + <% cluster_ips = p('cf_mysql.mysql.cluster_ips') %> + <% cluster_ips.delete(p('cf_mysql.proxy.arbitrator_ip')) %> <% cluster_ips.each_with_index do |ip, n| %> - "Host": <%= ip %> "Port": 3306 @@ -16,7 +16,7 @@ "StatusEndpoint": "galera_status" "Name": "backend-<%= n %>" <% end %> -"HealthPort": <%= p('proxy.health_port') %> +"HealthPort": <%= p('cf_mysql.proxy.health_port') %> "ProfilerPort": 6060 "StaticDir": "/var/vcap/packages/switchboard/static" "PidFile": "/var/vcap/sys/run/proxy/switchboard.pid" diff --git a/templates/bosh_lite_stubs/iaas_settings.yml b/manifest-generation/bosh-lite-stubs/iaas-settings.yml similarity index 100% rename from templates/bosh_lite_stubs/iaas_settings.yml rename to manifest-generation/bosh-lite-stubs/iaas-settings.yml diff --git a/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml new file mode 100644 index 00000000..a3f492dd --- /dev/null +++ b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml @@ -0,0 +1,28 @@ +property_overrides: + standalone: true + mysql: + admin_password: password + bootstrap_endpoint: + username: username + password: password + proxy: + api_username: username + api_password: password + broker: + auth_username: admin + auth_password: password + cookie_secret: 94046872-2602-4ca6-8d07-8b0da9762477 + dashboard_secret: p-mysql-secret + plans: + - name: 100mb + id: ab08f1bc-e6fc-4b56-a767-ee0fea6e3f20 + description: Shared MySQL Server + max_storage_mb: 10 + max_user_connections: 20 + - name: 1gb + id: 11d0aa36-dcec-4021-85f5-ea4d9a5c8342 + description: Shared MySQL Server + max_storage_mb: 20 + max_user_connections: 40 + acceptance_tests: + standalone_tests_only: true diff --git a/templates/bosh_lite_stubs/property_overrides.yml b/manifest-generation/bosh-lite-stubs/property-overrides.yml similarity index 100% rename from templates/bosh_lite_stubs/property_overrides.yml rename to manifest-generation/bosh-lite-stubs/property-overrides.yml diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml new file mode 100644 index 00000000..b51819ea --- /dev/null +++ b/manifest-generation/cf-mysql-template.yml @@ -0,0 +1,304 @@ +--- +name: (( config_from_cf.cf_deployment_name "-mysql" )) +director_uuid: (( config_from_cf.cf_director_uuid )) + +releases: +- name: cf-mysql + version: (( release_versions.cf-mysql || "latest" )) + +compilation: + workers: 4 + network: mysql1 + reuse_compilation_vms: true + cloud_properties: (( iaas_settings.compilation_cloud_properties )) + +update: + canaries: 1 + canary_watch_time: (( "30000-" canary_end_in_sec )) + update_watch_time: (( "30000-" canary_end_in_sec )) + max_in_flight: 1 + serial: true + +networks: + - name: mysql1 + type: manual + subnets: (( iaas_settings.subnet_configs.mysql1.subnets )) + - name: mysql2 + type: manual + subnets: (( iaas_settings.subnet_configs.mysql2.subnets )) + - name: mysql3 + type: manual + subnets: (( iaas_settings.subnet_configs.mysql3.subnets )) + +disk_pools: +- name: mysql-persistent-disk + disk_size: (( iaas_settings.disk_pools.mysql-persistent-disk.disk_size || 100000 )) + cloud_properties: (( iaas_settings.disk_pools.mysql-persistent-disk.cloud_properties || empty_hash )) + +resource_pools: +- name: mysql_z1 + network: mysql1 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties )) +- name: mysql_z2 + network: mysql2 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.mysql_z2.cloud_properties )) +- name: mysql_z3 + network: mysql3 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.mysql_z3.cloud_properties )) +- name: proxy_z1 + network: mysql1 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.proxy_z1.cloud_properties )) +- name: proxy_z2 + network: mysql2 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.proxy_z2.cloud_properties )) +- name: cf-mysql-broker_z1 + network: mysql1 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.cf-mysql-broker_z1.cloud_properties )) +- name: cf-mysql-broker_z2 + network: mysql2 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.cf-mysql-broker_z2.cloud_properties )) +- name: errands_z1 + network: mysql1 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.errands_z1.cloud_properties )) + +jobs: +- name: mysql_z1 + release: cf-mysql + template: mysql + instances: (( instance_count_overrides.mysql_z1.instances || 1 )) + resource_pool: mysql_z1 + persistent_disk_pool: mysql-persistent-disk + networks: + - name: mysql1 + static_ips: (( static_ips(0) )) + properties: + network_name: mysql1 + +- name: mysql_z2 + release: cf-mysql + template: mysql + instances: (( instance_count_overrides.mysql_z2.instances || 1 )) + resource_pool: mysql_z2 + persistent_disk_pool: mysql-persistent-disk + networks: + - name: mysql2 + static_ips: (( static_ips(0) )) + properties: + network_name: mysql2 + +- name: mysql_z3 + release: cf-mysql + template: mysql + # defaults to using an arbitrator in place of mysql node in z3 + instances: (( instance_count_overrides.mysql_z3.instances || 0 )) + resource_pool: mysql_z3 + persistent_disk_pool: mysql-persistent-disk + networks: + - name: mysql3 + static_ips: (( static_ips(0) )) + properties: + network_name: mysql3 + +- name: arbitrator_z3 + release: cf-mysql + template: arbitrator + instances: (( instance_count_overrides.arbitrator_z3.instances || 1 )) + resource_pool: mysql_z3 + networks: + - name: mysql3 + static_ips: (( static_ips(1) )) + properties: + network_name: mysql3 + +- name: proxy_z1 + release: cf-mysql + template: proxy + instances: (( instance_count_overrides.proxy_z1.instances || 1 )) + resource_pool: proxy_z1 + networks: + - name: mysql1 + static_ips: (( static_ips(1) )) + properties: + network_name: mysql1 + +- name: proxy_z2 + release: cf-mysql + template: proxy + instances: (( instance_count_overrides.proxy_z2.instances || 1 )) + resource_pool: proxy_z2 + networks: + - name: mysql2 + static_ips: (( static_ips(1) )) + properties: + network_name: mysql2 + +- name: cf-mysql-broker_z1 + release: cf-mysql + template: cf-mysql-broker + instances: (( instance_count_overrides.cf-mysql-broker_z1.instances || 1 )) + resource_pool: cf-mysql-broker_z1 + networks: + - name: mysql1 + properties: + network_name: mysql1 + +- name: cf-mysql-broker_z2 + release: cf-mysql + template: cf-mysql-broker + instances: (( instance_count_overrides.cf-mysql-broker_z2.instances || 1 )) + resource_pool: cf-mysql-broker_z2 + networks: + - name: mysql2 + properties: + network_name: mysql2 + +- name: broker-registrar + template: broker-registrar + release: cf-mysql + instances: 1 + resource_pool: errands_z1 + lifecycle: errand + networks: + - name: mysql1 + properties: {} + +- name: broker-deregistrar + template: broker-deregistrar + release: cf-mysql + instances: 1 + resource_pool: errands_z1 + lifecycle: errand + networks: + - name: mysql1 + properties: {} + +- name: acceptance-tests + template: acceptance-tests + release: cf-mysql + instances: 1 + resource_pool: errands_z1 + lifecycle: errand + networks: + - name: mysql1 + properties: {} + +- name: bootstrap + template: bootstrap + release: cf-mysql + instances: 1 + resource_pool: errands_z1 + lifecycle: errand + networks: + - name: mysql1 + properties: {} + +properties: + nats: + machines: (( config_from_cf.nats.machines )) + user: (( config_from_cf.nats.user )) + password: (( config_from_cf.nats.password )) + port: (( config_from_cf.nats.port )) + app_domains: (( config_from_cf.app_domains )) + domain: (( config_from_cf.domain )) + cf: + api_url: (( config_from_cf.api_url )) + app_domains: (( config_from_cf.app_domains )) + admin_username: (( config_from_cf.admin_username )) + admin_password: (( config_from_cf.admin_password )) + skip_ssl_validation: (( config_from_cf.skip_ssl_validation || nil )) + smoke_tests: + use_existing_org: (( config_from_cf.smoke_tests.use_existing_org || nil )) + org: (( config_from_cf.smoke_tests.org || nil )) + syslog_aggregator: (( property_overrides.syslog_aggregator || nil )) + cf_mysql: + standalone: (( property_overrides.standalone || false )) + external_host: (( "p-mysql." .properties.domain )) + host: (( property_overrides.host || jobs.proxy_z1.networks.mysql1.static_ips.[0] )) + mysql: + admin_username: (( property_overrides.mysql.admin_username || nil )) + admin_password: (( property_overrides.mysql.admin_password )) + port: (( property_overrides.mysql.port || nil )) + galera_port: (( property_overrides.mysql.galera_port || nil )) + healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) + bootstrap_endpoint: (( property_overrides.mysql.bootstrap_endpoint )) + cluster_ips: (( property_overrides.mysql.cluster_ips || default_mysql_cluster_ips )) + seeded_databases: (( property_overrides.mysql.seeded_databases || nil )) + database_startup_timeout: (( property_overrides.mysql.database_startup_timeout || 600 )) + max_heap_table_size: (( property_overrides.mysql.max_heap_table_size || nil )) + tmp_table_size: (( property_overrides.mysql.tmp_table_size || nil )) + wsrep_max_ws_rows: (( property_overrides.mysql.wsrep_max_ws_rows || nil )) + wsrep_max_ws_size: (( property_overrides.mysql.wsrep_max_ws_size || nil )) + skip_name_resolve: (( property_overrides.mysql.skip_name_resolve || nil )) + gcache_size: (( property_overrides.mysql.gcache_size || nil )) + ib_log_file_size: (( property_overrides.mysql.ib_log_file_size || nil )) + innodb_buffer_pool_size: (( property_overrides.mysql.innodb_buffer_pool_size || nil )) + max_connections: (( property_overrides.mysql.max_connections || nil )) + persistent_disk: (( disk_pools.mysql-persistent-disk.disk_size )) + proxy: + api_username: (( property_overrides.proxy.api_username )) + api_password: (( property_overrides.proxy.api_password )) + health_port: (( property_overrides.proxy.health_port || nil )) + api_force_https: (( property_overrides.proxy.api_force_https || nil )) + proxy_ips: (( property_overrides.proxy.proxy_ips || default_proxy_ips )) + arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) + port: (( property_overrides.proxy.port || nil )) + healthcheck_timeout_millis: (( property_overrides.proxy.healthcheck_timeout_millis || nil )) + api_port: (( property_overrides.proxy.api_port || nil )) + broker: + max_user_connections_default: (( property_overrides.broker.max_user_connections_default || nil )) + ssl_enabled: (( property_overrides.broker.ssl_enabled || nil )) + auth_username: (( property_overrides.broker.auth_username )) + auth_password: (( property_overrides.broker.auth_password )) + cookie_secret: (( property_overrides.broker.cookie_secret )) + external_host: (( property_overrides.broker.external_domain || .properties.cf_mysql.external_host )) + protocol: (( property_overrides.broker.protocol || nil )) + port: (( property_overrides.broker.port || nil )) + services: + - name: p-mysql + plan_updateable: true + id: (( property_overrides.broker.service_id || "44b26033-1f54-4087-b7bc-da9652c2a539" )) + description: MySQL databases on demand + tags: + - mysql + metadata: + displayName: "MySQL for Pivotal Cloud Foundry" + imageUrl: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAJo5JREFUeNrsXQtsVNeZvvPwzBiD3zyNwxgWhEKqmEibR9skdhQtaao2Rqtks4+qthKp1e5KxKhVCg3BbrKOst3lIW2rrUQVR5W2adgKp92ikG1ik2RbkpWCs4WWQomHVyAY8NhgPDOex97v+F4zHs97zrlz7z3/Jw1j/Ji5c+/9vv9x/v8/DoVgK9Rt7/arT37tv7XqozXT7zorK1clotFgYnp6PMtLDquPoPZ1YKxvd4DOsn3goFNgOYK3asRuUx81SQRvM/hQhpIEYlz7f1AViGG6SiQABH6WHMS+U/u61SKHDxGAp/CxJgzDqjAE6aqSABAyE75NI/iD2rPfZh8xoAnDYdfCquEr218coqtOAiC7K9+hEb5N0tMw5PBUHE5EpgcodCABsDvha5MI36HF8YRbQIgw4PR5D8dD4QEKGUgA7BLHg+yPSWzli7sx3e4hxeV8w11bO3B5y7YAnRESAKtZ+i2KdZJ2pobT6x2Oh8N74SGQZ0ACYFbig/Rf18hPEHGzulzqP44BR0XFq1d39A3QGSEBMIOLD0vfSTG94WIQdHi9/a5FC/dSiEACYCgaXtjeEZ+aImtvnnzBQCIa3auGB0N0NkgARFp8WPqdiv3W6O1xM3s9gUQ40qsKQT+dDRIAXqSHa/+M5uqTm2+R8MBZ6dvrXe3fc+HJpylpSAJAxCchICEgASDikxCQEJAAZCF/DxHf/kJwZfuLPXQ2SACSid+pUHJPnpve6wk4fb7eK8/29pMAyE38No34bUQLCW9+t3vI3VDfe3nLtiESAPni/N3KTAEPQe6wQHF4PP2+tau7ZcwPOCQkf6dGforzCXPzAwurumULCxwSER/x/Ssyufuwbs5K35zvOauqsv5NfHJy7v+nQkoiFpOGEE6fd8hVU9MlS3mxQxLy9yg2zO67Fqpkdjox3FMlu3OG7M75pOcFXRxmRCGuPk8pielp9n+bIeiuq907+u2dPSQA1rf6BxSLt+UyS161gBEd5Hb6fGrcWmGqY2SioIkBhMIOnoOWJLS1N+CwMflRzLPTilYf5Haprroak7Jns5G9EFGIh9THjUklpopCIjJtydyAq66md3Trjj0kANYgfq0W63dY6CZTXNWLLE/4XIAAxCYmlBgEYeK6xXIDvgHfujVddlspcNiM/G2ay296q8+sfHW14q5ZlDMxZ1dABGLjE+zZCuGCw+NBbmCzneoGHDYif4/m8pvb0tfVKupNJCxRR2JgiLfWa5cEocMGxDe9yw/33l1fx54JuREdC86KgWmv6aKFA96WVZYPCRwWJ3+rRn7TZfnh4rvr6hR3Y/3MLDtCUTkDiEF0bMyUCUSH2zXsaVrRdekbW4dJAIwnf4dGflPF+1ib1918Al+vIHrlqulqDpAXUK951+i3nh8gATCO/Fji222mYwLh3fW10ib0jAJqDKY/G2UrCabKC9TVdFtxqdBhQfLD6neaifgVS5fYdunOzOHB9GeXmWdgmnuhvq5f9QS6SADEEN9UyT4ivnmEIHL+gmk8AqvVCzgsRP5BxQTJPsT4FUsXk6tPoUFmEahaMOxb09JuBRFwEPnzPFGqpfc2NxHxTQ4sHUY+vVj2VQOIQHzyZrvZtzJzmJz8ID0q+/xlO0Eul+JubGBWn2AdwBvAqkE5i4rUeyfgWblis5mXCR0mJz8sf9nW01C441mxnOJ8K+cHVG+gnAVFWCb0LF/ablYRcBD507v7ID5V7tkoLDh3oWzegJlFwEHknwvd3afqPZt5Ayr59bCARMCkAlBO8oPwnuYmsvrkDUglAg4TkR+kP6qUIeHHYn2V/GT15fEGIAJlyg2A/KZZHXCaiPyD5SC/Z8Uyxeu/jcgvEXCtcc1x7cuAVmfVgsGm1/aZolnEYSLyG7rOz9b1V91GffmSA81F4TNnDa8bMEuxkBk8AMPbeVHN51u7hshPYPcA7gWjcz/xyZutoZOnXyn35y+r36s19jxp5Hsiy++9baXicDrp7ifMeIPqveCurWEOceq+CEJzEdHo+kWPPeq/+dbgG9IJgNbS+x1D4/3mJqViSSPd8YSMnqHD44F1VtmZMCoEaV20+cvjNw8NHpEmB6AN8zhg2IdE0me1n1x+Qv55gU8Chi0VsnkCNdWbyzFUxFEG8hu61g/Se1Y2EfkJhbnnkWmWHDRqAlG5agScBpNf7+k3jPxk+QnFEbLC0HsnEYnURi58+orRy4NGZ8IMy/jr5Kf1fUKpoSPbg9EIEYjGWsMjZwxdGTCMHdrc/m8a8V6Y1uNBpp/ITyhVBLBCoN5PCAmwzZkBnsD6RV/9kuPmfw8N2SYHoO3YM2gY+Zub6M4lcAfKh42YQajNoGg3YgcihwHkR0wzYkTcT+Qn2EYEPJ5g5fq1LaIrBY3IARiS9NMbeggEkcA9ZsSeD0gKGlEpKDRI1op9nhH9IVjCDw09VN1HMACummpWLCS6f4BVCgouEnIIJL9fmWnvrRVOfsr2F4zmqoXKyqrs9e8TkYhyPHiVTlY6csZirFhIdJ2Ael8H3Y0NGy9v2RYQEjYLPPYDRH7jUV3hUTbUNcwSfENtPftetUf9fm1DSa99fvKGcm5ypof+t6OXVIEIqwJxTf3+dfX7N6Q6z/oSoWgRUIWmNnr1GkKBdst4AEZs1U3lvTO4b8nymcfiZSrpF6mkX1iW45iYVr2FsatMGNjz5Yvse7b3BNQwIHTqtPCyYXddrZAtyR0CyG+I6y9rOy8s+6amVYz0eDYzED4cunBWOXT+jK1DCYN6B4IVS5dwDwVEhADCs/7IxMpG/ida1iqP+9cy4lsFCDnw2LphIwsf9p08prw+csp2ngHuxYoVy9gSoUDUxsbHuYcCXD0A1fp3agIgzhVqbCjXKKeyEX/rhrvK5tqLCBX2nTyu7PvjMdsJAaYOY8NSkXDVVHddeba333QCYETBD2qyEffL4urvuvsBS1n8QgCPoPvDd1muwE4IB84KHTaKVYHKDeu5FQjxXDjfLZL8bLOOVc1SkB9u86FNm21LfgAezf72R5mHYyew6dICd5LCqkDo1Ce7uYUvnKx/m/rUKfLEYoCnLMt9P/7iw2zpTgbAy2m2SXijWWh2r4pEIhLpXLL3pTYzeQBCl/wQ88uS9INFXGkjQuQD5DjsBDaERmCeCqsN0avXdppCALTEX5uoD4u4H4k/WZCrOs+u4YDdgHtW5KThRDTa1vjyzs6yC4BI68+265Ik7ifYD6J3m4qHQjvLKgBaxZ/fqieQQBCdDxDZoZoIR/yNfc/1lEUAtGW/LcJcf9V9oo06CVYH7mGRIWx8KrSllDmCpXgAaPMVsuwnWjkJBCMhcrt5LAuGPwk8Y6gAiLb+5PoTKBQwxgso1gMQZv2R9ZfZ9UcnnXSfOXhNilBA1H1dihdQsACItP5MKVfK7fofunDGduWx2YB+gF3HPpLis3pWLBfm2RbrBRTjAQiz/kiWiCyjtAqeev/XUkziAfkff+egFHMDmIFT721RCcFivYBiBGCLqJODZAnhFjHQNWdXwMvZdOiAdCPHWEJQkJGDF1BwaFKg+9+pCNrO29u8UnF6vcR+DeF4TBm6dJ5N2EGtfLNNKgTRBdgzfETpOfqBNJZ/ntX1eJRYcFyAG5DwVd5/35nQe0fy3l+w0IEgQqr+ZE/85bKUj6sPiMBT6+5gvQJWbBTaP3JKeT1wSqr8Rsb7HQlB9Z6P3Zjk70l7PeBof96/X4D1F7alt2+NX3FWVRHb84Q+Duy+JctKHvQp0tKD7G9qSU1ZrX1Gd31yUgmdDojxMCorN1/d0ZfXVuOFeABfF2X9ify3AAs/Homw1YBsXoFuSeENQBAwCRiDQfFcDg8Bx4PlvN8HZwaC5jslGAK2aeUqaVYCZkmq3vOivIDE9DS4mpcA5OUBaIM+R4RYf0mHe2bC1jvuYjP0QKJdx48W7TJDFPQR4ez/i5fN+VkhQKIOewTMfH2NWXN9FHixI8ER0qAN+HFV8FiYM3hQumuNicJTJ04Kee2KpUta8hkgmq8HICTzjy2WiPyZCbxffYAc+9XYGcM0C7XIQDZPolyf6wn/WkZ82cGWBVUOiNhrMHb9BjjbzUsAOoUIgER9/qUQBo+ejfey8dqIqc1G6nzcfBD+kaZV0g07ycNSCxGARDjcyUUAtOQf98IfFvuT9c8bcOdBIt1ysjzA6KU5+QAzER4JSl28ZBlvVqwXICIXgMKghhe2d+RKBubjAQhJ/lHRDx/PQNmwcTZOR+YdMTr6CRCnixYGEB1bjuE4mhcsZNbdzoNMxXkBi8uWDHTnsP6w/B28DwyWnzL//MnIMuppdgtKTuLBayhKcEpIIhJy8KFqxhvmvsdgItEBDo/17Q4W6wF0iPjAFPtnBjbcFCEOZiYv1QjMcIL3zkLaVmXgcH9G8cnxGtyz/3rmk5Aedtw6Kxfs3POQtwConBDRI+D0erNy2JnF/ferT638P2gdsTyHNew5ekSaz4sSYSoPFseNeDjcumTvS/5iPAAx7j9Z/7y8ALQE290TQKETtgcjiOVGNBjsKEYAHuN9IGiCoH7//IC1/nt/+TNmIe2GmVbgAenKf/MJj4U0xcXiGbnsyOD+Q4rGeB8H5qKRB1A49LJZ1MxbeU0dooZ4n1z+LNZ6LChqm/G6dKsBbqPcf4xCIvIXh3PaTrrVRz3KI6oIYKkv3XKfGYElSHgxIH8xPQMyhgHTn17SM/jc4PR5064GZBKAB0W4/4T8rH0moiAngPyA3hegFwNhjd4sy3vHWTfgrQrFfPIY2T6zjABXeJcHx0PhBwsRAO4egKummq5sHni8ZR1rloHFz+Uqp5YBY70f1XjoANxQWz/bDSgibNDfF4VFekdgoa49iI/dgdnnlrAbMBtXBPQHgNNdOXMAavyPpb+jvN3/yg3r6crmAb0dWCdZMZ2A6ZDcGjwjFqpAeLx5Ex0otvU3FakdgbK2A2fD1PET3MMAFRvH+nYP5/IA+Ft/cv+LJgoeSACiCxCxdLFDNFN7A4xOxMHaI28B0pt1ipHdwwCHpwLczikA/ON/cv9LAtz6p9dtYA80/Ohjtsw+aksXsE1NtxHpTRAGJCLTD+aTA2jj7f6TByBGDAB4BL8fu8ae0QlYriU21oxUV6/cXjuTf6CGIVN6zW1ZBUCN/9t4v6OzagFdTdHEw8ANZe0cdx8twfAWzt28Mc/l19uFC7HkqbkDllNQv8a4chryIU4EYhPXub5mY99zbVe2vziUyQPgXvtP7r/x0AeFzlWKjXRirCYAGBTCWQBiNybB8VkBcAqP/6nvvyDIWCVX7IwC+3sAQoznHI47RXoAqG2m2v/CBcBqM/9KAcKUfX88RhfeOP60phUArf3XT9a//Oj+4F0pRACJSxm6Hk3GIb/W6zPPA/DzfifnQhKAYgBCgBiYC2BXcqApCBugyrY5qEk41JpOANoo/jcfSfSWYLsIATwbtALbWdy4CoBPyOTsWa4nrwLcyTV+cbko/ufkDSR3Aj61boPlimoQ52Nj0P0jJ6npp1ABqPQxLnEuC74znQD4eR84ga8Q6J2Aelmtmefu6/sWYDMTcvNL5xLnseH+dALAdQWAxn4XB73CL9ugTFhR/Fz/ndQqPKM3CAXZZ/YkKLwaUW8MotFg2bnEWQBa5wiA1gHIXbUIhQNVdugGfHrdHcqu4x/l1Qk4QzxY2VPzyJX8rG/eke530pE61QMBudn7aZWEhVYUpr4vPud92v6HBGO5BM6jM1D3ALiP6hGUvJAGK7VeeXQCFhs/z/bsm4Rg8EqsmscoJxwVQnJptckhQBv3g6YEIDchgKXEw4rjtXTSW2mMmQwegMb5IV0Aani+sovW/4WAxfobG9hOwee1CTy/Hb2ouuLXTJNoQ4LydrY56HK2QShZen4iwHnrsJpkD4BvDsDppCtmgGeQvFuw7urrHYA84vRcMbz+rHcGGp18lC4M4CsArckCwFmtKumKlQHZevB1MZjzfy2pl9GapyQNqfW3nB5AJffOQGE5AIeLPIBigbl7omLxVIEwS0x+noqDysEpxnkhTKUlwOLx5vkz0pXIYpWDUB5OObUuQM6v6qIrViRAfjTJyCICW/MYf04QA3AfHgB3ASAPoDQgo2/3Tjm94/H1EbL+eXFKTGWtn4J1E4sA20Dz+FHbeQOoZUCXo0yDT8wKJAG5VgGic4nAD9hBFxNznmhZy8qDrZqF15uZfnzyGHUElsAtzl2BtRAAvk1A5P4LIY/e/MMmAKti8EjTKtOLAY4b3YDYx4CsPR9u8W4KctNptV5ocPzoVTZQA1V3M5uDLldur6sve9Wd3hE4s2nJJWoDtkgIQLAo4EqfG5m7d2DyBqF6IQ/PAp7k6kIUEhW7MSiBBIAgykNQH5lcbngNK6sK23VmIhIha04CkGecQoNAzO81UBLOmjkA/oNBFFoGJBBkFhU6BQSC3AJAUxoIBDlRQx4AgSC5B0AVGgSCnBgnD4BAkNwDIBAIJAB8EJ+cpLNKIAiACG6RB0AgkAdAIBBkBPUCWAzYOxDbh6UiWzMO1fITsgnAMNc4he/sckISdt/9wJx9AOZgw8aiX5ftJZBlGrG+10A6YGAJwaAcAH9usb0BgzxfkfPEEoIyM9J7/0OPCuv3X1m1sOh2YaO2/27Wjk/mRiYB3ApSCCA5+UsFPBIMKBEJjEPDRqm6N4IJQwh5aMpQ6XBoY8FHeL6ob42f2oIlIL9OyHv/62fCXh/7ICLvkQkQAgxOtftAEiwBhk4HeL9si3Osb3eAqEbkLyV8EHGM+Pw//uLDWckPYCQakpyEwgHuC1kGpERgaQCh3tq02TI76/IeTorPDfHLZ+syWH8ZVjhEcUrPAQwpHPcHTMTixOISb36r7LKLEIBnLA6LDsufz+cH8WVZhRDAqaFkAeCsVlPEZAnID/Dc1y852ZcLGEr69Pu/lubeEFVirwvAMFcPYHqa2CwB+QFsWsIDWWsc0rn+quWXaUlQwBLgcLIAjFMOoHyA5UO220rkZ7P/Ry+VvG1ZMclOvDc2SZEJAjg1npoD2Mn7gGmXIL5ur2hgzz7dpTdiWQ2kR7xfSBIRgoMdhYn8fHMAQd6vzsIAEgDLkJ+RfvSiYevpyPDvvueBgr0e2Vx/gSE14zxbBhzr2z3M+9UpDLAW+QGjiIW1/Xwz/cnAaoNsrr8oLumcd6ckBbhtFDqTtVxMTE+DrXfcpWwtoXlHFHo23qPt73ctZwyeDfrWYeni/d6N9xaU7Et2/bs/eFfK+0XACsCswU8WgABXASAPIC0KzXYbCcTkeOQswCmDeIH8pSYcyQOYw3VlNgTQ8DHXuCUWIxGwEPnNjEMSby8ODglYAvw4nQAMcT/4EAkAkb80yOz6C+TQ0DwBqFi6JMD94G/QgFDEvUR+cv1NxqHheQJwecu2QHJswAMxySYEI7mX2r2G2nYif3FAxl/2nn8BHAqM9e2eXfZ3p1EGP7c8QGSaPRyeCqlcfFis10dOzcavKLAhESgMWI2QfdyYzh9R1j81BwAc5q5gExO2d/EPbeqYQ3Cs7yeXtnZ/+K7tB1Zwd/0/lNv1F8idwxkFwLWwintBUMzGeQC9gSddHXvq9596/9c0mbcA158EUxh3MnsAV7a/OMRfxa7b8uIgts/WxALPYNc9989Wu+ntq7JbNXL9y8sdNf4fyhYCACQCOYAy3v3tuVt3dQ9BB0ptH3/nIIlAFjxFIimSM/O4PU8AHJ4K/nmAcfvkAdC2W0gNP0Rgd9LvIwzoOXqEmJ4Gsoz3KiNnDucUgERkeoA8gPTAEl+uIZXpgARh8t9hhYBEYC5kGu9VRs7M47Yr9Ruh945cqrz/vmfUL/n18iYSirOyUnF6vZa+KH+aGFfalq9UlvgWFPy3+LvzN2812nx0dVRprlqkbKhroLtdxdcOv6WMhmiUnE7+6Bj3Dv2gGv935/QAMikFhQHaMIoP3is6RkX4kLo8SC4vuf4GcSUtp9MOBXX6vIfjoXAnhQHpXVUk8rD2Xyj08VebDh2Y7b3Ha1lh/n+yCKLdt7RzeG2OiJLrbwhX0ub2HOm+Wbe9u1Z9GuN9BJ7mJsVdV2uLi1TKQA9dRHQSFDsQFHmEXL37qTg/eV3q/fXMDrj+kXMXRLx0XXIJcFYBAOqf//ZgIhpt43kErupFitd/m20uVimDPVDo8vjgwdn/QwTy9Sr0DjnaG89+CAfOivAAhlTyt6f19jOz1fmGCNdGQG1z+WJX1XXdP1LcXHwUEqUuD+Yz7BLkZyEIkd92ADcEuf8ZuZxRANy1tQMijkRAdrOsKCWRl255MNvMO7zPvb/8GSXMbOz+C0JGLrsy/WDyzbeDVQ+3dSRisWVcVS4UUiqWNNrqwv3i7CclLQ8ijj99fWZrhqFL59MuD8LiU5WcvRE5c44tmXPGsOr+v1ywBwDEw+G93N2cWMx2XkCpy4MYj528CrCTJfduWXmEGUR++1t/AaO/gKwczrU78IDD5eL/Ya/Yz4UFYZ/Kc686NLwgCXhey8bry4PJjUOI8/Fz5AW6P3yXGGJ3ARDHiayhfFZ2h947Eqps+0KrEo+v5+oFRKNoPVYcHo+tLiIIi2q/bFN12XCQwYPK/sBMvA+yIwzwqkKLZ4QT4XiMPfDzQpf5CNYDxn5PX74ihPyq+/9qKR6A4qioeFXEkU1/NmrLi5ktkQcvIdWaJ/e+s8ahex4gRkgGgVzIyd2cAnB1Rx/CAO5BO4Yd2HVsOAp00i0PHrpwNu3vY5NNHWzLrLtJBKSx/ioHBA3+QO3/QMkCwLwAr7ffYnFP2ZGayNNj/XRI/T6WB5+gGYIU+5eGvDiblwC4Fi3cK+TDI/Npo8KgZCQn8nSA1Kk1/yB/OrKjzBjFQgR7W3+BK2J5cTavFD9qAha03889GQigLsAu/QGpQCLvt5cvKY+tWs2SfHjga7S9Vnu8yudVgv/wvoeUxb7KeX/LtupWH3gNgj0ROXdelAFE8u9H+fyiO99XRDIwEY128D5SlguYnFScVVW2vMgs8ffBu2w3XN3i52oiQmKQBobY3Pqr97zAgbl5e+x5L/JPDb53YsHDD3YqsRh3c439z+3qBQCo8su1PKgD6/4/+MP/EUPsbv3PfyrK+gfSDf4oKQcwS9RwpFfEEUMJ7To9WAfc+WyNQyxnMHhwdkMRgn2Be12g9S+Io45CX71+x7fGEgK8AOweVLl+ne0vPkKBVE8AiULaN0AeTJ04Kcr6Y+mvrpA/cBb6Ds5Kn5AVAZwQuxYHJQP5gGSi4+u/OHSAyC8JcI8LXPkqmJsFC4B3tX+PiMIgAGuidl0WnOPqa9OAEBJsOjRATT6SAPe2wHV/cHJPoX9UcKfP9f/8RajqkYcq1Q/Txv8MJdSTFFHctTW2vhGwtIea/9cDFO/LBIz6ioeEVb++rLr/bwr3AER7ASxBYvOEIEBz+eSC4Pu6KOtftABcePLpoKhcgK6UgnqjCQTjXX/1XhY06HM29k838FOYAIj2Agw4YQSCoa6/QINWtPUvSQBEewGyhAIEcv3LZf0BR6nvXt/77EgiHPGL+GSYRuRbv1YRMZWIQDDC9Q+dOCXS+qPqr6WUF3CWegROn69X5AmkUIBArn9GlMw9B4+jELGJSDI8K5Yp7kbaRJNgHWC9P/LpJZFvkXGzD0M9AMDdUN8r0k3HibTr9CCC/YB7VTD5uVh/bgJwecu2IYfH0y/y04bPnKWlQYIl4n7cq4LRr1r/IdMIAOBbu7pb1LIgO7GR6ZmNEwgEM8f96j0quJwdHOvm9WLcBIAtCy6s6hb5ydFCaYBrRSAUHaoKbPPV0V3Ksl8qHLyPruF73xmMh8JtIs+AnbYZJ9gDArf1TgaXxF8y3LyP0FVT0xUPXT6qfimMoTjRTp8Prcl056UBxk3diknjRSdQMaPBqT5m3UWbjm0r+Xwj6See/LD6Xbxf1CHiSBd/v7dHVcSdIs8GVh28q/3SiQDiy8R0hN10M+SeUu9A9TkSMayVekYYPKoiONXzX6leCye7Do4KD/uZbOQPfxIwIkHdq1r/HksIACC6NoBZJPWmgwjYtVIQlhw3GCP79LQR8SUfLxDbvlVUsOvDHjb1HFjGXyW/AUvU3F1/YSHA7As31HdFr1w9KmJ8WKr62kEEGNFD6gNTkkMhS9c9zArV2FyxZmGbKg52CN8MJL8Q11+4B8BCgV0vPBO9cm236DNkRU8A7npsYkIbi35TuhoHXCtn1QLmLbiqqy0VOhhIfgBZ/z2WFACg4XvbDqgWrUP0+2BVAKsDpraM6AwbV0mvuvZ2H31WTF7BpYYKrppqVRAWmfpYkfATuKNPMrDBx2ahvBH9CXzr1nRNnTjVlohEhK7b6RfETCIASzFLemptzukRRSPB2esIEdDFwEyenYHkF+r6G+YBAEv2vtQWvXJ10Ag3t9yeAJGeP8wiBgaSH2jnVe6b9dwa8Ukm33w7sPArmxxqKNAm+r2QQIM1QbLJ4XQadnMgY4+Rz9MXLiqx4Dg2USHm8hJV9VwyUb02pn4dVhxulxoyeAwVdVxXA8mPJb9+Q0IvIy9k40s7DsSu3+gw4r2MSAwya6/eFNMSjDM3Y86gorFBcaken+hrbGDCz5C4fw5PjLxo3pZVXap6DxtikQUWaMDawx2cOn6C1X8T+cuTM8C5Z9cA47YnJ+1A/mEj4v6yeQDAsh/tao1c/GxQdFIw2RPwrGzisu4MFxAW3yoFOdLlCrCkqHoEPPpEWHnv+QtGkj+oxf3DthYAYPG/fK8jNj5xwKi171LLhkH86c8uk6W3UniwdEnRQmBgeW8yNqvkHzBcNMtxgW6+NXhiYceXxuM3px4xxl9MKNFrYyxxlK8I4OJHR68yKwCrr8TixCyrQL1WSBpGtevG+hTyTAjjb8IjZ9g9YyC6jUr6mcIDSPIEXlGJ2Wnke2K2IGYMZr0Jrlyd2cTRYtV5n1vepNyxfIVyW139vJ/97uIF5djFT5WzY9fk8whUD7Bi6eKccyWRUxC4d18mYLpPV9nOTbkvjlGVgqmxomdV87zssRVdfZD+m59/QHn09juUGl9lzt+HAPzPyGnlpx/9r/L+yJ/m/fyLLX+m/OLpv+dybPXf3WqJ0IBNnz5zrhy5HUMz/mkNYrkvCioFQ6dH/PHJm62GeYjqhQ6dOq14V93G3ENkkMMY4Wwh4sPK9z3awYhf6N/h8dd3/TkTgn/4+U+l8QrYyoF6nSHy3uYm1qXI4n3MmzT+2hue8TelAGCUWNNr+9pVERg0UgRwwSECTAAs1nkHK/2Tv+vKy+JnA8IFGUMCdu1PB8p57UH+dp6jvYqF0wwXBCKgkr9dOzGGwmrkh+WGi14q+YGDvz+myAzZyW8KD0AHTsiyH+3qMrJGwGpAvN/35czpkvHQFCO1nvDT8YXVa5TPLWtiz8nC8e+/eZdOqrFgDT5mIb+pBAC49I2tw6oItJMIpMe//eWTGS0/yPzy24eYCKRCT/bhb5EzePahTez/EIp88dV9P0ybNCQURH7DC30sJQAkAtldf3gA6YBEHrL6uQBxwO/BS8j0WgR5yG+aHEA6EfAsX9ru8HiCdO/MQLfaqdj+q4G8yJ8qBGTNifymFYBkEVDKkBg0Y+yfrrgHy3gUx5saw2Ymv6kFQBcBnEBn1QKpRSDTWv9/fPQhUYzIb18BAJAx9a1pkVoEsO6fDoW6/gTDyW/6ENZthbM5Wyx08vQrRpcNmwHNtXVp3X8jgaKhfHA2eE3K4qIkoKOvywrkt4wA6CKgPm0uRwNRuZEu/h+fmjL0GLLVHyTj5XcOseVISVHWxh5bhgCpGP3W813uxvpuu+4GlC9+d+mCQjAVuq1GfksKABOBrTv2uGqqN8u8TMijFJjABcwzFbl5BwlAek9ggNUKGDRj0GygQh5TQE/2DVj1A7itfPaxTIjkYHjkzCtGTRsui7t/8cI8wqdLDIo+holQ7uYZiRKAlkr22VIAgNnk4Pd7e2IT13facY+9c2Nj8wQAiUF8r5B6/lLw3V+9QdWDGhyeit5rPf/cY4fP4rTLRRn99s4ed2ODLcuHMxEP/QEEw+P9druQ31YCAFzesm2ocv3aFqfPN2Cnz3XwD8cyCkC6JUKCMJe/xYjtukgASgwJrj7/0mZtqdAW3oA+xy8VWAn4yd920YqAeKuPJb7NVo/3pRCA2ZBg6449akiw0eF220KxMxXXIA9Q6IQgeA4jO/6JhCM3cO9stOoSXz6wdTXN5JtvB6cO/+bVRV/9EjYmxbxBn1U/C0psQfa1i5fM+9nSRdVK1z2fV3wVFcq54FjaoSD6INB9f/U15W/uulvxuSuUd079kb1uut9LBfoOUn/X5lb/ZRT2hN47YutaE4csV3TJ3pf8sfHxV+KhcJtVPwMsNqx9rhoAhAwQAh1YMkyXK0hXtptpLLhEE4Fg9bG8F5Dhw7plEYDLW7bhgrY3vryzM35jcnciFrPctCFY9n/8+Wss7s+W/NNHf+cC5gQS5sX6/TJ9aKdsV/nKs739lRvWtzgrK/ut2E+Adf8H/+1fuXQD5tvhJwFA+hbZyC+lAABspWBHXxerG7BgkhCewFf2/aCkTT0gIPAmJAeuPUp5u+yY4accQJ5gYUEotDMRjvitePyI2zE1CLmBL7SsyZgXwKhwxPGoK8gkHFmTgPYp80U42CujxScByCYEfc/1xKdCW6yYH5gX36tigKQhle/Oi/P3qsTvoVNBApAWTa/tqw1/EnjGLkJAuEV89bFHVlefBICEgIhPIAEgISDiE0gAikLd9u5Oh9dj2WShBAgolNwjATBACNocbveWRDTaQWfDFECn3l67deqRAJgcrLz4+o0tiXC4k8KDsrj5/RrxA3Q6SADKioYXtnckpqe/riQSHXacSmQya/+qlefwkQDYOzyAJ9Dh9Hq3xMPhVjojXIDhm0jqDVBSjwTAUiFCNBjsUGLxxxLRaBudkYKAeP4NjfTk4pMAWN8zcPq8HfFQ+EF4COqDcgbzY3q49YfJ0pMAyCAIrQ5PRUciMg1BkNU7GEoi/DDdFSQA0qKx77m22I1J5AwgCHj22+wjBrRYHoQfpiU7EgBCjpBBEwJ4B3dqgmCVpOKwRviPNSs/TC49CQCBU+ig5Q8gDDVJomB0GDGURPZx7f9BcuVJAAjlFQh/UvhQy8FrAKF16x2gjLy98P8CDADMneNqQ66ddAAAAABJRU5ErkJggg==" + longDescription: "Creating a service instance provisions a database. Binding applications provisions unique credentials for each application to access the database." + providerDisplayName: "Pivotal Software" + documentationUrl: "https://github.com/cloudfoundry/cf-mysql-release/blob/master/README.md" + supportUrl: "https://support.pivotal.io" + dashboard_client: + id: p-mysql + secret: (( property_overrides.broker.dashboard_secret )) + plans: (( property_overrides.broker.plans )) + acceptance_tests: + smoke_tests_only: (( property_overrides.acceptance_tests.smoke_tests_only || nil )) + standalone_tests_only: (( property_overrides.acceptance_tests.standalone_tests_only || nil )) + timeout_scale: (( property_overrides.acceptance_tests.timeout_scale || nil )) + +# The keys below should not be included in the final stub +config_from_cf: (( merge )) +iaas_settings: (( merge )) +instance_count_overrides: (( merge || nil )) +property_overrides: (( merge )) +release_versions: (( merge || nil )) +empty_hash: {} +default_mysql_cluster_ips: +- (( jobs.mysql_z1.networks.mysql1.static_ips.[0] || nil )) +- (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) +- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +default_proxy_ips: +- (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) +- (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) +# spiff does not allow concating multiple fields of different types +# without adding new temporary properties +database_startup_timeout_str: (( "" .properties.cf_mysql.mysql.database_startup_timeout )) +canary_end_in_sec: (( database_startup_timeout_str "000" )) diff --git a/templates/sample_stubs/aws_iaas_settings.yml b/manifest-generation/examples/aws/iaas-settings.yml similarity index 100% rename from templates/sample_stubs/aws_iaas_settings.yml rename to manifest-generation/examples/aws/iaas-settings.yml diff --git a/manifest-generation/examples/minimal/instance-count-overrides.yml b/manifest-generation/examples/minimal/instance-count-overrides.yml new file mode 100644 index 00000000..b5e65431 --- /dev/null +++ b/manifest-generation/examples/minimal/instance-count-overrides.yml @@ -0,0 +1,11 @@ +instance_count_overrides: + - name: mysql_z2 + instances: 0 + - name: mysql_z3 + instances: 0 + - name: arbitrator_z3 + instances: 0 + - name: proxy_z2 + instances: 0 + - name: cf-mysql-broker_z2 + instances: 0 diff --git a/manifest-generation/examples/no-arbitrator/instance-count-overrides.yml b/manifest-generation/examples/no-arbitrator/instance-count-overrides.yml new file mode 100644 index 00000000..6489fabc --- /dev/null +++ b/manifest-generation/examples/no-arbitrator/instance-count-overrides.yml @@ -0,0 +1,5 @@ +instance_count_overrides: + - name: mysql_z3 + instances: 1 + - name: arbitrator_z3 + instances: 0 diff --git a/templates/sample_stubs/property_overrides.yml b/manifest-generation/examples/property-overrides.yml similarity index 100% rename from templates/sample_stubs/property_overrides.yml rename to manifest-generation/examples/property-overrides.yml diff --git a/templates/sample_stubs/release_versions.yml b/manifest-generation/examples/release-versions.yml similarity index 100% rename from templates/sample_stubs/release_versions.yml rename to manifest-generation/examples/release-versions.yml diff --git a/manifest-generation/examples/standalone/instance-count-overrides.yml b/manifest-generation/examples/standalone/instance-count-overrides.yml new file mode 100644 index 00000000..526a511c --- /dev/null +++ b/manifest-generation/examples/standalone/instance-count-overrides.yml @@ -0,0 +1,5 @@ +instance_count_overrides: + - name: cf-mysql-broker_z1 + instances: 0 + - name: cf-mysql-broker_z2 + instances: 0 diff --git a/manifest-generation/examples/standalone/property-overrides.yml b/manifest-generation/examples/standalone/property-overrides.yml new file mode 100644 index 00000000..18d5d8f5 --- /dev/null +++ b/manifest-generation/examples/standalone/property-overrides.yml @@ -0,0 +1,24 @@ +property_overrides: + standalone: true + host: YOUR_LOAD_BALANCER_ADDRESS # Optional, set to your Load Balancer address if configured + mysql: + admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process + bootstrap_endpoint: + username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP + password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP + proxy: + api_username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP + api_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP + acceptance_tests: + standalone_tests_only: true + broker: + auth_username: UNUSED_VALUE + auth_password: UNUSED_VALUE + cookie_secret: UNUSED_VALUE + auth_password: UNUSED_VALUE + dashboard_secret: UNUSED_VALUE + plans: + - name: UNUSED_VALUE + id: UNUSED_VALUE + description: UNUSED_VALUE + max_storage_mb: 100 diff --git a/manifest-generation/examples/standalone/standalone-cf-manifest.yml b/manifest-generation/examples/standalone/standalone-cf-manifest.yml new file mode 100644 index 00000000..7f9a752e --- /dev/null +++ b/manifest-generation/examples/standalone/standalone-cf-manifest.yml @@ -0,0 +1,16 @@ +# Standalone deployments have no CF, but these fields are required +# for all deployments. +name: cf +director_uuid: REPLACE_WITH_DIRECTOR_UUID +properties: + admin_username: UNUSED_VALUE + admin_password: UNUSED_VALUE + nats: + machines: [] + user: UNUSED_VALUE + password: UNUSED_VALUE + port: 0 + system_domain: UNUSED_VALUE + app_domains: [UNUSED_VALUE] + domain: UNUSED_VALUE + diff --git a/manifest-generation/examples/vsphere/iaas-settings.yml b/manifest-generation/examples/vsphere/iaas-settings.yml new file mode 100644 index 00000000..a8e71987 --- /dev/null +++ b/manifest-generation/examples/vsphere/iaas-settings.yml @@ -0,0 +1,161 @@ +properties: + datacenters: + - name: z1 + datacenter_name: REPLACE_WITH_YOUR_FIRST_DATACENTER + clusters: + - REPLACE_WITH_YOUR_FIRST_CLUSTER: + resource_pool: REPLACE_WITH_YOUR_FIRST_RESOURCE_POOL + - name: z2 + datacenter_name: REPLACE_WITH_YOUR_SECOND_DATACENTER + clusters: + - REPLACE_WITH_YOUR_SECOND_CLUSTER: + resource_pool: REPLACE_WITH_YOUR_SECOND_RESOURCE_POOL + - name: z3 + datacenter_name: REPLACE_WITH_YOUR_THIRD_DATACENTER + clusters: + - REPLACE_WITH_YOUR_THIRD_CLUSTER: + resource_pool: REPLACE_WITH_YOUR_THIRD_RESOURCE_POOL + +iaas_settings: + stemcell: &stemcell + name: bosh-vsphere-esxi-ubuntu-trusty-go_agent + version: latest + compilation_cloud_properties: + ram: 4096 + disk: 20480 + cpu: 4 + datacenters: + - name: (( .properties.datacenters.z1.datacenter_name )) + clusters: (( .properties.datacenters.z1.clusters )) + resource_pool_cloud_properties: + - name: mysql_z1 + cloud_properties: + ram: 4096 + disk: 10480 + cpu: 2 + datacenters: + - name: (( .properties.datacenters.z1.datacenter_name )) + clusters: (( .properties.datacenters.z1.clusters )) + + - name: mysql_z2 + cloud_properties: + ram: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.ram )) + disk: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.disk )) + cpu: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.cpu )) + datacenters: + - name: (( .properties.datacenters.z2.datacenter_name )) + clusters: (( .properties.datacenters.z2.clusters )) + + - name: mysql_z3 + cloud_properties: + ram: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.ram )) + disk: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.disk )) + cpu: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.cpu )) + datacenters: + - name: (( .properties.datacenters.z3.datacenter_name )) + clusters: (( .properties.datacenters.z3.clusters )) + + - name: arbitrator_z3 + cloud_properties: + ram: 2048 + disk: 4096 + cpu: 2 + datacenters: + - name: (( .properties.datacenters.z3.datacenter_name )) + clusters: (( .properties.datacenters.z3.clusters )) + + - name: proxy_z1 + cloud_properties: + ram: 2048 + disk: 4096 + cpu: 2 + datacenters: + - name: (( .properties.datacenters.z1.datacenter_name )) + clusters: (( .properties.datacenters.z1.clusters )) + + - name: proxy_z2 + cloud_properties: + ram: (( iaas_settings.resource_pool_cloud_properties.proxy_z1.cloud_properties.ram )) + disk: (( iaas_settings.resource_pool_cloud_properties.proxy_z1.cloud_properties.disk )) + cpu: (( iaas_settings.resource_pool_cloud_properties.proxy_z1.cloud_properties.cpu )) + datacenters: + - name: (( .properties.datacenters.z2.datacenter_name )) + clusters: (( .properties.datacenters.z2.clusters )) + + - name: cf-mysql-broker_z1 + cloud_properties: + ram: 2048 + disk: 4096 + cpu: 1 + datacenters: + - name: (( .properties.datacenters.z1.datacenter_name )) + clusters: (( .properties.datacenters.z1.clusters )) + + - name: cf-mysql-broker_z2 + cloud_properties: + ram: (( iaas_settings.resource_pool_cloud_properties.cf-mysql-broker_z1.cloud_properties.ram )) + disk: (( iaas_settings.resource_pool_cloud_properties.cf-mysql-broker_z1.cloud_properties.disk )) + cpu: (( iaas_settings.resource_pool_cloud_properties.cf-mysql-broker_z1.cloud_properties.cpu )) + datacenters: + - name: (( .properties.datacenters.z2.datacenter_name )) + clusters: (( .properties.datacenters.z2.clusters )) + + - name: errands_z1 + cloud_properties: + ram: 2048 + disk: 4096 + cpu: 1 + datacenters: + - name: (( .properties.datacenters.z1.datacenter_name )) + clusters: (( .properties.datacenters.z1.clusters )) + + disk_pools: + - name: mysql-persistent-disk + disk_size: 100000 + +# NOTE: +# Replace 10.0.{0,1,2} part of the octet +# with your network configuration + subnet_configs: + - name: mysql1 + subnets: + - range: 10.0.0.0/24 + gateway: 10.0.0.1 + reserved: + - 10.0.0.2-10.0.0.109 + - 10.0.0.130-10.0.0.254 + static: + - 10.0.0.120-10.0.0.124 + dns: + - 10.0.0.3 + - 8.8.8.8 + cloud_properties: + name: REPLACE_WITH_YOUR_SUBNET_NAME #This should match your vSphere network name, e.g. "VM Network". + - name: mysql2 + subnets: + - range: 10.0.1.0/24 + gateway: 10.0.1.1 + reserved: + - 10.0.1.2-10.0.1.109 + - 10.0.1.130-10.0.1.254 + static: + - 10.0.1.120-10.0.1.124 + dns: + - 10.0.1.3 + - 8.8.8.8 + cloud_properties: + name: REPLACE_WITH_YOUR_SUBNET_NAME_2 + - name: mysql3 + subnets: + - range: 10.0.2.0/24 + gateway: 10.0.2.1 + reserved: + - 10.0.2.2-10.0.2.109 + - 10.0.2.130-10.0.2.254 + static: + - 10.0.2.120-10.0.2.124 + dns: + - 10.0.2.3 + - 8.8.8.8 + cloud_properties: + name: REPLACE_WITH_YOUR_SUBNET_NAME_3 diff --git a/templates/bosh.yml b/manifest-generation/misc-templates/bosh.yml similarity index 100% rename from templates/bosh.yml rename to manifest-generation/misc-templates/bosh.yml diff --git a/templates/config-from-cf-internal.yml b/manifest-generation/misc-templates/config-from-cf-internal.yml similarity index 100% rename from templates/config-from-cf-internal.yml rename to manifest-generation/misc-templates/config-from-cf-internal.yml diff --git a/templates/config-from-cf.yml b/manifest-generation/misc-templates/config-from-cf.yml similarity index 100% rename from templates/config-from-cf.yml rename to manifest-generation/misc-templates/config-from-cf.yml diff --git a/scripts/generate-bosh-lite-manifest b/scripts/generate-bosh-lite-manifest new file mode 100755 index 00000000..4c4e6b17 --- /dev/null +++ b/scripts/generate-bosh-lite-manifest @@ -0,0 +1,31 @@ +#!/bin/bash + +set -eu + +MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +RELEASE_DIR="$( cd "${MY_DIR}/.." && pwd )" + +DIRECTOR_IP=${DIRECTOR_IP:-192.168.50.4} +BOSH_LITE_USERNAME=${BOSH_LITE_USERNAME:-admin} +BOSH_LITE_PASSWORD=${BOSH_LITE_PASSWORD:-admin} + +pushd "${RELEASE_DIR}" > /dev/null + + bosh -n target "${DIRECTOR_IP}" + bosh -n login "${BOSH_LITE_USERNAME}" "${BOSH_LITE_PASSWORD}" + + tmpdir=$(mktemp -d /tmp/mysql_manifest.XXXXX) + trap '{ rm -rf ${tmpdir}; }' EXIT + + bosh -n download manifest cf-warden "${tmpdir}/cf-warden.yml" + + ./scripts/generate-deployment-manifest \ + -c "${tmpdir}/cf-warden.yml" \ + -p ./manifest-generation/bosh-lite-stubs/property-overrides.yml \ + -i ./manifest-generation/bosh-lite-stubs/iaas-settings.yml \ + > cf-mysql.yml + + bosh deployment cf-mysql.yml +popd > /dev/null + +echo "CF-MySQL Manifest was generated at ${RELEASE_DIR}/cf-mysql.yml" diff --git a/scripts/generate-deployment-manifest b/scripts/generate-deployment-manifest new file mode 100755 index 00000000..a032c202 --- /dev/null +++ b/scripts/generate-deployment-manifest @@ -0,0 +1,104 @@ +#!/bin/bash + +MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +RELEASE_DIR="$( cd "${MY_DIR}/.." && pwd )" +MANIFEST_DIR="$( cd "${RELEASE_DIR}/manifest-generation" && pwd )" + +argument_error=false + +function usage(){ + >&2 echo " Usage: + Mandatory arguments: + -c CF Manifest + -p Property overrides stub file + -i Infrastructure type stub file + Optional arguments + -n Instance count overrides stub file (single node, 3 node) + -v Release versions stub file + " + exit 1 +} + +while getopts "c:p:i:n:v:" opt; do + case $opt in + c) + cf_manifest=$OPTARG + ;; + p) + property_overrides=$OPTARG + ;; + i) + infrastructure_type=$OPTARG + ;; + n) + instance_counts=$OPTARG + ;; + v) + release_versions=$OPTARG + ;; + *) + echo "Unknown arguments" + usage + ;; + esac +done + +if [ -z "${infrastructure_type}" ]; then + >&2 echo "infrastructure_type stub file is missing" + argument_error=true +elif [ ! -f "${infrastructure_type}" ]; then + >&2 echo "infrastructure_type stub ${infrastructure_type} is not a regular file" + argument_error=true +fi + +if [ -z "${cf_manifest}" ]; then + >&2 echo "cf manifest file is missing" + argument_error=true +elif [ ! -f "${cf_manifest}" ]; then + >&2 echo "cf manifest ${cf_manifest} is not a regular file" + argument_error=true +fi + +if [ -z ${property_overrides} ]; then + >&2 echo "property-overrides stub file is missing" + argument_error=true +elif [ ! -f "${property_overrides}" ]; then + >&2 echo "Property Overrides stub '${property_overrides}' is not a regular file" + argument_error=true +fi + +if [[ ! -z "${instance_counts}" && ! -f "${instance_counts}" ]]; then + >&2 echo "Instance Count Overrides stub '${instance_counts}' is not a regular file" + argument_error=true +fi + +if [[ ! -z "${release_versions}" && ! -f "${release_versions}" ]]; then + >&2 echo "Release versions stub '${release_versions}' is not a regular file" + argument_error=true +fi + +if [ "$argument_error" = "true" ]; then + usage +fi + +tmpdir=$(mktemp -d /tmp/mysql_manifest.XXXXX) +trap '{ rm -rf ${tmpdir}; }' EXIT + +spiff merge \ + ${MANIFEST_DIR}/misc-templates/config-from-cf.yml \ + ${MANIFEST_DIR}/misc-templates/config-from-cf-internal.yml \ + ${cf_manifest} \ + > ${tmpdir}/config-from-cf.yml + +spiff merge \ + ${MANIFEST_DIR}/cf-mysql-template.yml \ + ${tmpdir}/config-from-cf.yml \ + ${infrastructure_type} \ + ${instance_counts} \ + ${property_overrides} \ + ${release_versions} \ + > ${tmpdir}/mysql-config.yml + +spiff merge \ + ${MANIFEST_DIR}/misc-templates/bosh.yml \ + ${tmpdir}/mysql-config.yml diff --git a/scripts/test_sample_stubs b/scripts/test_sample_stubs index 818bb508..f9b639bc 100755 --- a/scripts/test_sample_stubs +++ b/scripts/test_sample_stubs @@ -7,6 +7,20 @@ MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" # This script assumes that it lives one directory below the release directory. RELEASE_DIR="$( cd ${MY_DIR}/.. && pwd )" +# test the following cases + +# aws +# +# vsphere +# +# bosh-lite +# +# standalone +# +# no arbitrator +# +# minimal + ${RELEASE_DIR}/generate_deployment_manifest \ aws \ ${RELEASE_DIR}/templates/sample_stubs/sample_plans_stub.yml \ diff --git a/templates/cf-infrastructure-aws.yml b/templates/cf-infrastructure-aws.yml deleted file mode 100644 index 80f39ee9..00000000 --- a/templates/cf-infrastructure-aws.yml +++ /dev/null @@ -1,74 +0,0 @@ -meta: ~ - -stemcell: &stemcell - name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent - version: latest - -compilation: - cloud_properties: - instance_type: (( merge || "c4.large" )) - availability_zone: (( properties.template_only.aws.availability_zone )) - ephemeral_disk: - size: 30000 # in MB - type: gp2 - -networks: (( merge )) - -resource_pools: -- name: mysql_z1 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.large" )) - availability_zone: (( properties.template_only.aws.availability_zone )) - -- name: mysql_z2 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.large" )) - availability_zone: (( properties.template_only.aws.availability_zone2 )) - -- name: mysql_z3 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.large" )) - availability_zone: (( properties.template_only.aws.availability_zone3 )) - -- name: proxy_z1 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.medium" )) - availability_zone: (( properties.template_only.aws.availability_zone )) - elbs: (( merge || nil )) - -- name: proxy_z2 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.medium" )) - availability_zone: (( properties.template_only.aws.availability_zone2 )) - elbs: (( merge || nil )) - -- name: cf-mysql-broker_z1 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.medium" )) - availability_zone: (( properties.template_only.aws.availability_zone )) - -- name: cf-mysql-broker_z2 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.medium" )) - availability_zone: (( properties.template_only.aws.availability_zone2 )) - -- name: errands_z1 - stemcell: *stemcell - cloud_properties: - instance_type: (( merge || "m3.medium" )) - availability_zone: (( properties.template_only.aws.availability_zone )) - -disk_pools: -- name: mysql-persistent-disk - cloud_properties: - type: gp2 - -properties: - template_only: (( merge )) diff --git a/templates/cf-infrastructure-vsphere.yml b/templates/cf-infrastructure-vsphere.yml deleted file mode 100644 index dc72bcb3..00000000 --- a/templates/cf-infrastructure-vsphere.yml +++ /dev/null @@ -1,109 +0,0 @@ -meta: ~ - -stemcell: &stemcell - name: bosh-vsphere-esxi-ubuntu-trusty-go_agent - version: latest - -compilation: - cloud_properties: - ram: (( merge || 4096 )) - disk: (( merge || 20480 )) - cpu: (( merge || 4 )) - datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) - -networks: (( merge )) - -properties: - datacenters: - - name: z1 - datacenter_name: (( merge )) - clusters: (( merge )) - - name: z2 - datacenter_name: (( merge )) - clusters: (( merge )) - - name: z3 - datacenter_name: (( merge )) - clusters: (( merge )) - -resource_pools: -- name: mysql_z1 - stemcell: *stemcell - cloud_properties: - ram: (( merge || 4096 )) - disk: (( merge || 10480 )) - cpu: (( merge || 2 )) - datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) - -- name: mysql_z2 - stemcell: *stemcell - cloud_properties: - ram: (( resource_pools.mysql_z1.cloud_properties.ram )) - disk: (( resource_pools.mysql_z1.cloud_properties.disk )) - cpu: (( resource_pools.mysql_z1.cloud_properties.cpu )) - datacenters: - - name: (( .properties.datacenters.z2.datacenter_name )) - clusters: (( .properties.datacenters.z2.clusters )) - -- name: mysql_z3 - stemcell: *stemcell - cloud_properties: - ram: (( resource_pools.mysql_z1.cloud_properties.ram )) - disk: (( resource_pools.mysql_z1.cloud_properties.disk )) - cpu: (( resource_pools.mysql_z1.cloud_properties.cpu )) - datacenters: - - name: (( .properties.datacenters.z3.datacenter_name )) - clusters: (( .properties.datacenters.z3.clusters )) - -- name: proxy_z1 - stemcell: *stemcell - cloud_properties: - ram: (( merge || 2048 )) - disk: (( merge || 4096 )) - cpu: (( merge || 1 )) - datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) - -- name: proxy_z2 - stemcell: *stemcell - cloud_properties: - ram: (( resource_pools.proxy_z1.cloud_properties.ram )) - disk: (( resource_pools.proxy_z1.cloud_properties.disk )) - cpu: (( resource_pools.proxy_z1.cloud_properties.cpu )) - datacenters: - - name: (( .properties.datacenters.z2.datacenter_name )) - clusters: (( .properties.datacenters.z2.clusters )) - -- name: cf-mysql-broker_z1 - stemcell: *stemcell - cloud_properties: - ram: (( merge || 2048 )) - disk: (( merge || 4096 )) - cpu: (( merge || 1 )) - datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) - -- name: cf-mysql-broker_z2 - stemcell: *stemcell - cloud_properties: - ram: (( resource_pools.cf-mysql-broker_z1.cloud_properties.ram )) - disk: (( resource_pools.cf-mysql-broker_z1.cloud_properties.disk )) - cpu: (( resource_pools.cf-mysql-broker_z1.cloud_properties.cpu )) - datacenters: - - name: (( .properties.datacenters.z2.datacenter_name )) - clusters: (( .properties.datacenters.z2.clusters )) - -- name: errands_z1 - stemcell: *stemcell - cloud_properties: - ram: (( merge || 2048 )) - disk: (( merge || 4096 )) - cpu: (( merge || 1 )) - datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) \ No newline at end of file diff --git a/templates/cf-infrastructure-warden.yml b/templates/cf-infrastructure-warden.yml deleted file mode 100644 index 7aa3848a..00000000 --- a/templates/cf-infrastructure-warden.yml +++ /dev/null @@ -1,2141 +0,0 @@ -meta: - fog_config: - provider: Local - local_root: /var/vcap/store - -compilation: - workers: 6 - network: services1 - reuse_compilation_vms: true - cloud_properties: - name: random - -stemcell: &stemcell - name: bosh-warden-boshlite-ubuntu-trusty-go_agent - version: latest - -resource_pools: -- name: mysql_z1 - network: mysql1 - stemcell: *stemcell - cloud_properties: {name: random} -- name: mysql_z2 - network: mysql1 - stemcell: *stemcell - cloud_properties: {name: random} -- name: mysql_z3 - network: mysql1 - stemcell: *stemcell - cloud_properties: {name: random} -- name: proxy_z1 - network: mysql1 - stemcell: *stemcell - cloud_properties: {name: random} -- name: proxy_z2 - network: mysql2 - stemcell: *stemcell - cloud_properties: {name: random} -- name: cf-mysql-broker_z1 - network: mysql1 - stemcell: *stemcell - cloud_properties: {name: random} -- name: cf-mysql-broker_z2 - network: mysql2 - stemcell: *stemcell - cloud_properties: {name: random} - -networks: -- name: services1 - subnets: - - cloud_properties: - name: random - range: 10.244.1.0/30 - reserved: - - 10.244.1.1 - static: - - 10.244.1.2 - - cloud_properties: - name: random - range: 10.244.1.4/30 - reserved: - - 10.244.1.5 - static: - - 10.244.1.6 - - cloud_properties: - name: random - range: 10.244.1.8/30 - reserved: - - 10.244.1.9 - static: - - 10.244.1.10 - - cloud_properties: - name: random - range: 10.244.1.12/30 - reserved: - - 10.244.1.13 - static: - - 10.244.1.14 - - cloud_properties: - name: random - range: 10.244.1.16/30 - reserved: - - 10.244.1.17 - static: - - 10.244.1.18 - - cloud_properties: - name: random - range: 10.244.1.20/30 - reserved: - - 10.244.1.21 - static: - - 10.244.1.22 - - cloud_properties: - name: random - range: 10.244.1.24/30 - reserved: - - 10.244.1.25 - static: - - 10.244.1.26 - - cloud_properties: - name: random - range: 10.244.1.28/30 - reserved: - - 10.244.1.29 - static: - - 10.244.1.30 - - cloud_properties: - name: random - range: 10.244.1.32/30 - reserved: - - 10.244.1.33 - static: - - 10.244.1.34 - - cloud_properties: - name: random - range: 10.244.1.36/30 - reserved: - - 10.244.1.37 - static: - - 10.244.1.38 - - cloud_properties: - name: random - range: 10.244.1.40/30 - reserved: - - 10.244.1.41 - static: - - 10.244.1.42 - - cloud_properties: - name: random - range: 10.244.1.44/30 - reserved: - - 10.244.1.45 - static: - - 10.244.1.46 - - cloud_properties: - name: random - range: 10.244.1.48/30 - reserved: - - 10.244.1.49 - static: - - 10.244.1.50 - - cloud_properties: - name: random - range: 10.244.1.52/30 - reserved: - - 10.244.1.53 - static: - - 10.244.1.54 - - cloud_properties: - name: random - range: 10.244.1.56/30 - reserved: - - 10.244.1.57 - static: - - 10.244.1.58 - - cloud_properties: - name: random - range: 10.244.1.60/30 - reserved: - - 10.244.1.61 - static: - - 10.244.1.62 - - cloud_properties: - name: random - range: 10.244.1.64/30 - reserved: - - 10.244.1.65 - static: - - 10.244.1.66 - - cloud_properties: - name: random - range: 10.244.1.68/30 - reserved: - - 10.244.1.69 - static: - - 10.244.1.70 - - cloud_properties: - name: random - range: 10.244.1.72/30 - reserved: - - 10.244.1.73 - static: - - 10.244.1.74 - - cloud_properties: - name: random - range: 10.244.1.76/30 - reserved: - - 10.244.1.77 - static: - - 10.244.1.78 - - cloud_properties: - name: random - range: 10.244.1.80/30 - reserved: - - 10.244.1.81 - static: - - 10.244.1.82 - - cloud_properties: - name: random - range: 10.244.1.84/30 - reserved: - - 10.244.1.85 - static: - - 10.244.1.86 - - cloud_properties: - name: random - range: 10.244.1.88/30 - reserved: - - 10.244.1.89 - static: - - 10.244.1.90 - - cloud_properties: - name: random - range: 10.244.1.92/30 - reserved: - - 10.244.1.93 - static: - - 10.244.1.94 - - cloud_properties: - name: random - range: 10.244.1.96/30 - reserved: - - 10.244.1.97 - static: - - 10.244.1.98 - - cloud_properties: - name: random - range: 10.244.1.100/30 - reserved: - - 10.244.1.101 - static: - - 10.244.1.102 - - cloud_properties: - name: random - range: 10.244.1.104/30 - reserved: - - 10.244.1.105 - static: - - 10.244.1.106 - - cloud_properties: - name: random - range: 10.244.1.108/30 - reserved: - - 10.244.1.109 - static: - - 10.244.1.110 - - cloud_properties: - name: random - range: 10.244.1.112/30 - reserved: - - 10.244.1.113 - static: - - 10.244.1.114 - - cloud_properties: - name: random - range: 10.244.1.116/30 - reserved: - - 10.244.1.117 - static: - - 10.244.1.118 - - cloud_properties: - name: random - range: 10.244.1.120/30 - reserved: - - 10.244.1.121 - static: - - 10.244.1.122 - - cloud_properties: - name: random - range: 10.244.1.124/30 - reserved: - - 10.244.1.125 - static: - - 10.244.1.126 - - cloud_properties: - name: random - range: 10.244.1.128/30 - reserved: - - 10.244.1.129 - static: [] - - cloud_properties: - name: random - range: 10.244.1.132/30 - reserved: - - 10.244.1.133 - static: [] - - cloud_properties: - name: random - range: 10.244.1.136/30 - reserved: - - 10.244.1.137 - static: [] - - cloud_properties: - name: random - range: 10.244.1.140/30 - reserved: - - 10.244.1.141 - static: [] - - cloud_properties: - name: random - range: 10.244.1.144/30 - reserved: - - 10.244.1.145 - static: [] - - cloud_properties: - name: random - range: 10.244.1.148/30 - reserved: - - 10.244.1.149 - static: [] - - cloud_properties: - name: random - range: 10.244.1.152/30 - reserved: - - 10.244.1.153 - static: [] - - cloud_properties: - name: random - range: 10.244.1.156/30 - reserved: - - 10.244.1.157 - static: [] - - cloud_properties: - name: random - range: 10.244.1.160/30 - reserved: - - 10.244.1.161 - static: [] - - cloud_properties: - name: random - range: 10.244.1.164/30 - reserved: - - 10.244.1.165 - static: [] - - cloud_properties: - name: random - range: 10.244.1.168/30 - reserved: - - 10.244.1.169 - static: [] - - cloud_properties: - name: random - range: 10.244.1.172/30 - reserved: - - 10.244.1.173 - static: [] - - cloud_properties: - name: random - range: 10.244.1.176/30 - reserved: - - 10.244.1.177 - static: [] - - cloud_properties: - name: random - range: 10.244.1.180/30 - reserved: - - 10.244.1.181 - static: [] - - cloud_properties: - name: random - range: 10.244.1.184/30 - reserved: - - 10.244.1.185 - static: [] - - cloud_properties: - name: random - range: 10.244.1.188/30 - reserved: - - 10.244.1.189 - static: [] - - cloud_properties: - name: random - range: 10.244.1.192/30 - reserved: - - 10.244.1.193 - static: [] - - cloud_properties: - name: random - range: 10.244.1.196/30 - reserved: - - 10.244.1.197 - static: [] - - cloud_properties: - name: random - range: 10.244.1.200/30 - reserved: - - 10.244.1.201 - static: [] - - cloud_properties: - name: random - range: 10.244.1.204/30 - reserved: - - 10.244.1.205 - static: [] - - cloud_properties: - name: random - range: 10.244.1.208/30 - reserved: - - 10.244.1.209 - static: [] - - cloud_properties: - name: random - range: 10.244.1.212/30 - reserved: - - 10.244.1.213 - static: [] - - cloud_properties: - name: random - range: 10.244.1.216/30 - reserved: - - 10.244.1.217 - static: [] - - cloud_properties: - name: random - range: 10.244.1.220/30 - reserved: - - 10.244.1.221 - static: [] - - cloud_properties: - name: random - range: 10.244.1.224/30 - reserved: - - 10.244.1.225 - static: [] - - cloud_properties: - name: random - range: 10.244.1.228/30 - reserved: - - 10.244.1.229 - static: [] - - cloud_properties: - name: random - range: 10.244.1.232/30 - reserved: - - 10.244.1.233 - static: [] - - cloud_properties: - name: random - range: 10.244.1.236/30 - reserved: - - 10.244.1.237 - static: [] - - cloud_properties: - name: random - range: 10.244.1.240/30 - reserved: - - 10.244.1.241 - static: [] - - cloud_properties: - name: random - range: 10.244.1.244/30 - reserved: - - 10.244.1.245 - static: [] - - cloud_properties: - name: random - range: 10.244.1.248/30 - reserved: - - 10.244.1.249 - static: [] - - cloud_properties: - name: random - range: 10.244.1.252/30 - reserved: - - 10.244.1.253 - static: [] -- name: services2 - subnets: - - cloud_properties: - name: random - range: 10.244.3.0/30 - reserved: - - 10.244.3.1 - static: - - 10.244.3.2 - - cloud_properties: - name: random - range: 10.244.3.4/30 - reserved: - - 10.244.3.5 - static: - - 10.244.3.6 - - cloud_properties: - name: random - range: 10.244.3.8/30 - reserved: - - 10.244.3.9 - static: - - 10.244.3.10 - - cloud_properties: - name: random - range: 10.244.3.12/30 - reserved: - - 10.244.3.13 - static: - - 10.244.3.14 - - cloud_properties: - name: random - range: 10.244.3.16/30 - reserved: - - 10.244.3.17 - static: - - 10.244.3.18 - - cloud_properties: - name: random - range: 10.244.3.20/30 - reserved: - - 10.244.3.21 - static: - - 10.244.3.22 - - cloud_properties: - name: random - range: 10.244.3.24/30 - reserved: - - 10.244.3.25 - static: - - 10.244.3.26 - - cloud_properties: - name: random - range: 10.244.3.28/30 - reserved: - - 10.244.3.29 - static: - - 10.244.3.30 - - cloud_properties: - name: random - range: 10.244.3.32/30 - reserved: - - 10.244.3.33 - static: - - 10.244.3.34 - - cloud_properties: - name: random - range: 10.244.3.36/30 - reserved: - - 10.244.3.37 - static: - - 10.244.3.38 - - cloud_properties: - name: random - range: 10.244.3.40/30 - reserved: - - 10.244.3.41 - static: - - 10.244.3.42 - - cloud_properties: - name: random - range: 10.244.3.44/30 - reserved: - - 10.244.3.45 - static: - - 10.244.3.46 - - cloud_properties: - name: random - range: 10.244.3.48/30 - reserved: - - 10.244.3.49 - static: - - 10.244.3.50 - - cloud_properties: - name: random - range: 10.244.3.52/30 - reserved: - - 10.244.3.53 - static: - - 10.244.3.54 - - cloud_properties: - name: random - range: 10.244.3.56/30 - reserved: - - 10.244.3.57 - static: - - 10.244.3.58 - - cloud_properties: - name: random - range: 10.244.3.60/30 - reserved: - - 10.244.3.61 - static: - - 10.244.3.62 - - cloud_properties: - name: random - range: 10.244.3.64/30 - reserved: - - 10.244.3.65 - static: - - 10.244.3.66 - - cloud_properties: - name: random - range: 10.244.3.68/30 - reserved: - - 10.244.3.69 - static: - - 10.244.3.70 - - cloud_properties: - name: random - range: 10.244.3.72/30 - reserved: - - 10.244.3.73 - static: - - 10.244.3.74 - - cloud_properties: - name: random - range: 10.244.3.76/30 - reserved: - - 10.244.3.77 - static: - - 10.244.3.78 - - cloud_properties: - name: random - range: 10.244.3.80/30 - reserved: - - 10.244.3.81 - static: - - 10.244.3.82 - - cloud_properties: - name: random - range: 10.244.3.84/30 - reserved: - - 10.244.3.85 - static: - - 10.244.3.86 - - cloud_properties: - name: random - range: 10.244.3.88/30 - reserved: - - 10.244.3.89 - static: - - 10.244.3.90 - - cloud_properties: - name: random - range: 10.244.3.92/30 - reserved: - - 10.244.3.93 - static: - - 10.244.3.94 - - cloud_properties: - name: random - range: 10.244.3.96/30 - reserved: - - 10.244.3.97 - static: - - 10.244.3.98 - - cloud_properties: - name: random - range: 10.244.3.100/30 - reserved: - - 10.244.3.101 - static: - - 10.244.3.102 - - cloud_properties: - name: random - range: 10.244.3.104/30 - reserved: - - 10.244.3.105 - static: - - 10.244.3.106 - - cloud_properties: - name: random - range: 10.244.3.108/30 - reserved: - - 10.244.3.109 - static: - - 10.244.3.110 - - cloud_properties: - name: random - range: 10.244.3.112/30 - reserved: - - 10.244.3.113 - static: - - 10.244.3.114 - - cloud_properties: - name: random - range: 10.244.3.116/30 - reserved: - - 10.244.3.117 - static: - - 10.244.3.118 - - cloud_properties: - name: random - range: 10.244.3.120/30 - reserved: - - 10.244.3.121 - static: - - 10.244.3.122 - - cloud_properties: - name: random - range: 10.244.3.124/30 - reserved: - - 10.244.3.125 - static: - - 10.244.3.126 - - cloud_properties: - name: random - range: 10.244.3.128/30 - reserved: - - 10.244.3.129 - static: [] - - cloud_properties: - name: random - range: 10.244.3.132/30 - reserved: - - 10.244.3.133 - static: [] - - cloud_properties: - name: random - range: 10.244.3.136/30 - reserved: - - 10.244.3.137 - static: [] - - cloud_properties: - name: random - range: 10.244.3.140/30 - reserved: - - 10.244.3.141 - static: [] - - cloud_properties: - name: random - range: 10.244.3.144/30 - reserved: - - 10.244.3.145 - static: [] - - cloud_properties: - name: random - range: 10.244.3.148/30 - reserved: - - 10.244.3.149 - static: [] - - cloud_properties: - name: random - range: 10.244.3.152/30 - reserved: - - 10.244.3.153 - static: [] - - cloud_properties: - name: random - range: 10.244.3.156/30 - reserved: - - 10.244.3.157 - static: [] - - cloud_properties: - name: random - range: 10.244.3.160/30 - reserved: - - 10.244.3.161 - static: [] - - cloud_properties: - name: random - range: 10.244.3.164/30 - reserved: - - 10.244.3.165 - static: [] - - cloud_properties: - name: random - range: 10.244.3.168/30 - reserved: - - 10.244.3.169 - static: [] - - cloud_properties: - name: random - range: 10.244.3.172/30 - reserved: - - 10.244.3.173 - static: [] - - cloud_properties: - name: random - range: 10.244.3.176/30 - reserved: - - 10.244.3.177 - static: [] - - cloud_properties: - name: random - range: 10.244.3.180/30 - reserved: - - 10.244.3.181 - static: [] - - cloud_properties: - name: random - range: 10.244.3.184/30 - reserved: - - 10.244.3.185 - static: [] - - cloud_properties: - name: random - range: 10.244.3.188/30 - reserved: - - 10.244.3.189 - static: [] - - cloud_properties: - name: random - range: 10.244.3.192/30 - reserved: - - 10.244.3.193 - static: [] - - cloud_properties: - name: random - range: 10.244.3.196/30 - reserved: - - 10.244.3.197 - static: [] - - cloud_properties: - name: random - range: 10.244.3.200/30 - reserved: - - 10.244.3.201 - static: [] - - cloud_properties: - name: random - range: 10.244.3.204/30 - reserved: - - 10.244.3.205 - static: [] - - cloud_properties: - name: random - range: 10.244.3.208/30 - reserved: - - 10.244.3.209 - static: [] - - cloud_properties: - name: random - range: 10.244.3.212/30 - reserved: - - 10.244.3.213 - static: [] - - cloud_properties: - name: random - range: 10.244.3.216/30 - reserved: - - 10.244.3.217 - static: [] - - cloud_properties: - name: random - range: 10.244.3.220/30 - reserved: - - 10.244.3.221 - static: [] - - cloud_properties: - name: random - range: 10.244.3.224/30 - reserved: - - 10.244.3.225 - static: [] - - cloud_properties: - name: random - range: 10.244.3.228/30 - reserved: - - 10.244.3.229 - static: [] - - cloud_properties: - name: random - range: 10.244.3.232/30 - reserved: - - 10.244.3.233 - static: [] - - cloud_properties: - name: random - range: 10.244.3.236/30 - reserved: - - 10.244.3.237 - static: [] - - cloud_properties: - name: random - range: 10.244.3.240/30 - reserved: - - 10.244.3.241 - static: [] - - cloud_properties: - name: random - range: 10.244.3.244/30 - reserved: - - 10.244.3.245 - static: [] - - cloud_properties: - name: random - range: 10.244.3.248/30 - reserved: - - 10.244.3.249 - static: [] - - cloud_properties: - name: random - range: 10.244.3.252/30 - reserved: - - 10.244.3.253 - static: [] - -- name: mysql1 - subnets: - - cloud_properties: - name: random - range: 10.244.7.0/30 - reserved: - - 10.244.7.1 - static: - - 10.244.7.2 - - cloud_properties: - name: random - range: 10.244.7.4/30 - reserved: - - 10.244.7.5 - static: - - 10.244.7.6 - - cloud_properties: - name: random - range: 10.244.7.8/30 - reserved: - - 10.244.7.9 - static: - - 10.244.7.10 - - cloud_properties: - name: random - range: 10.244.7.12/30 - reserved: - - 10.244.7.13 - static: - - 10.244.7.14 - - cloud_properties: - name: random - range: 10.244.7.16/30 - reserved: - - 10.244.7.17 - static: - - 10.244.7.18 - - cloud_properties: - name: random - range: 10.244.7.20/30 - reserved: - - 10.244.7.21 - static: - - 10.244.7.22 - - cloud_properties: - name: random - range: 10.244.7.24/30 - reserved: - - 10.244.7.25 - static: - - 10.244.7.26 - - cloud_properties: - name: random - range: 10.244.7.28/30 - reserved: - - 10.244.7.29 - static: - - 10.244.7.30 - - cloud_properties: - name: random - range: 10.244.7.32/30 - reserved: - - 10.244.7.33 - static: - - 10.244.7.34 - - cloud_properties: - name: random - range: 10.244.7.36/30 - reserved: - - 10.244.7.37 - static: - - 10.244.7.38 - - cloud_properties: - name: random - range: 10.244.7.40/30 - reserved: - - 10.244.7.41 - static: - - 10.244.7.42 - - cloud_properties: - name: random - range: 10.244.7.44/30 - reserved: - - 10.244.7.45 - static: - - 10.244.7.46 - - cloud_properties: - name: random - range: 10.244.7.48/30 - reserved: - - 10.244.7.49 - static: - - 10.244.7.50 - - cloud_properties: - name: random - range: 10.244.7.52/30 - reserved: - - 10.244.7.53 - static: - - 10.244.7.54 - - cloud_properties: - name: random - range: 10.244.7.56/30 - reserved: - - 10.244.7.57 - static: - - 10.244.7.58 - - cloud_properties: - name: random - range: 10.244.7.60/30 - reserved: - - 10.244.7.61 - static: - - 10.244.7.62 - - cloud_properties: - name: random - range: 10.244.7.64/30 - reserved: - - 10.244.7.65 - static: - - 10.244.7.66 - - cloud_properties: - name: random - range: 10.244.7.68/30 - reserved: - - 10.244.7.69 - static: - - 10.244.7.70 - - cloud_properties: - name: random - range: 10.244.7.72/30 - reserved: - - 10.244.7.73 - static: - - 10.244.7.74 - - cloud_properties: - name: random - range: 10.244.7.76/30 - reserved: - - 10.244.7.77 - static: - - 10.244.7.78 - - cloud_properties: - name: random - range: 10.244.7.80/30 - reserved: - - 10.244.7.81 - static: - - 10.244.7.82 - - cloud_properties: - name: random - range: 10.244.7.84/30 - reserved: - - 10.244.7.85 - static: - - 10.244.7.86 - - cloud_properties: - name: random - range: 10.244.7.88/30 - reserved: - - 10.244.7.89 - static: - - 10.244.7.90 - - cloud_properties: - name: random - range: 10.244.7.92/30 - reserved: - - 10.244.7.93 - static: - - 10.244.7.94 - - cloud_properties: - name: random - range: 10.244.7.96/30 - reserved: - - 10.244.7.97 - static: - - 10.244.7.98 - - cloud_properties: - name: random - range: 10.244.7.100/30 - reserved: - - 10.244.7.101 - static: - - 10.244.7.102 - - cloud_properties: - name: random - range: 10.244.7.104/30 - reserved: - - 10.244.7.105 - static: - - 10.244.7.106 - - cloud_properties: - name: random - range: 10.244.7.108/30 - reserved: - - 10.244.7.109 - static: - - 10.244.7.110 - - cloud_properties: - name: random - range: 10.244.7.112/30 - reserved: - - 10.244.7.113 - static: - - 10.244.7.114 - - cloud_properties: - name: random - range: 10.244.7.116/30 - reserved: - - 10.244.7.117 - static: - - 10.244.7.118 - - cloud_properties: - name: random - range: 10.244.7.120/30 - reserved: - - 10.244.7.121 - static: - - 10.244.7.122 - - cloud_properties: - name: random - range: 10.244.7.124/30 - reserved: - - 10.244.7.125 - static: - - 10.244.7.126 - - cloud_properties: - name: random - range: 10.244.7.128/30 - reserved: - - 10.244.7.129 - static: [] - - cloud_properties: - name: random - range: 10.244.7.132/30 - reserved: - - 10.244.7.133 - static: [] - - cloud_properties: - name: random - range: 10.244.7.136/30 - reserved: - - 10.244.7.137 - static: [] - - cloud_properties: - name: random - range: 10.244.7.140/30 - reserved: - - 10.244.7.141 - static: [] - - cloud_properties: - name: random - range: 10.244.7.144/30 - reserved: - - 10.244.7.145 - static: [] - - cloud_properties: - name: random - range: 10.244.7.148/30 - reserved: - - 10.244.7.149 - static: [] - - cloud_properties: - name: random - range: 10.244.7.152/30 - reserved: - - 10.244.7.153 - static: [] - - cloud_properties: - name: random - range: 10.244.7.156/30 - reserved: - - 10.244.7.157 - static: [] - - cloud_properties: - name: random - range: 10.244.7.160/30 - reserved: - - 10.244.7.161 - static: [] - - cloud_properties: - name: random - range: 10.244.7.164/30 - reserved: - - 10.244.7.165 - static: [] - - cloud_properties: - name: random - range: 10.244.7.168/30 - reserved: - - 10.244.7.169 - static: [] - - cloud_properties: - name: random - range: 10.244.7.172/30 - reserved: - - 10.244.7.173 - static: [] - - cloud_properties: - name: random - range: 10.244.7.176/30 - reserved: - - 10.244.7.177 - static: [] - - cloud_properties: - name: random - range: 10.244.7.180/30 - reserved: - - 10.244.7.181 - static: [] - - cloud_properties: - name: random - range: 10.244.7.184/30 - reserved: - - 10.244.7.185 - static: [] - - cloud_properties: - name: random - range: 10.244.7.188/30 - reserved: - - 10.244.7.189 - static: [] - - cloud_properties: - name: random - range: 10.244.7.192/30 - reserved: - - 10.244.7.193 - static: [] - - cloud_properties: - name: random - range: 10.244.7.196/30 - reserved: - - 10.244.7.197 - static: [] - - cloud_properties: - name: random - range: 10.244.7.200/30 - reserved: - - 10.244.7.201 - static: [] - - cloud_properties: - name: random - range: 10.244.7.204/30 - reserved: - - 10.244.7.205 - static: [] - - cloud_properties: - name: random - range: 10.244.7.208/30 - reserved: - - 10.244.7.209 - static: [] - - cloud_properties: - name: random - range: 10.244.7.212/30 - reserved: - - 10.244.7.213 - static: [] - - cloud_properties: - name: random - range: 10.244.7.216/30 - reserved: - - 10.244.7.217 - static: [] - - cloud_properties: - name: random - range: 10.244.7.220/30 - reserved: - - 10.244.7.221 - static: [] - - cloud_properties: - name: random - range: 10.244.7.224/30 - reserved: - - 10.244.7.225 - static: [] - - cloud_properties: - name: random - range: 10.244.7.228/30 - reserved: - - 10.244.7.229 - static: [] - - cloud_properties: - name: random - range: 10.244.7.232/30 - reserved: - - 10.244.7.233 - static: [] - - cloud_properties: - name: random - range: 10.244.7.236/30 - reserved: - - 10.244.7.237 - static: [] - - cloud_properties: - name: random - range: 10.244.7.240/30 - reserved: - - 10.244.7.241 - static: [] - - cloud_properties: - name: random - range: 10.244.7.244/30 - reserved: - - 10.244.7.245 - static: [] - - cloud_properties: - name: random - range: 10.244.7.248/30 - reserved: - - 10.244.7.249 - static: [] - - cloud_properties: - name: random - range: 10.244.7.252/30 - reserved: - - 10.244.7.253 - static: [] - -- name: mysql2 - subnets: - - cloud_properties: - name: random - range: 10.244.8.0/30 - reserved: - - 10.244.8.1 - static: - - 10.244.8.2 - - cloud_properties: - name: random - range: 10.244.8.4/30 - reserved: - - 10.244.8.5 - static: - - 10.244.8.6 - - cloud_properties: - name: random - range: 10.244.8.8/30 - reserved: - - 10.244.8.9 - static: - - 10.244.8.10 - - cloud_properties: - name: random - range: 10.244.8.12/30 - reserved: - - 10.244.8.13 - static: - - 10.244.8.14 - - cloud_properties: - name: random - range: 10.244.8.16/30 - reserved: - - 10.244.8.17 - static: - - 10.244.8.18 - - cloud_properties: - name: random - range: 10.244.8.20/30 - reserved: - - 10.244.8.21 - static: - - 10.244.8.22 - - cloud_properties: - name: random - range: 10.244.8.24/30 - reserved: - - 10.244.8.25 - static: - - 10.244.8.26 - - cloud_properties: - name: random - range: 10.244.8.28/30 - reserved: - - 10.244.8.29 - static: - - 10.244.8.30 - - cloud_properties: - name: random - range: 10.244.8.32/30 - reserved: - - 10.244.8.33 - static: - - 10.244.8.34 - - cloud_properties: - name: random - range: 10.244.8.36/30 - reserved: - - 10.244.8.37 - static: - - 10.244.8.38 - - cloud_properties: - name: random - range: 10.244.8.40/30 - reserved: - - 10.244.8.41 - static: - - 10.244.8.42 - - cloud_properties: - name: random - range: 10.244.8.44/30 - reserved: - - 10.244.8.45 - static: - - 10.244.8.46 - - cloud_properties: - name: random - range: 10.244.8.48/30 - reserved: - - 10.244.8.49 - static: - - 10.244.8.50 - - cloud_properties: - name: random - range: 10.244.8.52/30 - reserved: - - 10.244.8.53 - static: - - 10.244.8.54 - - cloud_properties: - name: random - range: 10.244.8.56/30 - reserved: - - 10.244.8.57 - static: - - 10.244.8.58 - - cloud_properties: - name: random - range: 10.244.8.60/30 - reserved: - - 10.244.8.61 - static: - - 10.244.8.62 - - cloud_properties: - name: random - range: 10.244.8.64/30 - reserved: - - 10.244.8.65 - static: - - 10.244.8.66 - - cloud_properties: - name: random - range: 10.244.8.68/30 - reserved: - - 10.244.8.69 - static: - - 10.244.8.70 - - cloud_properties: - name: random - range: 10.244.8.72/30 - reserved: - - 10.244.8.73 - static: - - 10.244.8.74 - - cloud_properties: - name: random - range: 10.244.8.76/30 - reserved: - - 10.244.8.77 - static: - - 10.244.8.78 - - cloud_properties: - name: random - range: 10.244.8.80/30 - reserved: - - 10.244.8.81 - static: - - 10.244.8.82 - - cloud_properties: - name: random - range: 10.244.8.84/30 - reserved: - - 10.244.8.85 - static: - - 10.244.8.86 - - cloud_properties: - name: random - range: 10.244.8.88/30 - reserved: - - 10.244.8.89 - static: - - 10.244.8.90 - - cloud_properties: - name: random - range: 10.244.8.92/30 - reserved: - - 10.244.8.93 - static: - - 10.244.8.94 - - cloud_properties: - name: random - range: 10.244.8.96/30 - reserved: - - 10.244.8.97 - static: - - 10.244.8.98 - - cloud_properties: - name: random - range: 10.244.8.100/30 - reserved: - - 10.244.8.101 - static: - - 10.244.8.102 - - cloud_properties: - name: random - range: 10.244.8.104/30 - reserved: - - 10.244.8.105 - static: - - 10.244.8.106 - - cloud_properties: - name: random - range: 10.244.8.108/30 - reserved: - - 10.244.8.109 - static: - - 10.244.8.110 - - cloud_properties: - name: random - range: 10.244.8.112/30 - reserved: - - 10.244.8.113 - static: - - 10.244.8.114 - - cloud_properties: - name: random - range: 10.244.8.116/30 - reserved: - - 10.244.8.117 - static: - - 10.244.8.118 - - cloud_properties: - name: random - range: 10.244.8.120/30 - reserved: - - 10.244.8.121 - static: - - 10.244.8.122 - - cloud_properties: - name: random - range: 10.244.8.124/30 - reserved: - - 10.244.8.125 - static: - - 10.244.8.126 - - cloud_properties: - name: random - range: 10.244.8.128/30 - reserved: - - 10.244.8.129 - static: [] - - cloud_properties: - name: random - range: 10.244.8.132/30 - reserved: - - 10.244.8.133 - static: [] - - cloud_properties: - name: random - range: 10.244.8.136/30 - reserved: - - 10.244.8.137 - static: [] - - cloud_properties: - name: random - range: 10.244.8.140/30 - reserved: - - 10.244.8.141 - static: [] - - cloud_properties: - name: random - range: 10.244.8.144/30 - reserved: - - 10.244.8.145 - static: [] - - cloud_properties: - name: random - range: 10.244.8.148/30 - reserved: - - 10.244.8.149 - static: [] - - cloud_properties: - name: random - range: 10.244.8.152/30 - reserved: - - 10.244.8.153 - static: [] - - cloud_properties: - name: random - range: 10.244.8.156/30 - reserved: - - 10.244.8.157 - static: [] - - cloud_properties: - name: random - range: 10.244.8.160/30 - reserved: - - 10.244.8.161 - static: [] - - cloud_properties: - name: random - range: 10.244.8.164/30 - reserved: - - 10.244.8.165 - static: [] - - cloud_properties: - name: random - range: 10.244.8.168/30 - reserved: - - 10.244.8.169 - static: [] - - cloud_properties: - name: random - range: 10.244.8.172/30 - reserved: - - 10.244.8.173 - static: [] - - cloud_properties: - name: random - range: 10.244.8.176/30 - reserved: - - 10.244.8.177 - static: [] - - cloud_properties: - name: random - range: 10.244.8.180/30 - reserved: - - 10.244.8.181 - static: [] - - cloud_properties: - name: random - range: 10.244.8.184/30 - reserved: - - 10.244.8.185 - static: [] - - cloud_properties: - name: random - range: 10.244.8.188/30 - reserved: - - 10.244.8.189 - static: [] - - cloud_properties: - name: random - range: 10.244.8.192/30 - reserved: - - 10.244.8.193 - static: [] - - cloud_properties: - name: random - range: 10.244.8.196/30 - reserved: - - 10.244.8.197 - static: [] - - cloud_properties: - name: random - range: 10.244.8.200/30 - reserved: - - 10.244.8.201 - static: [] - - cloud_properties: - name: random - range: 10.244.8.204/30 - reserved: - - 10.244.8.205 - static: [] - - cloud_properties: - name: random - range: 10.244.8.208/30 - reserved: - - 10.244.8.209 - static: [] - - cloud_properties: - name: random - range: 10.244.8.212/30 - reserved: - - 10.244.8.213 - static: [] - - cloud_properties: - name: random - range: 10.244.8.216/30 - reserved: - - 10.244.8.217 - static: [] - - cloud_properties: - name: random - range: 10.244.8.220/30 - reserved: - - 10.244.8.221 - static: [] - - cloud_properties: - name: random - range: 10.244.8.224/30 - reserved: - - 10.244.8.225 - static: [] - - cloud_properties: - name: random - range: 10.244.8.228/30 - reserved: - - 10.244.8.229 - static: [] - - cloud_properties: - name: random - range: 10.244.8.232/30 - reserved: - - 10.244.8.233 - static: [] - - cloud_properties: - name: random - range: 10.244.8.236/30 - reserved: - - 10.244.8.237 - static: [] - - cloud_properties: - name: random - range: 10.244.8.240/30 - reserved: - - 10.244.8.241 - static: [] - - cloud_properties: - name: random - range: 10.244.8.244/30 - reserved: - - 10.244.8.245 - static: [] - - cloud_properties: - name: random - range: 10.244.8.248/30 - reserved: - - 10.244.8.249 - static: [] - - cloud_properties: - name: random - range: 10.244.8.252/30 - reserved: - - 10.244.8.253 - static: [] - -- name: mysql3 - subnets: - - cloud_properties: - name: random - range: 10.244.9.0/30 - reserved: - - 10.244.9.1 - static: - - 10.244.9.2 - - cloud_properties: - name: random - range: 10.244.9.4/30 - reserved: - - 10.244.9.5 - static: - - 10.244.9.6 - - cloud_properties: - name: random - range: 10.244.9.8/30 - reserved: - - 10.244.9.9 - static: - - 10.244.9.10 - - cloud_properties: - name: random - range: 10.244.9.12/30 - reserved: - - 10.244.9.13 - static: - - 10.244.9.14 - - cloud_properties: - name: random - range: 10.244.9.16/30 - reserved: - - 10.244.9.17 - static: - - 10.244.9.18 - - cloud_properties: - name: random - range: 10.244.9.20/30 - reserved: - - 10.244.9.21 - static: - - 10.244.9.22 - - cloud_properties: - name: random - range: 10.244.9.24/30 - reserved: - - 10.244.9.25 - static: - - 10.244.9.26 - - cloud_properties: - name: random - range: 10.244.9.28/30 - reserved: - - 10.244.9.29 - static: - - 10.244.9.30 - - cloud_properties: - name: random - range: 10.244.9.32/30 - reserved: - - 10.244.9.33 - static: - - 10.244.9.34 - - cloud_properties: - name: random - range: 10.244.9.36/30 - reserved: - - 10.244.9.37 - static: - - 10.244.9.38 - - cloud_properties: - name: random - range: 10.244.9.40/30 - reserved: - - 10.244.9.41 - static: - - 10.244.9.42 - - cloud_properties: - name: random - range: 10.244.9.44/30 - reserved: - - 10.244.9.45 - static: - - 10.244.9.46 - - cloud_properties: - name: random - range: 10.244.9.48/30 - reserved: - - 10.244.9.49 - static: - - 10.244.9.50 - - cloud_properties: - name: random - range: 10.244.9.52/30 - reserved: - - 10.244.9.53 - static: - - 10.244.9.54 - - cloud_properties: - name: random - range: 10.244.9.56/30 - reserved: - - 10.244.9.57 - static: - - 10.244.9.58 - - cloud_properties: - name: random - range: 10.244.9.60/30 - reserved: - - 10.244.9.61 - static: - - 10.244.9.62 - - cloud_properties: - name: random - range: 10.244.9.64/30 - reserved: - - 10.244.9.65 - static: - - 10.244.9.66 - - cloud_properties: - name: random - range: 10.244.9.68/30 - reserved: - - 10.244.9.69 - static: - - 10.244.9.70 - - cloud_properties: - name: random - range: 10.244.9.72/30 - reserved: - - 10.244.9.73 - static: - - 10.244.9.74 - - cloud_properties: - name: random - range: 10.244.9.76/30 - reserved: - - 10.244.9.77 - static: - - 10.244.9.78 - - cloud_properties: - name: random - range: 10.244.9.80/30 - reserved: - - 10.244.9.81 - static: - - 10.244.9.82 - - cloud_properties: - name: random - range: 10.244.9.84/30 - reserved: - - 10.244.9.85 - static: - - 10.244.9.86 - - cloud_properties: - name: random - range: 10.244.9.88/30 - reserved: - - 10.244.9.89 - static: - - 10.244.9.90 - - cloud_properties: - name: random - range: 10.244.9.92/30 - reserved: - - 10.244.9.93 - static: - - 10.244.9.94 - - cloud_properties: - name: random - range: 10.244.9.96/30 - reserved: - - 10.244.9.97 - static: - - 10.244.9.98 - - cloud_properties: - name: random - range: 10.244.9.100/30 - reserved: - - 10.244.9.101 - static: - - 10.244.9.102 - - cloud_properties: - name: random - range: 10.244.9.104/30 - reserved: - - 10.244.9.105 - static: - - 10.244.9.106 - - cloud_properties: - name: random - range: 10.244.9.108/30 - reserved: - - 10.244.9.109 - static: - - 10.244.9.110 - - cloud_properties: - name: random - range: 10.244.9.112/30 - reserved: - - 10.244.9.113 - static: - - 10.244.9.114 - - cloud_properties: - name: random - range: 10.244.9.116/30 - reserved: - - 10.244.9.117 - static: - - 10.244.9.118 - - cloud_properties: - name: random - range: 10.244.9.120/30 - reserved: - - 10.244.9.121 - static: - - 10.244.9.122 - - cloud_properties: - name: random - range: 10.244.9.124/30 - reserved: - - 10.244.9.125 - static: - - 10.244.9.126 - - cloud_properties: - name: random - range: 10.244.9.128/30 - reserved: - - 10.244.9.129 - static: [] - - cloud_properties: - name: random - range: 10.244.9.132/30 - reserved: - - 10.244.9.133 - static: [] - - cloud_properties: - name: random - range: 10.244.9.136/30 - reserved: - - 10.244.9.137 - static: [] - - cloud_properties: - name: random - range: 10.244.9.140/30 - reserved: - - 10.244.9.141 - static: [] - - cloud_properties: - name: random - range: 10.244.9.144/30 - reserved: - - 10.244.9.145 - static: [] - - cloud_properties: - name: random - range: 10.244.9.148/30 - reserved: - - 10.244.9.149 - static: [] - - cloud_properties: - name: random - range: 10.244.9.152/30 - reserved: - - 10.244.9.153 - static: [] - - cloud_properties: - name: random - range: 10.244.9.156/30 - reserved: - - 10.244.9.157 - static: [] - - cloud_properties: - name: random - range: 10.244.9.160/30 - reserved: - - 10.244.9.161 - static: [] - - cloud_properties: - name: random - range: 10.244.9.164/30 - reserved: - - 10.244.9.165 - static: [] - - cloud_properties: - name: random - range: 10.244.9.168/30 - reserved: - - 10.244.9.169 - static: [] - - cloud_properties: - name: random - range: 10.244.9.172/30 - reserved: - - 10.244.9.173 - static: [] - - cloud_properties: - name: random - range: 10.244.9.176/30 - reserved: - - 10.244.9.177 - static: [] - - cloud_properties: - name: random - range: 10.244.9.180/30 - reserved: - - 10.244.9.181 - static: [] - - cloud_properties: - name: random - range: 10.244.9.184/30 - reserved: - - 10.244.9.185 - static: [] - - cloud_properties: - name: random - range: 10.244.9.188/30 - reserved: - - 10.244.9.189 - static: [] - - cloud_properties: - name: random - range: 10.244.9.192/30 - reserved: - - 10.244.9.193 - static: [] - - cloud_properties: - name: random - range: 10.244.9.196/30 - reserved: - - 10.244.9.197 - static: [] - - cloud_properties: - name: random - range: 10.244.9.200/30 - reserved: - - 10.244.9.201 - static: [] - - cloud_properties: - name: random - range: 10.244.9.204/30 - reserved: - - 10.244.9.205 - static: [] - - cloud_properties: - name: random - range: 10.244.9.208/30 - reserved: - - 10.244.9.209 - static: [] - - cloud_properties: - name: random - range: 10.244.9.212/30 - reserved: - - 10.244.9.213 - static: [] - - cloud_properties: - name: random - range: 10.244.9.216/30 - reserved: - - 10.244.9.217 - static: [] - - cloud_properties: - name: random - range: 10.244.9.220/30 - reserved: - - 10.244.9.221 - static: [] - - cloud_properties: - name: random - range: 10.244.9.224/30 - reserved: - - 10.244.9.225 - static: [] - - cloud_properties: - name: random - range: 10.244.9.228/30 - reserved: - - 10.244.9.229 - static: [] - - cloud_properties: - name: random - range: 10.244.9.232/30 - reserved: - - 10.244.9.233 - static: [] - - cloud_properties: - name: random - range: 10.244.9.236/30 - reserved: - - 10.244.9.237 - static: [] - - cloud_properties: - name: random - range: 10.244.9.240/30 - reserved: - - 10.244.9.241 - static: [] - - cloud_properties: - name: random - range: 10.244.9.244/30 - reserved: - - 10.244.9.245 - static: [] - - cloud_properties: - name: random - range: 10.244.9.248/30 - reserved: - - 10.244.9.249 - static: [] - - cloud_properties: - name: random - range: 10.244.9.252/30 - reserved: - - 10.244.9.253 - static: [] - diff --git a/templates/cf-mysql-node3-arbitrator.yml b/templates/cf-mysql-node3-arbitrator.yml deleted file mode 100644 index 7df1f8bf..00000000 --- a/templates/cf-mysql-node3-arbitrator.yml +++ /dev/null @@ -1,6 +0,0 @@ -jobs: -- name: arbitrator_z3 - instances: 1 - -- name: mysql_z3 - instances: 0 diff --git a/templates/cf-mysql-node3-mysql.yml b/templates/cf-mysql-node3-mysql.yml deleted file mode 100644 index 2783a46b..00000000 --- a/templates/cf-mysql-node3-mysql.yml +++ /dev/null @@ -1,6 +0,0 @@ -jobs: -- name: arbitrator_z3 - instances: 0 - -- name: mysql_z3 - instances: 1 diff --git a/templates/cf-mysql-template.yml b/templates/cf-mysql-template.yml deleted file mode 100644 index 71a5ac18..00000000 --- a/templates/cf-mysql-template.yml +++ /dev/null @@ -1,386 +0,0 @@ ---- -meta: - # override this in your stub to set the environment name, - # which is used as a prefix for the deployment name - # - # e.g. `environment: cf-tabasco` makes a deployment named cf-tabasco-mysql - environment: ~ - - syslog_aggregator: ~ - -properties: - app_domains: (( merge )) - domain: (( merge )) - nats: - user: (( merge )) - password: (( merge )) - port: (( merge )) - machines: (( merge )) - cf: - api_url: (( "https://api." .properties.domain )) - admin_username: (( merge )) - admin_password: (( merge )) - apps_domain: (( .properties.app_domains.[0] )) - skip_ssl_validation: (( merge || nil )) - standalone: (( merge || false )) - default_mysql_cluster_ips: - - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] || nil )) - - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) - - (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) - default_proxy_ips: - - (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) - - (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) - temp: - # spiff does not allow concating multiple fields of different types - # without adding new temporary properties - database_startup_timeout_str: (( "" jobs.mysql_z1.properties.database_startup_timeout )) - canary_end_in_sec: (( .properties.temp.database_startup_timeout_str "000" )) - -name: (( meta.environment "-mysql" )) -director_uuid: (( merge )) - -releases: -- name: cf-mysql - version: (( merge || "latest" )) - -compilation: - workers: 4 - network: compilation - reuse_compilation_vms: true - cloud_properties: (( merge )) - -update: - canaries: 1 - canary_watch_time: (( "30000-" .properties.temp.canary_end_in_sec )) - update_watch_time: (( "30000-" .properties.temp.canary_end_in_sec )) - max_in_flight: 1 - serial: true - -networks: (( merge )) - -resource_pools: -- name: mysql_z1 - network: mysql1 - stemcell: (( merge )) - cloud_properties: (( merge )) -- name: mysql_z2 - network: mysql2 - stemcell: (( merge )) - cloud_properties: (( merge )) -- name: mysql_z3 - network: mysql3 - stemcell: (( merge )) - cloud_properties: (( merge )) -- name: proxy_z1 - network: mysql1 - stemcell: (( merge || resource_pools.mysql_z1.stemcell )) - cloud_properties: (( merge || resource_pools.mysql_z1.cloud_properties )) -- name: proxy_z2 - network: mysql2 - stemcell: (( merge || resource_pools.mysql_z2.stemcell )) - cloud_properties: (( merge || resource_pools.mysql_z2.cloud_properties )) -- name: cf-mysql-broker_z1 - network: mysql1 - stemcell: (( merge || resource_pools.mysql_z1.stemcell )) - cloud_properties: (( merge || resource_pools.mysql_z1.cloud_properties )) -- name: cf-mysql-broker_z2 - network: mysql2 - stemcell: (( merge || resource_pools.mysql_z2.stemcell )) - cloud_properties: (( merge || resource_pools.mysql_z2.cloud_properties )) -- name: errands_z1 - network: mysql1 - stemcell: (( merge || resource_pools.mysql_z1.stemcell )) - cloud_properties: (( merge || resource_pools.mysql_z1.cloud_properties )) - -disk_pools: -- name: mysql-persistent-disk - disk_size: (( merge || 100000 )) - cloud_properties: - <<: (( merge || nil )) - -jobs: -- name: mysql_z1 - release: cf-mysql - template: mysql - instances: (( merge || 1 )) - resource_pool: mysql_z1 - persistent_disk_pool: (( disk_pools.[0].name )) - networks: - - name: mysql1 - static_ips: (( static_ips(0) )) - properties: - <<: (( merge || nil )) - admin_password: (( merge )) - cluster_ips: (( merge || .properties.default_mysql_cluster_ips )) - seeded_databases: ~ - database_startup_timeout: (( merge || 600 )) - max_heap_table_size: (( merge || nil )) - tmp_table_size: (( merge || nil )) - wsrep_max_ws_rows: (( merge || nil )) - wsrep_max_ws_size: (( merge || nil )) - syslog_aggregator: (( meta.syslog_aggregator )) - network_name: mysql1 - skip_name_resolve: (( merge || nil )) - bootstrap_endpoint: - username: (( merge )) - password: (( merge )) - -- name: mysql_z2 - release: cf-mysql - template: mysql - instances: (( merge || 1 )) - resource_pool: mysql_z2 - persistent_disk_pool: (( disk_pools.[0].name )) - networks: - - name: mysql2 - static_ips: (( merge || static_ips(0) )) - properties: - <<: (( merge || nil )) - admin_password: (( jobs.mysql_z1.properties.admin_password )) - cluster_ips: (( jobs.mysql_z1.properties.cluster_ips )) - seeded_databases: ~ - database_startup_timeout: (( jobs.mysql_z1.properties.database_startup_timeout )) - max_heap_table_size: (( jobs.mysql_z1.properties.max_heap_table_size )) - tmp_table_size: (( jobs.mysql_z1.properties.tmp_table_size )) - wsrep_max_ws_rows: (( jobs.mysql_z1.properties.wsrep_max_ws_rows )) - wsrep_max_ws_size: (( jobs.mysql_z1.properties.wsrep_max_ws_size )) - syslog_aggregator: (( meta.syslog_aggregator )) - network_name: mysql2 - skip_name_resolve: (( jobs.mysql_z1.properties.skip_name_resolve )) - bootstrap_endpoint: (( jobs.mysql_z1.properties.bootstrap_endpoint )) - -- name: mysql_z3 - release: cf-mysql - template: mysql - instances: (( merge || 0 )) - resource_pool: mysql_z3 - persistent_disk_pool: (( disk_pools.[0].name )) - networks: - - name: mysql3 - static_ips: (( merge || static_ips(0) )) - properties: - <<: (( merge || nil )) - admin_password: (( jobs.mysql_z1.properties.admin_password )) - cluster_ips: (( jobs.mysql_z1.properties.cluster_ips )) - seeded_databases: ~ - database_startup_timeout: (( jobs.mysql_z1.properties.database_startup_timeout )) - max_heap_table_size: (( jobs.mysql_z1.properties.max_heap_table_size )) - tmp_table_size: (( jobs.mysql_z1.properties.tmp_table_size )) - wsrep_max_ws_rows: (( jobs.mysql_z1.properties.wsrep_max_ws_rows )) - wsrep_max_ws_size: (( jobs.mysql_z1.properties.wsrep_max_ws_size )) - syslog_aggregator: (( meta.syslog_aggregator )) - network_name: mysql3 - skip_name_resolve: (( jobs.mysql_z1.properties.skip_name_resolve )) - bootstrap_endpoint: (( jobs.mysql_z1.properties.bootstrap_endpoint )) - -- name: arbitrator_z3 - release: cf-mysql - template: arbitrator - instances: (( merge || 1 )) - resource_pool: mysql_z3 - persistent_disk_pool: (( merge || disk_pools.[0].name )) - networks: - - name: mysql3 - static_ips: (( merge || static_ips(1) )) - properties: - <<: (( merge || nil )) - admin_password: (( merge || jobs.mysql_z1.properties.admin_password )) - cluster_ips: (( merge || jobs.mysql_z1.properties.cluster_ips )) - network_name: mysql3 - bootstrap_endpoint: (( merge || jobs.mysql_z1.properties.bootstrap_endpoint )) - -- name: proxy_z1 - release: cf-mysql - template: proxy - instances: (( merge || 1 )) - resource_pool: proxy_z1 - networks: - - name: mysql1 - static_ips: (( merge || static_ips(1) )) - properties: - <<: (( merge || nil )) - proxy: - <<: (( merge || nil )) - health_port: (( merge || nil )) - api_force_https: (( merge || nil )) - api_username: (( merge )) - api_password: (( merge )) - proxy_ips: (( merge || .properties.default_proxy_ips )) - cluster_ips: (( jobs.mysql_z1.properties.cluster_ips )) - network_name: mysql1 - external_host: (( "p-mysql." .properties.domain )) - nats: (( .properties.nats )) - syslog_aggregator: (( meta.syslog_aggregator )) - standalone: (( .properties.standalone )) - arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) - -- name: proxy_z2 - release: cf-mysql - template: proxy - instances: (( merge || 1 )) - resource_pool: proxy_z2 - networks: - - name: mysql2 - static_ips: (( merge || static_ips(1) )) - properties: - <<: (( merge || nil )) - proxy: - <<: (( merge || nil )) - health_port: (( jobs.proxy_z1.properties.proxy.health_port )) - api_force_https: (( jobs.proxy_z1.properties.proxy.api_force_https )) - api_username: (( jobs.proxy_z1.properties.proxy.api_username )) - api_password: (( jobs.proxy_z1.properties.proxy.api_password )) - proxy_ips: (( jobs.proxy_z1.properties.proxy.proxy_ips )) - cluster_ips: (( jobs.mysql_z1.properties.cluster_ips )) - network_name: mysql2 - external_host: (( "p-mysql." .properties.domain )) - nats: (( .properties.nats )) - syslog_aggregator: (( meta.syslog_aggregator )) - standalone: (( .properties.standalone )) - arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) - -- name: cf-mysql-broker_z1 - release: cf-mysql - template: cf-mysql-broker - instances: (( merge || 1 )) - resource_pool: cf-mysql-broker_z1 - networks: - - name: mysql1 - properties: - <<: (( merge || nil )) - max_user_connections_default: (( merge || 40 )) - syslog_aggregator: (( meta.syslog_aggregator )) - network_name: mysql1 - ssl_enabled: (( merge || nil )) - skip_ssl_validation: (( .properties.cf.skip_ssl_validation )) - auth_username: (( merge )) - auth_password: (( merge )) - cookie_secret: (( merge )) - external_host: (( merge || "p-mysql." .properties.domain )) - cc_api_uri: (( .properties.cf.api_url )) - nats: (( .properties.nats )) - networks: - broker_network: mysql1 - services: - - name: p-mysql - plan_updateable: (( merge || true )) - id: 44b26033-1f54-4087-b7bc-da9652c2a539 - description: MySQL databases on demand - tags: - - mysql - metadata: - displayName: "MySQL for Pivotal Cloud Foundry" - imageUrl: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAJo5JREFUeNrsXQtsVNeZvvPwzBiD3zyNwxgWhEKqmEibR9skdhQtaao2Rqtks4+qthKp1e5KxKhVCg3BbrKOst3lIW2rrUQVR5W2adgKp92ikG1ik2RbkpWCs4WWQomHVyAY8NhgPDOex97v+F4zHs97zrlz7z3/Jw1j/Ji5c+/9vv9x/v8/DoVgK9Rt7/arT37tv7XqozXT7zorK1clotFgYnp6PMtLDquPoPZ1YKxvd4DOsn3goFNgOYK3asRuUx81SQRvM/hQhpIEYlz7f1AViGG6SiQABH6WHMS+U/u61SKHDxGAp/CxJgzDqjAE6aqSABAyE75NI/iD2rPfZh8xoAnDYdfCquEr218coqtOAiC7K9+hEb5N0tMw5PBUHE5EpgcodCABsDvha5MI36HF8YRbQIgw4PR5D8dD4QEKGUgA7BLHg+yPSWzli7sx3e4hxeV8w11bO3B5y7YAnRESAKtZ+i2KdZJ2pobT6x2Oh8N74SGQZ0ACYFbig/Rf18hPEHGzulzqP44BR0XFq1d39A3QGSEBMIOLD0vfSTG94WIQdHi9/a5FC/dSiEACYCgaXtjeEZ+aImtvnnzBQCIa3auGB0N0NkgARFp8WPqdiv3W6O1xM3s9gUQ40qsKQT+dDRIAXqSHa/+M5uqTm2+R8MBZ6dvrXe3fc+HJpylpSAJAxCchICEgASDikxCQEJAAZCF/DxHf/kJwZfuLPXQ2SACSid+pUHJPnpve6wk4fb7eK8/29pMAyE38No34bUQLCW9+t3vI3VDfe3nLtiESAPni/N3KTAEPQe6wQHF4PP2+tau7ZcwPOCQkf6dGforzCXPzAwurumULCxwSER/x/Ssyufuwbs5K35zvOauqsv5NfHJy7v+nQkoiFpOGEE6fd8hVU9MlS3mxQxLy9yg2zO67Fqpkdjox3FMlu3OG7M75pOcFXRxmRCGuPk8pielp9n+bIeiuq907+u2dPSQA1rf6BxSLt+UyS161gBEd5Hb6fGrcWmGqY2SioIkBhMIOnoOWJLS1N+CwMflRzLPTilYf5Haprroak7Jns5G9EFGIh9THjUklpopCIjJtydyAq66md3Trjj0kANYgfq0W63dY6CZTXNWLLE/4XIAAxCYmlBgEYeK6xXIDvgHfujVddlspcNiM/G2ay296q8+sfHW14q5ZlDMxZ1dABGLjE+zZCuGCw+NBbmCzneoGHDYif4/m8pvb0tfVKupNJCxRR2JgiLfWa5cEocMGxDe9yw/33l1fx54JuREdC86KgWmv6aKFA96WVZYPCRwWJ3+rRn7TZfnh4rvr6hR3Y/3MLDtCUTkDiEF0bMyUCUSH2zXsaVrRdekbW4dJAIwnf4dGflPF+1ib1918Al+vIHrlqulqDpAXUK951+i3nh8gATCO/Fji222mYwLh3fW10ib0jAJqDKY/G2UrCabKC9TVdFtxqdBhQfLD6neaifgVS5fYdunOzOHB9GeXmWdgmnuhvq5f9QS6SADEEN9UyT4ivnmEIHL+gmk8AqvVCzgsRP5BxQTJPsT4FUsXk6tPoUFmEahaMOxb09JuBRFwEPnzPFGqpfc2NxHxTQ4sHUY+vVj2VQOIQHzyZrvZtzJzmJz8ID0q+/xlO0Eul+JubGBWn2AdwBvAqkE5i4rUeyfgWblis5mXCR0mJz8sf9nW01C441mxnOJ8K+cHVG+gnAVFWCb0LF/ablYRcBD507v7ID5V7tkoLDh3oWzegJlFwEHknwvd3afqPZt5Ayr59bCARMCkAlBO8oPwnuYmsvrkDUglAg4TkR+kP6qUIeHHYn2V/GT15fEGIAJlyg2A/KZZHXCaiPyD5SC/Z8Uyxeu/jcgvEXCtcc1x7cuAVmfVgsGm1/aZolnEYSLyG7rOz9b1V91GffmSA81F4TNnDa8bMEuxkBk8AMPbeVHN51u7hshPYPcA7gWjcz/xyZutoZOnXyn35y+r36s19jxp5Hsiy++9baXicDrp7ifMeIPqveCurWEOceq+CEJzEdHo+kWPPeq/+dbgG9IJgNbS+x1D4/3mJqViSSPd8YSMnqHD44F1VtmZMCoEaV20+cvjNw8NHpEmB6AN8zhg2IdE0me1n1x+Qv55gU8Chi0VsnkCNdWbyzFUxFEG8hu61g/Se1Y2EfkJhbnnkWmWHDRqAlG5agScBpNf7+k3jPxk+QnFEbLC0HsnEYnURi58+orRy4NGZ8IMy/jr5Kf1fUKpoSPbg9EIEYjGWsMjZwxdGTCMHdrc/m8a8V6Y1uNBpp/ITyhVBLBCoN5PCAmwzZkBnsD6RV/9kuPmfw8N2SYHoO3YM2gY+Zub6M4lcAfKh42YQajNoGg3YgcihwHkR0wzYkTcT+Qn2EYEPJ5g5fq1LaIrBY3IARiS9NMbeggEkcA9ZsSeD0gKGlEpKDRI1op9nhH9IVjCDw09VN1HMACummpWLCS6f4BVCgouEnIIJL9fmWnvrRVOfsr2F4zmqoXKyqrs9e8TkYhyPHiVTlY6csZirFhIdJ2Ael8H3Y0NGy9v2RYQEjYLPPYDRH7jUV3hUTbUNcwSfENtPftetUf9fm1DSa99fvKGcm5ypof+t6OXVIEIqwJxTf3+dfX7N6Q6z/oSoWgRUIWmNnr1GkKBdst4AEZs1U3lvTO4b8nymcfiZSrpF6mkX1iW45iYVr2FsatMGNjz5Yvse7b3BNQwIHTqtPCyYXddrZAtyR0CyG+I6y9rOy8s+6amVYz0eDYzED4cunBWOXT+jK1DCYN6B4IVS5dwDwVEhADCs/7IxMpG/ida1iqP+9cy4lsFCDnw2LphIwsf9p08prw+csp2ngHuxYoVy9gSoUDUxsbHuYcCXD0A1fp3agIgzhVqbCjXKKeyEX/rhrvK5tqLCBX2nTyu7PvjMdsJAaYOY8NSkXDVVHddeba333QCYETBD2qyEffL4urvuvsBS1n8QgCPoPvDd1muwE4IB84KHTaKVYHKDeu5FQjxXDjfLZL8bLOOVc1SkB9u86FNm21LfgAezf72R5mHYyew6dICd5LCqkDo1Ce7uYUvnKx/m/rUKfLEYoCnLMt9P/7iw2zpTgbAy2m2SXijWWh2r4pEIhLpXLL3pTYzeQBCl/wQ88uS9INFXGkjQuQD5DjsBDaERmCeCqsN0avXdppCALTEX5uoD4u4H4k/WZCrOs+u4YDdgHtW5KThRDTa1vjyzs6yC4BI68+265Ik7ifYD6J3m4qHQjvLKgBaxZ/fqieQQBCdDxDZoZoIR/yNfc/1lEUAtGW/LcJcf9V9oo06CVYH7mGRIWx8KrSllDmCpXgAaPMVsuwnWjkJBCMhcrt5LAuGPwk8Y6gAiLb+5PoTKBQwxgso1gMQZv2R9ZfZ9UcnnXSfOXhNilBA1H1dihdQsACItP5MKVfK7fofunDGduWx2YB+gF3HPpLis3pWLBfm2RbrBRTjAQiz/kiWiCyjtAqeev/XUkziAfkff+egFHMDmIFT721RCcFivYBiBGCLqJODZAnhFjHQNWdXwMvZdOiAdCPHWEJQkJGDF1BwaFKg+9+pCNrO29u8UnF6vcR+DeF4TBm6dJ5N2EGtfLNNKgTRBdgzfETpOfqBNJZ/ntX1eJRYcFyAG5DwVd5/35nQe0fy3l+w0IEgQqr+ZE/85bKUj6sPiMBT6+5gvQJWbBTaP3JKeT1wSqr8Rsb7HQlB9Z6P3Zjk70l7PeBof96/X4D1F7alt2+NX3FWVRHb84Q+Duy+JctKHvQp0tKD7G9qSU1ZrX1Gd31yUgmdDojxMCorN1/d0ZfXVuOFeABfF2X9ify3AAs/Homw1YBsXoFuSeENQBAwCRiDQfFcDg8Bx4PlvN8HZwaC5jslGAK2aeUqaVYCZkmq3vOivIDE9DS4mpcA5OUBaIM+R4RYf0mHe2bC1jvuYjP0QKJdx48W7TJDFPQR4ez/i5fN+VkhQKIOewTMfH2NWXN9FHixI8ER0qAN+HFV8FiYM3hQumuNicJTJ04Kee2KpUta8hkgmq8HICTzjy2WiPyZCbxffYAc+9XYGcM0C7XIQDZPolyf6wn/WkZ82cGWBVUOiNhrMHb9BjjbzUsAOoUIgER9/qUQBo+ejfey8dqIqc1G6nzcfBD+kaZV0g07ycNSCxGARDjcyUUAtOQf98IfFvuT9c8bcOdBIt1ysjzA6KU5+QAzER4JSl28ZBlvVqwXICIXgMKghhe2d+RKBubjAQhJ/lHRDx/PQNmwcTZOR+YdMTr6CRCnixYGEB1bjuE4mhcsZNbdzoNMxXkBi8uWDHTnsP6w/B28DwyWnzL//MnIMuppdgtKTuLBayhKcEpIIhJy8KFqxhvmvsdgItEBDo/17Q4W6wF0iPjAFPtnBjbcFCEOZiYv1QjMcIL3zkLaVmXgcH9G8cnxGtyz/3rmk5Aedtw6Kxfs3POQtwConBDRI+D0erNy2JnF/ferT638P2gdsTyHNew5ekSaz4sSYSoPFseNeDjcumTvS/5iPAAx7j9Z/7y8ALQE290TQKETtgcjiOVGNBjsKEYAHuN9IGiCoH7//IC1/nt/+TNmIe2GmVbgAenKf/MJj4U0xcXiGbnsyOD+Q4rGeB8H5qKRB1A49LJZ1MxbeU0dooZ4n1z+LNZ6LChqm/G6dKsBbqPcf4xCIvIXh3PaTrrVRz3KI6oIYKkv3XKfGYElSHgxIH8xPQMyhgHTn17SM/jc4PR5064GZBKAB0W4/4T8rH0moiAngPyA3hegFwNhjd4sy3vHWTfgrQrFfPIY2T6zjABXeJcHx0PhBwsRAO4egKummq5sHni8ZR1rloHFz+Uqp5YBY70f1XjoANxQWz/bDSgibNDfF4VFekdgoa49iI/dgdnnlrAbMBtXBPQHgNNdOXMAavyPpb+jvN3/yg3r6crmAb0dWCdZMZ2A6ZDcGjwjFqpAeLx5Ex0otvU3FakdgbK2A2fD1PET3MMAFRvH+nYP5/IA+Ft/cv+LJgoeSACiCxCxdLFDNFN7A4xOxMHaI28B0pt1ipHdwwCHpwLczikA/ON/cv9LAtz6p9dtYA80/Ohjtsw+aksXsE1NtxHpTRAGJCLTD+aTA2jj7f6TByBGDAB4BL8fu8ae0QlYriU21oxUV6/cXjuTf6CGIVN6zW1ZBUCN/9t4v6OzagFdTdHEw8ANZe0cdx8twfAWzt28Mc/l19uFC7HkqbkDllNQv8a4chryIU4EYhPXub5mY99zbVe2vziUyQPgXvtP7r/x0AeFzlWKjXRirCYAGBTCWQBiNybB8VkBcAqP/6nvvyDIWCVX7IwC+3sAQoznHI47RXoAqG2m2v/CBcBqM/9KAcKUfX88RhfeOP60phUArf3XT9a//Oj+4F0pRACJSxm6Hk3GIb/W6zPPA/DzfifnQhKAYgBCgBiYC2BXcqApCBugyrY5qEk41JpOANoo/jcfSfSWYLsIATwbtALbWdy4CoBPyOTsWa4nrwLcyTV+cbko/ufkDSR3Aj61boPlimoQ52Nj0P0jJ6npp1ABqPQxLnEuC74znQD4eR84ga8Q6J2Aelmtmefu6/sWYDMTcvNL5xLnseH+dALAdQWAxn4XB73CL9ugTFhR/Fz/ndQqPKM3CAXZZ/YkKLwaUW8MotFg2bnEWQBa5wiA1gHIXbUIhQNVdugGfHrdHcqu4x/l1Qk4QzxY2VPzyJX8rG/eke530pE61QMBudn7aZWEhVYUpr4vPud92v6HBGO5BM6jM1D3ALiP6hGUvJAGK7VeeXQCFhs/z/bsm4Rg8EqsmscoJxwVQnJptckhQBv3g6YEIDchgKXEw4rjtXTSW2mMmQwegMb5IV0Aani+sovW/4WAxfobG9hOwee1CTy/Hb2ouuLXTJNoQ4LydrY56HK2QShZen4iwHnrsJpkD4BvDsDppCtmgGeQvFuw7urrHYA84vRcMbz+rHcGGp18lC4M4CsArckCwFmtKumKlQHZevB1MZjzfy2pl9GapyQNqfW3nB5AJffOQGE5AIeLPIBigbl7omLxVIEwS0x+noqDysEpxnkhTKUlwOLx5vkz0pXIYpWDUB5OObUuQM6v6qIrViRAfjTJyCICW/MYf04QA3AfHgB3ASAPoDQgo2/3Tjm94/H1EbL+eXFKTGWtn4J1E4sA20Dz+FHbeQOoZUCXo0yDT8wKJAG5VgGic4nAD9hBFxNznmhZy8qDrZqF15uZfnzyGHUElsAtzl2BtRAAvk1A5P4LIY/e/MMmAKti8EjTKtOLAY4b3YDYx4CsPR9u8W4KctNptV5ocPzoVTZQA1V3M5uDLldur6sve9Wd3hE4s2nJJWoDtkgIQLAo4EqfG5m7d2DyBqF6IQ/PAp7k6kIUEhW7MSiBBIAgykNQH5lcbngNK6sK23VmIhIha04CkGecQoNAzO81UBLOmjkA/oNBFFoGJBBkFhU6BQSC3AJAUxoIBDlRQx4AgSC5B0AVGgSCnBgnD4BAkNwDIBAIJAB8EJ+cpLNKIAiACG6RB0AgkAdAIBBkBPUCWAzYOxDbh6UiWzMO1fITsgnAMNc4he/sckISdt/9wJx9AOZgw8aiX5ftJZBlGrG+10A6YGAJwaAcAH9usb0BgzxfkfPEEoIyM9J7/0OPCuv3X1m1sOh2YaO2/27Wjk/mRiYB3ApSCCA5+UsFPBIMKBEJjEPDRqm6N4IJQwh5aMpQ6XBoY8FHeL6ob42f2oIlIL9OyHv/62fCXh/7ICLvkQkQAgxOtftAEiwBhk4HeL9si3Osb3eAqEbkLyV8EHGM+Pw//uLDWckPYCQakpyEwgHuC1kGpERgaQCh3tq02TI76/IeTorPDfHLZ+syWH8ZVjhEcUrPAQwpHPcHTMTixOISb36r7LKLEIBnLA6LDsufz+cH8WVZhRDAqaFkAeCsVlPEZAnID/Dc1y852ZcLGEr69Pu/lubeEFVirwvAMFcPYHqa2CwB+QFsWsIDWWsc0rn+quWXaUlQwBLgcLIAjFMOoHyA5UO220rkZ7P/Ry+VvG1ZMclOvDc2SZEJAjg1npoD2Mn7gGmXIL5ur2hgzz7dpTdiWQ2kR7xfSBIRgoMdhYn8fHMAQd6vzsIAEgDLkJ+RfvSiYevpyPDvvueBgr0e2Vx/gSE14zxbBhzr2z3M+9UpDLAW+QGjiIW1/Xwz/cnAaoNsrr8oLumcd6ckBbhtFDqTtVxMTE+DrXfcpWwtoXlHFHo23qPt73ctZwyeDfrWYeni/d6N9xaU7Et2/bs/eFfK+0XACsCswU8WgABXASAPIC0KzXYbCcTkeOQswCmDeIH8pSYcyQOYw3VlNgTQ8DHXuCUWIxGwEPnNjEMSby8ODglYAvw4nQAMcT/4EAkAkb80yOz6C+TQ0DwBqFi6JMD94G/QgFDEvUR+cv1NxqHheQJwecu2QHJswAMxySYEI7mX2r2G2nYif3FAxl/2nn8BHAqM9e2eXfZ3p1EGP7c8QGSaPRyeCqlcfFis10dOzcavKLAhESgMWI2QfdyYzh9R1j81BwAc5q5gExO2d/EPbeqYQ3Cs7yeXtnZ/+K7tB1Zwd/0/lNv1F8idwxkFwLWwintBUMzGeQC9gSddHXvq9596/9c0mbcA158EUxh3MnsAV7a/OMRfxa7b8uIgts/WxALPYNc9989Wu+ntq7JbNXL9y8sdNf4fyhYCACQCOYAy3v3tuVt3dQ9BB0ptH3/nIIlAFjxFIimSM/O4PU8AHJ4K/nmAcfvkAdC2W0gNP0Rgd9LvIwzoOXqEmJ4Gsoz3KiNnDucUgERkeoA8gPTAEl+uIZXpgARh8t9hhYBEYC5kGu9VRs7M47Yr9Ruh945cqrz/vmfUL/n18iYSirOyUnF6vZa+KH+aGFfalq9UlvgWFPy3+LvzN2812nx0dVRprlqkbKhroLtdxdcOv6WMhmiUnE7+6Bj3Dv2gGv935/QAMikFhQHaMIoP3is6RkX4kLo8SC4vuf4GcSUtp9MOBXX6vIfjoXAnhQHpXVUk8rD2Xyj08VebDh2Y7b3Ha1lh/n+yCKLdt7RzeG2OiJLrbwhX0ub2HOm+Wbe9u1Z9GuN9BJ7mJsVdV2uLi1TKQA9dRHQSFDsQFHmEXL37qTg/eV3q/fXMDrj+kXMXRLx0XXIJcFYBAOqf//ZgIhpt43kErupFitd/m20uVimDPVDo8vjgwdn/QwTy9Sr0DjnaG89+CAfOivAAhlTyt6f19jOz1fmGCNdGQG1z+WJX1XXdP1LcXHwUEqUuD+Yz7BLkZyEIkd92ADcEuf8ZuZxRANy1tQMijkRAdrOsKCWRl255MNvMO7zPvb/8GSXMbOz+C0JGLrsy/WDyzbeDVQ+3dSRisWVcVS4UUiqWNNrqwv3i7CclLQ8ijj99fWZrhqFL59MuD8LiU5WcvRE5c44tmXPGsOr+v1ywBwDEw+G93N2cWMx2XkCpy4MYj528CrCTJfduWXmEGUR++1t/AaO/gKwczrU78IDD5eL/Ya/Yz4UFYZ/Kc686NLwgCXhey8bry4PJjUOI8/Fz5AW6P3yXGGJ3ARDHiayhfFZ2h947Eqps+0KrEo+v5+oFRKNoPVYcHo+tLiIIi2q/bFN12XCQwYPK/sBMvA+yIwzwqkKLZ4QT4XiMPfDzQpf5CNYDxn5PX74ihPyq+/9qKR6A4qioeFXEkU1/NmrLi5ktkQcvIdWaJ/e+s8ahex4gRkgGgVzIyd2cAnB1Rx/CAO5BO4Yd2HVsOAp00i0PHrpwNu3vY5NNHWzLrLtJBKSx/ioHBA3+QO3/QMkCwLwAr7ffYnFP2ZGayNNj/XRI/T6WB5+gGYIU+5eGvDiblwC4Fi3cK+TDI/Npo8KgZCQn8nSA1Kk1/yB/OrKjzBjFQgR7W3+BK2J5cTavFD9qAha03889GQigLsAu/QGpQCLvt5cvKY+tWs2SfHjga7S9Vnu8yudVgv/wvoeUxb7KeX/LtupWH3gNgj0ROXdelAFE8u9H+fyiO99XRDIwEY128D5SlguYnFScVVW2vMgs8ffBu2w3XN3i52oiQmKQBobY3Pqr97zAgbl5e+x5L/JPDb53YsHDD3YqsRh3c439z+3qBQCo8su1PKgD6/4/+MP/EUPsbv3PfyrK+gfSDf4oKQcwS9RwpFfEEUMJ7To9WAfc+WyNQyxnMHhwdkMRgn2Be12g9S+Io45CX71+x7fGEgK8AOweVLl+ne0vPkKBVE8AiULaN0AeTJ04Kcr6Y+mvrpA/cBb6Ds5Kn5AVAZwQuxYHJQP5gGSi4+u/OHSAyC8JcI8LXPkqmJsFC4B3tX+PiMIgAGuidl0WnOPqa9OAEBJsOjRATT6SAPe2wHV/cHJPoX9UcKfP9f/8RajqkYcq1Q/Txv8MJdSTFFHctTW2vhGwtIea/9cDFO/LBIz6ioeEVb++rLr/bwr3AER7ASxBYvOEIEBz+eSC4Pu6KOtftABcePLpoKhcgK6UgnqjCQTjXX/1XhY06HM29k838FOYAIj2Agw4YQSCoa6/QINWtPUvSQBEewGyhAIEcv3LZf0BR6nvXt/77EgiHPGL+GSYRuRbv1YRMZWIQDDC9Q+dOCXS+qPqr6WUF3CWegROn69X5AmkUIBArn9GlMw9B4+jELGJSDI8K5Yp7kbaRJNgHWC9P/LpJZFvkXGzD0M9AMDdUN8r0k3HibTr9CCC/YB7VTD5uVh/bgJwecu2IYfH0y/y04bPnKWlQYIl4n7cq4LRr1r/IdMIAOBbu7pb1LIgO7GR6ZmNEwgEM8f96j0quJwdHOvm9WLcBIAtCy6s6hb5ydFCaYBrRSAUHaoKbPPV0V3Ksl8qHLyPruF73xmMh8JtIs+AnbYZJ9gDArf1TgaXxF8y3LyP0FVT0xUPXT6qfimMoTjRTp8Prcl056UBxk3diknjRSdQMaPBqT5m3UWbjm0r+Xwj6See/LD6Xbxf1CHiSBd/v7dHVcSdIs8GVh28q/3SiQDiy8R0hN10M+SeUu9A9TkSMayVekYYPKoiONXzX6leCye7Do4KD/uZbOQPfxIwIkHdq1r/HksIACC6NoBZJPWmgwjYtVIQlhw3GCP79LQR8SUfLxDbvlVUsOvDHjb1HFjGXyW/AUvU3F1/YSHA7As31HdFr1w9KmJ8WKr62kEEGNFD6gNTkkMhS9c9zArV2FyxZmGbKg52CN8MJL8Q11+4B8BCgV0vPBO9cm236DNkRU8A7npsYkIbi35TuhoHXCtn1QLmLbiqqy0VOhhIfgBZ/z2WFACg4XvbDqgWrUP0+2BVAKsDpraM6AwbV0mvuvZ2H31WTF7BpYYKrppqVRAWmfpYkfATuKNPMrDBx2ahvBH9CXzr1nRNnTjVlohEhK7b6RfETCIASzFLemptzukRRSPB2esIEdDFwEyenYHkF+r6G+YBAEv2vtQWvXJ10Ag3t9yeAJGeP8wiBgaSH2jnVe6b9dwa8Ukm33w7sPArmxxqKNAm+r2QQIM1QbLJ4XQadnMgY4+Rz9MXLiqx4Dg2USHm8hJV9VwyUb02pn4dVhxulxoyeAwVdVxXA8mPJb9+Q0IvIy9k40s7DsSu3+gw4r2MSAwya6/eFNMSjDM3Y86gorFBcaken+hrbGDCz5C4fw5PjLxo3pZVXap6DxtikQUWaMDawx2cOn6C1X8T+cuTM8C5Z9cA47YnJ+1A/mEj4v6yeQDAsh/tao1c/GxQdFIw2RPwrGzisu4MFxAW3yoFOdLlCrCkqHoEPPpEWHnv+QtGkj+oxf3DthYAYPG/fK8jNj5xwKi171LLhkH86c8uk6W3UniwdEnRQmBgeW8yNqvkHzBcNMtxgW6+NXhiYceXxuM3px4xxl9MKNFrYyxxlK8I4OJHR68yKwCrr8TixCyrQL1WSBpGtevG+hTyTAjjb8IjZ9g9YyC6jUr6mcIDSPIEXlGJ2Wnke2K2IGYMZr0Jrlyd2cTRYtV5n1vepNyxfIVyW139vJ/97uIF5djFT5WzY9fk8whUD7Bi6eKccyWRUxC4d18mYLpPV9nOTbkvjlGVgqmxomdV87zssRVdfZD+m59/QHn09juUGl9lzt+HAPzPyGnlpx/9r/L+yJ/m/fyLLX+m/OLpv+dybPXf3WqJ0IBNnz5zrhy5HUMz/mkNYrkvCioFQ6dH/PHJm62GeYjqhQ6dOq14V93G3ENkkMMY4Wwh4sPK9z3awYhf6N/h8dd3/TkTgn/4+U+l8QrYyoF6nSHy3uYm1qXI4n3MmzT+2hue8TelAGCUWNNr+9pVERg0UgRwwSECTAAs1nkHK/2Tv+vKy+JnA8IFGUMCdu1PB8p57UH+dp6jvYqF0wwXBCKgkr9dOzGGwmrkh+WGi14q+YGDvz+myAzZyW8KD0AHTsiyH+3qMrJGwGpAvN/35czpkvHQFCO1nvDT8YXVa5TPLWtiz8nC8e+/eZdOqrFgDT5mIb+pBAC49I2tw6oItJMIpMe//eWTGS0/yPzy24eYCKRCT/bhb5EzePahTez/EIp88dV9P0ybNCQURH7DC30sJQAkAtldf3gA6YBEHrL6uQBxwO/BS8j0WgR5yG+aHEA6EfAsX9ru8HiCdO/MQLfaqdj+q4G8yJ8qBGTNifymFYBkEVDKkBg0Y+yfrrgHy3gUx5saw2Ymv6kFQBcBnEBn1QKpRSDTWv9/fPQhUYzIb18BAJAx9a1pkVoEsO6fDoW6/gTDyW/6ENZthbM5Wyx08vQrRpcNmwHNtXVp3X8jgaKhfHA2eE3K4qIkoKOvywrkt4wA6CKgPm0uRwNRuZEu/h+fmjL0GLLVHyTj5XcOseVISVHWxh5bhgCpGP3W813uxvpuu+4GlC9+d+mCQjAVuq1GfksKABOBrTv2uGqqN8u8TMijFJjABcwzFbl5BwlAek9ggNUKGDRj0GygQh5TQE/2DVj1A7itfPaxTIjkYHjkzCtGTRsui7t/8cI8wqdLDIo+holQ7uYZiRKAlkr22VIAgNnk4Pd7e2IT13facY+9c2Nj8wQAiUF8r5B6/lLw3V+9QdWDGhyeit5rPf/cY4fP4rTLRRn99s4ed2ODLcuHMxEP/QEEw+P9druQ31YCAFzesm2ocv3aFqfPN2Cnz3XwD8cyCkC6JUKCMJe/xYjtukgASgwJrj7/0mZtqdAW3oA+xy8VWAn4yd920YqAeKuPJb7NVo/3pRCA2ZBg6449akiw0eF220KxMxXXIA9Q6IQgeA4jO/6JhCM3cO9stOoSXz6wdTXN5JtvB6cO/+bVRV/9EjYmxbxBn1U/C0psQfa1i5fM+9nSRdVK1z2fV3wVFcq54FjaoSD6INB9f/U15W/uulvxuSuUd079kb1uut9LBfoOUn/X5lb/ZRT2hN47YutaE4csV3TJ3pf8sfHxV+KhcJtVPwMsNqx9rhoAhAwQAh1YMkyXK0hXtptpLLhEE4Fg9bG8F5Dhw7plEYDLW7bhgrY3vryzM35jcnciFrPctCFY9n/8+Wss7s+W/NNHf+cC5gQS5sX6/TJ9aKdsV/nKs739lRvWtzgrK/ut2E+Adf8H/+1fuXQD5tvhJwFA+hbZyC+lAABspWBHXxerG7BgkhCewFf2/aCkTT0gIPAmJAeuPUp5u+yY4accQJ5gYUEotDMRjvitePyI2zE1CLmBL7SsyZgXwKhwxPGoK8gkHFmTgPYp80U42CujxScByCYEfc/1xKdCW6yYH5gX36tigKQhle/Oi/P3qsTvoVNBApAWTa/tqw1/EnjGLkJAuEV89bFHVlefBICEgIhPIAEgISDiE0gAikLd9u5Oh9dj2WShBAgolNwjATBACNocbveWRDTaQWfDFECn3l67deqRAJgcrLz4+o0tiXC4k8KDsrj5/RrxA3Q6SADKioYXtnckpqe/riQSHXacSmQya/+qlefwkQDYOzyAJ9Dh9Hq3xMPhVjojXIDhm0jqDVBSjwTAUiFCNBjsUGLxxxLRaBudkYKAeP4NjfTk4pMAWN8zcPq8HfFQ+EF4COqDcgbzY3q49YfJ0pMAyCAIrQ5PRUciMg1BkNU7GEoi/DDdFSQA0qKx77m22I1J5AwgCHj22+wjBrRYHoQfpiU7EgBCjpBBEwJ4B3dqgmCVpOKwRviPNSs/TC49CQCBU+ig5Q8gDDVJomB0GDGURPZx7f9BcuVJAAjlFQh/UvhQy8FrAKF16x2gjLy98P8CDADMneNqQ66ddAAAAABJRU5ErkJggg==" - longDescription: "Creating a service instance provisions a database. Binding applications provisions unique credentials for each application to access the database." - providerDisplayName: "Pivotal Software" - documentationUrl: "https://github.com/cloudfoundry/cf-mysql-release/blob/master/README.md" - supportUrl: "https://support.pivotal.io" - dashboard_client: - id: p-mysql - secret: (( merge )) - plans: (( merge )) - mysql_node: - host: (( merge || jobs.proxy_z1.networks.mysql1.static_ips.[0] )) - admin_password: (( jobs.mysql_z1.properties.admin_password )) - persistent_disk: (( disk_pools.[0].disk_size )) - -- name: cf-mysql-broker_z2 - release: cf-mysql - template: cf-mysql-broker - instances: (( merge || 1 )) - resource_pool: cf-mysql-broker_z2 - networks: - - name: mysql2 - properties: - <<: (( merge || nil )) - max_user_connections_default: (( merge || 40 )) - syslog_aggregator: (( meta.syslog_aggregator )) - network_name: mysql2 - auth_username: (( jobs.cf-mysql-broker_z1.properties.auth_username )) - auth_password: (( jobs.cf-mysql-broker_z1.properties.auth_password )) - cookie_secret: (( jobs.cf-mysql-broker_z1.properties.cookie_secret )) - ssl_enabled: (( jobs.cf-mysql-broker_z1.properties.ssl_enabled || nil )) - skip_ssl_validation: (( jobs.cf-mysql-broker_z1.properties.skip_ssl_validation )) - external_host: (( "p-mysql." .properties.domain )) - cc_api_uri: (( jobs.cf-mysql-broker_z1.properties.cc_api_uri )) - nats: (( .properties.nats )) - networks: - broker_network: mysql2 - services: (( jobs.cf-mysql-broker_z1.properties.services )) - mysql_node: (( jobs.cf-mysql-broker_z1.properties.mysql_node )) - -- name: broker-registrar - template: broker-registrar - release: cf-mysql - instances: 1 - resource_pool: errands_z1 - lifecycle: errand - networks: - - name: mysql1 - properties: - cf: (( .properties.cf )) - broker: - name: p-mysql - host: (( jobs.cf-mysql-broker_z1.properties.external_host )) - username: (( jobs.cf-mysql-broker_z1.properties.auth_username )) - password: (( jobs.cf-mysql-broker_z1.properties.auth_password )) - services: (( jobs.cf-mysql-broker_z1.properties.services )) - -- name: broker-deregistrar - template: broker-deregistrar - release: cf-mysql - instances: 1 - resource_pool: errands_z1 - lifecycle: errand - networks: - - name: mysql1 - properties: - cf: (( .properties.cf )) - broker: - name: p-mysql - -- name: acceptance-tests - template: acceptance-tests - release: cf-mysql - instances: 1 - resource_pool: errands_z1 - lifecycle: errand - networks: - - name: mysql1 - properties: - <<: (( merge || nil )) - smoke_tests_only: (( merge || nil )) - standalone_tests_only: (( .properties.standalone )) - proxy: - external_host: (( jobs.proxy_z1.properties.external_host )) - proxy_count: (( merge || 2 )) - api_username: (( jobs.proxy_z1.properties.proxy.api_username )) - api_password: (( jobs.proxy_z1.properties.proxy.api_password )) - api_force_https: (( jobs.proxy_z1.properties.proxy.api_force_https )) - skip_ssl_validation: (( merge || nil )) - cf: (( .properties.cf )) - broker: - host: (( jobs.cf-mysql-broker_z1.properties.external_host )) - service: - max_user_connections_default: (( jobs.cf-mysql-broker_z1.properties.max_user_connections_default )) - name: (( jobs.cf-mysql-broker_z1.properties.services.[0].name )) - plans: (( jobs.cf-mysql-broker_z1.properties.services.[0].plans )) - standalone: - host: (( merge || jobs.proxy_z1.networks.[0].static_ips.[0] )) - port: (( jobs.mysql_z1.properties.port || nil )) - username: (( jobs.mysql_z1.properties.admin_username || nil )) - password: (( jobs.mysql_z1.properties.admin_password || nil )) - -- name: bootstrap - template: bootstrap - release: cf-mysql - instances: 1 - resource_pool: errands_z1 - lifecycle: errand - networks: - - name: mysql1 - properties: - cluster_ips: (( jobs.mysql_z1.properties.cluster_ips )) - healthcheck_port: (( merge || nil )) - database_startup_timeout: (( jobs.mysql_z1.properties.database_startup_timeout )) - bootstrap_endpoint: (( jobs.mysql_z1.properties.bootstrap_endpoint )) diff --git a/templates/sample_stubs/instance_count_overrides.yml b/templates/sample_stubs/instance_count_overrides.yml deleted file mode 100644 index fbcfe630..00000000 --- a/templates/sample_stubs/instance_count_overrides.yml +++ /dev/null @@ -1 +0,0 @@ -instance_count_overrides: {} diff --git a/templates/sample_stubs/sample_aws_stub.yml b/templates/sample_stubs/sample_aws_stub.yml deleted file mode 100644 index bfa1a55f..00000000 --- a/templates/sample_stubs/sample_aws_stub.yml +++ /dev/null @@ -1,132 +0,0 @@ -director_uuid: REPLACE_WITH_YOUR_DIRECTOR_UUID - -meta: - # Used as a prefix for the deployment name - # - # e.g. `environment: cf-tabasco` makes a deployment named cf-tabasco-mysql - environment: REPLACE_WITH_YOUR_ENVIRONMENT_NAME - -# NOTE: -# Replace 10.0.{0,1,2} part of the octet -# with your network configuration -networks: -- name: mysql1 - type: manual - subnets: - - range: 10.10.32.0/24 - reserved: - - 10.10.32.2 - 10.10.32.9 - - 10.10.32.130 - 10.10.32.254 - static: - - 10.10.32.10 - 10.10.32.14 - gateway: 10.10.32.1 - dns: - - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 - cloud_properties: - subnet: (( properties.template_only.aws.subnet_ids.mysql1 )) -- name: mysql2 - type: manual - subnets: - - range: 10.10.33.0/24 - reserved: - - 10.10.33.2 - 10.10.33.9 - - 10.10.33.130 - 10.10.33.254 - static: - - 10.10.33.10 - 10.10.33.14 - gateway: 10.10.33.1 - dns: - - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 - cloud_properties: - subnet: (( properties.template_only.aws.subnet_ids.mysql2 )) -- name: mysql3 - type: manual - subnets: - - range: 10.10.34.0/24 - reserved: - - 10.10.34.2 - 10.10.34.9 - - 10.10.34.130 - 10.10.34.254 - static: - - 10.10.34.10 - 10.10.34.14 - gateway: 10.10.34.1 - dns: - - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 - cloud_properties: - subnet: (( properties.template_only.aws.subnet_ids.mysql3 )) -- name: compilation - type: manual - subnets: - - range: 10.10.35.0/24 - name: default_unused - reserved: - - 10.10.35.2 - 10.10.35.9 - - 10.10.35.130 - 10.10.35.254 - static: - - 10.10.35.10 - 10.10.35.14 - gateway: 10.10.35.1 - dns: - - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 - cloud_properties: - subnet: (( properties.template_only.aws.subnet_ids.compilation )) - -properties: - cf: - # these come from your cf-release manifest - admin_username: REPLACE_WITH_YOUR_CF_ADMIN_USERNAME - admin_password: REPLACE_WITH_YOUR_CF_ADMIN_PASSWORD - skip_ssl_validation: REPLACE_WITH_TRUE_OR_FALSE # True if you don't have a trusted SSL certificate, false if you do - - domain: REPLACE_WITH_YOUR_CF_SYSTEM_DOMAIN # The broker registers a route at p-msyql.external_domain. - app_domains: - - REPLACE_WITH_YOUR_CF_APPS_DOMAIN - nats: - user: nats - password: REPLACE_WITH_NATS_PASSWORD_FROM_CF_RELEASE - machines: - - REPLACE_WITH_NATS_IP_ADDRESS_FROM_CF_RELEASE - port: 4222 - - template_only: - aws: - availability_zone: REPLACE_WITH_YOUR_AWS_AVAILABILITY_ZONE_1 #From the EC2 page of the AWS console, like us-east-1a - availability_zone2: REPLACE_WITH_YOUR_AWS_AVAILABILITY_ZONE_2 #From the EC2 page of the AWS console, like us-east-1b - availability_zone3: REPLACE_WITH_YOUR_AWS_AVAILABILITY_ZONE_3 #From the EC2 page of the AWS console, like us-east-1c - subnet_ids: - # From VPC/Subnets page of AWS console. Availability zone must match the value set above. - mysql1: REPLACE_WITH_YOUR_SUBNET_ID_1 - mysql2: REPLACE_WITH_YOUR_SUBNET_ID_2 - mysql3: REPLACE_WITH_YOUR_SUBNET_ID_3 - compilation: REPLACE_WITH_YOUR_COMPILATION_SUBNET - - -jobs: -# Jobs for cf-mysql-release are below -- name: mysql_z1 - properties: - admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process - bootstrap_endpoint: - username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP - password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP - -- name: cf-mysql-broker_z1 - properties: - cf: - admin_username: (( .properties.cf.admin_username )) - admin_password: (( .properties.cf.admin_password )) - # The username & password cloud controller will use to authenticate with the service broker. - auth_username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP # eg. cc - auth_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP - cookie_secret: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP - services: - - name: p-mysql - dashboard_client: - secret: REPLACE_WITH_PASSWORD_YOU_MAKE_UP #Secret used by MySQL dashboard to fetch an OAuth token from UAA - plans: - - name: REPLACE_WITH_YOUR_PLAN_NAME - max_storage_mb: REPLACE_WITH_PLAN_SIZE_IN_MB - -- name: proxy_z1 - release: cf-mysql - properties: - proxy: - api_username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP - api_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP diff --git a/templates/sample_stubs/sample_plans_stub.yml b/templates/sample_stubs/sample_plans_stub.yml deleted file mode 100644 index 67f91130..00000000 --- a/templates/sample_stubs/sample_plans_stub.yml +++ /dev/null @@ -1,36 +0,0 @@ -jobs: -- name: cf-mysql-broker_z1 - properties: - services: - - name: p-mysql - plans: - - name: 100mb - id: ab08f1bc-e6fc-4b56-a767-ee0fea6e3f20 - description: Shared MySQL Server - max_storage_mb: 100 - max_user_connections: 20 - metadata: - costs: - - amount: - usd: 0.0 - unit: MONTH - bullets: - - Shared MySQL server - - 100 MB storage - - 20 concurrent connections - displayName: "100 MB" - - name: 1gb - id: 11d0aa36-dcec-4021-85f5-ea4d9a5c8342 - description: Shared MySQL Server - max_storage_mb: 1000 - max_user_connections: 40 - metadata: - costs: - - amount: - usd: 0.0 - unit: MONTH - bullets: - - Shared MySQL server - - 1000 MB storage - - 40 concurrent connections - displayName: "1 GB" diff --git a/templates/sample_stubs/sample_vsphere_stub.yml b/templates/sample_stubs/sample_vsphere_stub.yml deleted file mode 100644 index 50d0d193..00000000 --- a/templates/sample_stubs/sample_vsphere_stub.yml +++ /dev/null @@ -1,135 +0,0 @@ -director_uuid: REPLACE_WITH_YOUR_DIRECTOR_UUID - -meta: - # Used as a prefix for the deployment name - # - # e.g. `environment: cf-tabasco` makes a deployment named cf-tabasco-mysql - environment: REPLACE_WITH_YOUR_ENVIRONMENT_NAME - -# NOTE: -# Replace 10.0.{0,1,2} part of the octet -# with your network configuration -networks: -- name: mysql1 - subnets: - - range: 10.0.0.0/24 - gateway: 10.0.0.1 - reserved: - - 10.0.0.2-10.0.0.109 - - 10.0.0.130-10.0.0.254 - static: - - 10.0.0.120 - - 10.0.0.121 - - 10.0.0.122 - - 10.0.0.123 - - 10.0.0.124 - dns: - - 10.0.0.3 - - 8.8.8.8 - cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME #This should match your vSphere network name, e.g. "VM Network". -- name: mysql2 - subnets: - - range: 10.0.1.0/24 - gateway: 10.0.1.1 - reserved: - - 10.0.1.2-10.0.1.109 - - 10.0.1.130-10.0.1.254 - static: - - 10.0.1.120 - - 10.0.1.121 - - 10.0.1.122 - - 10.0.1.123 - - 10.0.1.124 - dns: - - 10.0.1.3 - - 8.8.8.8 - cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME_2 -- name: mysql3 - subnets: - - range: 10.0.2.0/24 - gateway: 10.0.2.1 - reserved: - - 10.0.2.2-10.0.2.109 - - 10.0.2.130-10.0.2.254 - static: - - 10.0.2.120 - - 10.0.2.121 - - 10.0.2.122 - - 10.0.2.123 - - 10.0.2.124 - dns: - - 10.0.2.3 - - 8.8.8.8 - cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME_3 -- name: compilation - subnets: - - range: 10.0.3.0/24 - gateway: 10.0.3.1 - reserved: - - 10.0.3.2-10.0.3.169 - - 10.0.3.190-10.0.3.254 - static: - - 10.0.3.170-10.0.3.174 - dns: - - 10.0.3.3 - - 8.8.8.8 - cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME_4 - -properties: - cf: - admin_username: REPLACE_WITH_YOUR_CF_ADMIN_USERNAME - admin_password: REPLACE_WITH_YOUR_CF_ADMIN_PASSWORD - skip_ssl_validation: REPLACE_WITH_TRUE_OR_FALSE # True if you don't have a trusted SSL certificate, false if you do - domain: REPLACE_WITH_YOUR_CF_SYSTEM_DOMAIN # The broker registers a route at p-mysql.external_domain - app_domains: - - REPLACE_WITH_YOUR_CF_APPS_DOMAIN - nats: - machines: - - REPLACE_WITH_NATS_IP_ADDRESS_FROM_CF_RELEASE - password: REPLACE_WITH_NATS_PASSWORD_FROM_CF_RELEASE - user: nats - port: 4222 - # A multi-AZ VSphere environment can be distributed across different datacenters and/or clusters - datacenters: - - name: z1 - datacenter_name: REPLACE_WITH_YOUR_FIRST_DATACENTER - clusters: - - REPLACE_WITH_YOUR_FIRST_CLUSTER: - resource_pool: REPLACE_WITH_YOUR_FIRST_RESOURCE_POOL - - name: z2 - datacenter_name: REPLACE_WITH_YOUR_SECOND_DATACENTER - clusters: - - REPLACE_WITH_YOUR_SECOND_CLUSTER: - resource_pool: REPLACE_WITH_YOUR_SECOND_RESOURCE_POOL - - name: z3 - datacenter_name: REPLACE_WITH_YOUR_THIRD_DATACENTER - clusters: - - REPLACE_WITH_YOUR_THIRD_CLUSTER: - resource_pool: REPLACE_WITH_YOUR_THIRD_RESOURCE_POOL - -jobs: -- name: proxy_z1 - properties: - proxy: - api_username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP # Used for basic auth in front of proxy api - api_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP -- name: mysql_z1 - properties: - admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process - bootstrap_endpoint: - username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP - password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP -- name: cf-mysql-broker_z1 - properties: - # The username & password cloud controller will use to authenticate with the service broker. - auth_username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP #eg. cc - auth_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP - cookie_secret: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP - services: - - name: p-mysql - dashboard_client: - secret: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #Secret used by MySQL dashboard to fetch an OAuth token from UAA From 1a9eab5936809890d45db6659a798479acae134a Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Thu, 31 Dec 2015 14:14:56 -0800 Subject: [PATCH 022/110] Update test-sample-stubs to test all the new stubs - Tests that all IaaS and use case example can be merged via Spiff [#110775864] Signed-off-by: Lyle Franklin --- scripts/test-sample-stubs | 49 +++++++++++++++++++++++++++++++++++++++ scripts/test_sample_stubs | 32 ------------------------- 2 files changed, 49 insertions(+), 32 deletions(-) create mode 100755 scripts/test-sample-stubs delete mode 100755 scripts/test_sample_stubs diff --git a/scripts/test-sample-stubs b/scripts/test-sample-stubs new file mode 100755 index 00000000..128051ff --- /dev/null +++ b/scripts/test-sample-stubs @@ -0,0 +1,49 @@ +#!/bin/bash + +set -eux + +MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# This script assumes that it lives one directory below the release directory. +RELEASE_DIR="$( cd ${MY_DIR}/.. && pwd )" + +FAKE_CF_MANIFEST="${RELEASE_DIR}/manifest-generation/examples/standalone/standalone-cf-manifest.yml" + +# aws +${RELEASE_DIR}/scripts/generate-deployment-manifest \ + -c "${FAKE_CF_MANIFEST}" \ + -p "${RELEASE_DIR}"/manifest-generation/examples/property-overrides.yml \ + -i "${RELEASE_DIR}"/manifest-generation/examples/aws/iaas-settings.yml + +# vsphere +${RELEASE_DIR}/scripts/generate-deployment-manifest \ + -c "${FAKE_CF_MANIFEST}" \ + -p "${RELEASE_DIR}"/manifest-generation/examples/property-overrides.yml \ + -i "${RELEASE_DIR}"/manifest-generation/examples/vsphere/iaas-settings.yml + +# bosh-lite +${RELEASE_DIR}/scripts/generate-deployment-manifest \ + -c "${FAKE_CF_MANIFEST}" \ + -p "${RELEASE_DIR}"/manifest-generation/bosh-lite-stubs/property-overrides.yml \ + -i "${RELEASE_DIR}"/manifest-generation/bosh-lite-stubs/iaas-settings.yml + +# standalone +${RELEASE_DIR}/scripts/generate-deployment-manifest \ + -c "${FAKE_CF_MANIFEST}" \ + -p "${RELEASE_DIR}"/manifest-generation/examples/standalone/property-overrides.yml \ + -i "${RELEASE_DIR}"/manifest-generation/bosh-lite-stubs/iaas-settings.yml \ + -n "${RELEASE_DIR}"/manifest-generation/examples/standalone/instance-count-overrides.yml + +# no arbitrator +${RELEASE_DIR}/scripts/generate-deployment-manifest \ + -c "${FAKE_CF_MANIFEST}" \ + -p "${RELEASE_DIR}"/manifest-generation/examples/property-overrides.yml \ + -i "${RELEASE_DIR}"/manifest-generation/bosh-lite-stubs/iaas-settings.yml \ + -n "${RELEASE_DIR}"/manifest-generation/examples/no-arbitrator/instance-count-overrides.yml + +# minimal +${RELEASE_DIR}/scripts/generate-deployment-manifest \ + -c "${FAKE_CF_MANIFEST}" \ + -p "${RELEASE_DIR}"/manifest-generation/examples/property-overrides.yml \ + -i "${RELEASE_DIR}"/manifest-generation/bosh-lite-stubs/iaas-settings.yml \ + -n "${RELEASE_DIR}"/manifest-generation/examples/minimal/instance-count-overrides.yml diff --git a/scripts/test_sample_stubs b/scripts/test_sample_stubs deleted file mode 100755 index f9b639bc..00000000 --- a/scripts/test_sample_stubs +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/bash - -set -eux - -MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" - -# This script assumes that it lives one directory below the release directory. -RELEASE_DIR="$( cd ${MY_DIR}/.. && pwd )" - -# test the following cases - -# aws -# -# vsphere -# -# bosh-lite -# -# standalone -# -# no arbitrator -# -# minimal - -${RELEASE_DIR}/generate_deployment_manifest \ - aws \ - ${RELEASE_DIR}/templates/sample_stubs/sample_plans_stub.yml \ - ${RELEASE_DIR}/templates/sample_stubs/sample_aws_stub.yml - -${RELEASE_DIR}/generate_deployment_manifest \ - vsphere \ - ${RELEASE_DIR}/templates/sample_stubs/sample_plans_stub.yml \ - ${RELEASE_DIR}/templates/sample_stubs/sample_vsphere_stub.yml From 3372529694009c21ae13fe31274b3edac1bcf0e3 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 31 Dec 2015 15:58:17 -0800 Subject: [PATCH 023/110] Update release_versions format to match property-overrides - Format: ``` release_versions: - name: cf-mysql version: 25 ``` Signed-off-by: Ben Calegari [#110775864] --- manifest-generation/cf-mysql-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index b51819ea..be8bf12b 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -4,7 +4,7 @@ director_uuid: (( config_from_cf.cf_director_uuid )) releases: - name: cf-mysql - version: (( release_versions.cf-mysql || "latest" )) + version: (( release_versions.cf-mysql.version || "latest" )) compilation: workers: 4 From b3ba5816edc56f90c13efbe7256772b6a50f1062 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Thu, 31 Dec 2015 15:59:30 -0800 Subject: [PATCH 024/110] Allow multiple property-overrides to be merged together - Allows more flexibility when creating more complex deployments [#110775864] Signed-off-by: Lyle Franklin --- manifest-generation/examples/instance-count-overrides.yml | 1 + manifest-generation/examples/property-overrides.yml | 1 + manifest-generation/examples/standalone/property-overrides.yml | 1 + 3 files changed, 3 insertions(+) create mode 100644 manifest-generation/examples/instance-count-overrides.yml diff --git a/manifest-generation/examples/instance-count-overrides.yml b/manifest-generation/examples/instance-count-overrides.yml new file mode 100644 index 00000000..f969f03e --- /dev/null +++ b/manifest-generation/examples/instance-count-overrides.yml @@ -0,0 +1 @@ +instance_count_overrides: [] diff --git a/manifest-generation/examples/property-overrides.yml b/manifest-generation/examples/property-overrides.yml index 233147ad..21a9e963 100644 --- a/manifest-generation/examples/property-overrides.yml +++ b/manifest-generation/examples/property-overrides.yml @@ -1,4 +1,5 @@ property_overrides: + <<: (( merge || nil )) syslog_aggregator: ~ standalone: false host: YOUR_LOAD_BALANCER_ADDRESS # Optional, set to your Load Balancer address if configured diff --git a/manifest-generation/examples/standalone/property-overrides.yml b/manifest-generation/examples/standalone/property-overrides.yml index 18d5d8f5..02873682 100644 --- a/manifest-generation/examples/standalone/property-overrides.yml +++ b/manifest-generation/examples/standalone/property-overrides.yml @@ -1,4 +1,5 @@ property_overrides: + <<: (( merge || nil )) standalone: true host: YOUR_LOAD_BALANCER_ADDRESS # Optional, set to your Load Balancer address if configured mysql: From 21c20f5680cad969bc177d14b4b3c13ed85878c3 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 31 Dec 2015 16:00:10 -0800 Subject: [PATCH 025/110] Provide a boilerplate CF manifest for bosh-lites - Removes the need to pull admin creds from a deployment [#110775864] Signed-off-by: Ben Calegari --- .../bosh-lite-stubs/cf-manifest.yml | 15 +++++++++++++++ scripts/generate-bosh-lite-manifest | 14 ++++++++------ 2 files changed, 23 insertions(+), 6 deletions(-) create mode 100644 manifest-generation/bosh-lite-stubs/cf-manifest.yml diff --git a/manifest-generation/bosh-lite-stubs/cf-manifest.yml b/manifest-generation/bosh-lite-stubs/cf-manifest.yml new file mode 100644 index 00000000..031d5e75 --- /dev/null +++ b/manifest-generation/bosh-lite-stubs/cf-manifest.yml @@ -0,0 +1,15 @@ +name: cf-warden +director_uuid: REPLACE_WITH_DIRECTOR_UUID +properties: + admin_username: admin + admin_password: admin + nats: + machines: [10.244.0.6] + password: nats + port: 4222 + user: nats + system_domain: bosh-lite.com + app_domains: + - bosh-lite.com + domain: bosh-lite.com + skip_ssl_validation: true diff --git a/scripts/generate-bosh-lite-manifest b/scripts/generate-bosh-lite-manifest index 4c4e6b17..49f44573 100755 --- a/scripts/generate-bosh-lite-manifest +++ b/scripts/generate-bosh-lite-manifest @@ -9,18 +9,20 @@ DIRECTOR_IP=${DIRECTOR_IP:-192.168.50.4} BOSH_LITE_USERNAME=${BOSH_LITE_USERNAME:-admin} BOSH_LITE_PASSWORD=${BOSH_LITE_PASSWORD:-admin} +tmpdir=$(mktemp -d /tmp/mysql_manifest.XXXXX) +trap '{ rm -rf ${tmpdir}; }' EXIT + pushd "${RELEASE_DIR}" > /dev/null bosh -n target "${DIRECTOR_IP}" bosh -n login "${BOSH_LITE_USERNAME}" "${BOSH_LITE_PASSWORD}" - - tmpdir=$(mktemp -d /tmp/mysql_manifest.XXXXX) - trap '{ rm -rf ${tmpdir}; }' EXIT - - bosh -n download manifest cf-warden "${tmpdir}/cf-warden.yml" + DIRECTOR_UUID="$(bosh status --uuid)" + sed "s/REPLACE_WITH_DIRECTOR_UUID/$DIRECTOR_UUID/g" \ + "${RELEASE_DIR}/manifest-generation/bosh-lite-stubs/cf-manifest.yml" \ + > "${tmpdir}/bosh-lite-cf-manifest.yml" ./scripts/generate-deployment-manifest \ - -c "${tmpdir}/cf-warden.yml" \ + -c "${tmpdir}/bosh-lite-cf-manifest.yml" \ -p ./manifest-generation/bosh-lite-stubs/property-overrides.yml \ -i ./manifest-generation/bosh-lite-stubs/iaas-settings.yml \ > cf-mysql.yml From ffbdfe2f5f43898c4dd66f06e60646e75e0701f2 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Fri, 1 Jan 2016 13:58:34 -0800 Subject: [PATCH 026/110] Make bosh templates tolerant of nil values in cluster_ips - Spiff's limited support of array manipulation results in nil values in cluster_ips in certain configurations, e.g. minimal deployment - All job templates now filter out nil elements before rendering [#110775864] --- jobs/acceptance-tests/templates/errand.sh.erb | 11 ++++++++--- jobs/arbitrator/templates/garbd_config.erb | 7 ++++++- jobs/bootstrap/templates/config.yml.erb | 7 ++++++- jobs/mysql/templates/mariadb_ctl_config.yml.erb | 7 ++++++- jobs/mysql/templates/my.cnf.erb | 5 ++++- jobs/proxy/templates/switchboard.yml.erb | 10 ++++++++-- manifest-generation/cf-mysql-template.yml | 4 ++-- 7 files changed, 40 insertions(+), 11 deletions(-) diff --git a/jobs/acceptance-tests/templates/errand.sh.erb b/jobs/acceptance-tests/templates/errand.sh.erb index 810433f6..3adca9d7 100644 --- a/jobs/acceptance-tests/templates/errand.sh.erb +++ b/jobs/acceptance-tests/templates/errand.sh.erb @@ -3,8 +3,13 @@ set -e -x cd /var/vcap/packages/acceptance-tests/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests -<% mysql_service = p('cf_mysql.broker.services').first %> -<% default_max_user_connections = mysql_service['max_user_connections_default'] || 40 %> +<% +mysql_service = p('cf_mysql.broker.services').first +default_max_user_connections = mysql_service['max_user_connections_default'] || 40 + +# proxy_ips may contain nil values due to lack of flexibility in spiff +proxy_ips = p('cf_mysql.proxy.proxy_ips').compact +%> cat > integration_config.json < integration_config.json <, "proxy": { - <% urls = p('cf_mysql.proxy.proxy_ips').each_with_index.map do |_, index| + <% urls = proxy_ips.each_with_index.map do |_, index| "\"https://proxy-#{index}-#{p('cf_mysql.external_host')}\"" end %> "dashboard_urls": [<%= urls.join(",") %>], diff --git a/jobs/arbitrator/templates/garbd_config.erb b/jobs/arbitrator/templates/garbd_config.erb index 2dad092f..62d58ad5 100644 --- a/jobs/arbitrator/templates/garbd_config.erb +++ b/jobs/arbitrator/templates/garbd_config.erb @@ -1,4 +1,9 @@ +<% +# cluster_ips may contain nil values due to lack of flexibility in spiff +cluster_ips = p('cf_mysql.mysql.cluster_ips').compact +%> + group = cf-mariadb-galera-cluster <% galera_port = p('cf_mysql.mysql.galera_port') %> -<% node_addresses = p('cf_mysql.mysql.cluster_ips').map { |ip| "#{ip}:#{galera_port}" } %> +<% node_addresses = cluster_ips.map { |ip| "#{ip}:#{galera_port}" } %> address = gcomm://<%= node_addresses.join(',') %> diff --git a/jobs/bootstrap/templates/config.yml.erb b/jobs/bootstrap/templates/config.yml.erb index ca4c3df4..1ff378a4 100644 --- a/jobs/bootstrap/templates/config.yml.erb +++ b/jobs/bootstrap/templates/config.yml.erb @@ -1,5 +1,10 @@ +<% +# cluster_ips may contain nil values due to lack of flexibility in spiff +cluster_ips = p('cf_mysql.mysql.cluster_ips').compact +%> + HealthcheckURLs: - <% p('cf_mysql.mysql.cluster_ips').each do |ip| %> + <% cluster_ips.each do |ip| %> - http://<%= ip %>:<%= p('cf_mysql.mysql.healthcheck_port') %> <% end %> DatabaseStartupTimeout: <%= p('cf_mysql.mysql.database_startup_timeout') %> diff --git a/jobs/mysql/templates/mariadb_ctl_config.yml.erb b/jobs/mysql/templates/mariadb_ctl_config.yml.erb index 3be2767c..ff5a7cf0 100644 --- a/jobs/mysql/templates/mariadb_ctl_config.yml.erb +++ b/jobs/mysql/templates/mariadb_ctl_config.yml.erb @@ -1,4 +1,9 @@ --- +<% +# cluster_ips may contain nil values due to lack of flexibility in spiff +cluster_ips = p('cf_mysql.mysql.cluster_ips').compact +%> + LogFileLocation: /var/vcap/sys/log/mysql/mariadb_ctrl.combined.log PidFile: /var/vcap/sys/run/mariadb_ctl/mariadb_ctl.pid Db: @@ -18,7 +23,7 @@ Upgrader: Manager: StateFileLocation: /var/vcap/store/mysql/state.txt ClusterIps: - <% p('cf_mysql.mysql.cluster_ips').each do |ip| %> + <% cluster_ips.each do |ip| %> - <%= ip %> <% end %> MyIP: <%= spec.networks.send(p('network_name')).ip %> diff --git a/jobs/mysql/templates/my.cnf.erb b/jobs/mysql/templates/my.cnf.erb index d38dc7ae..5ee075d0 100644 --- a/jobs/mysql/templates/my.cnf.erb +++ b/jobs/mysql/templates/my.cnf.erb @@ -1,5 +1,8 @@ <% base_folder = '/var/vcap/sys/run/mysql' + +# cluster_ips may contain nil values due to lack of flexibility in spiff +cluster_ips = p('cf_mysql.mysql.cluster_ips').compact %> <% @@ -24,7 +27,7 @@ nice = 0 # GALERA options: wsrep_provider=/var/vcap/packages/mariadb/lib/plugin/libgalera_smm.so wsrep_provider_options="gcache.size=<%= p('cf_mysql.mysql.gcache_size') %>M;pc.recovery=TRUE" -wsrep_cluster_address="gcomm://<%= p('cf_mysql.mysql.cluster_ips').join(",") %>" +wsrep_cluster_address="gcomm://<%= cluster_ips.join(",") %>" wsrep_node_address='<%= spec.networks.send(p('network_name')).ip %>' wsrep_node_name='mysql/<%= index %>' wsrep_cluster_name='cf-mariadb-galera-cluster' diff --git a/jobs/proxy/templates/switchboard.yml.erb b/jobs/proxy/templates/switchboard.yml.erb index e3a77f0f..91ba6c89 100644 --- a/jobs/proxy/templates/switchboard.yml.erb +++ b/jobs/proxy/templates/switchboard.yml.erb @@ -1,3 +1,11 @@ +<% +# cluster_ips may contain nil values due to lack of flexibility in spiff +cluster_ips = p('cf_mysql.mysql.cluster_ips').compact + +# do not route traffic to arbitrator +cluster_ips.delete(p('cf_mysql.proxy.arbitrator_ip')) +%> + "API": "Port": <%= p('cf_mysql.proxy.api_port') %> "ForceHttps": <%= p('cf_mysql.proxy.api_force_https') %> @@ -7,8 +15,6 @@ "Port": <%= p('cf_mysql.proxy.port') %> "HealthcheckTimeoutMillis": <%= p("cf_mysql.proxy.healthcheck_timeout_millis") %> "Backends": - <% cluster_ips = p('cf_mysql.mysql.cluster_ips') %> - <% cluster_ips.delete(p('cf_mysql.proxy.arbitrator_ip')) %> <% cluster_ips.each_with_index do |ip, n| %> - "Host": <%= ip %> "Port": 3306 diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index be8bf12b..d17b5950 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -283,7 +283,7 @@ properties: smoke_tests_only: (( property_overrides.acceptance_tests.smoke_tests_only || nil )) standalone_tests_only: (( property_overrides.acceptance_tests.standalone_tests_only || nil )) timeout_scale: (( property_overrides.acceptance_tests.timeout_scale || nil )) - + # The keys below should not be included in the final stub config_from_cf: (( merge )) iaas_settings: (( merge )) @@ -292,7 +292,7 @@ property_overrides: (( merge )) release_versions: (( merge || nil )) empty_hash: {} default_mysql_cluster_ips: -- (( jobs.mysql_z1.networks.mysql1.static_ips.[0] || nil )) +- (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) - (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) default_proxy_ips: From bb329f396f2eae847a445673c1e301e74c7dd921 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Fri, 1 Jan 2016 14:00:50 -0800 Subject: [PATCH 027/110] Instruct user to add their own CF_DEPLOYMENT_NAME to standalone stub - Not all cf deployments have the name `cf`, e.g. `cf-warden` [#110775864] --- .../examples/standalone/standalone-cf-manifest.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/manifest-generation/examples/standalone/standalone-cf-manifest.yml b/manifest-generation/examples/standalone/standalone-cf-manifest.yml index 7f9a752e..75b8986c 100644 --- a/manifest-generation/examples/standalone/standalone-cf-manifest.yml +++ b/manifest-generation/examples/standalone/standalone-cf-manifest.yml @@ -1,6 +1,6 @@ # Standalone deployments have no CF, but these fields are required # for all deployments. -name: cf +name: REPLACE_WITH_CF_DEPLOYMENT_NAME director_uuid: REPLACE_WITH_DIRECTOR_UUID properties: admin_username: UNUSED_VALUE @@ -13,4 +13,3 @@ properties: system_domain: UNUSED_VALUE app_domains: [UNUSED_VALUE] domain: UNUSED_VALUE - From 954001666ed8f5c132736e44c34bae81c3db68fe Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Mon, 4 Jan 2016 10:54:08 -0800 Subject: [PATCH 028/110] Update manifest generation instructions to reflect new flags [#110775864] Signed-off-by: Aditya Anchuri --- README.md | 100 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 68 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 090b5452..9656a7fb 100644 --- a/README.md +++ b/README.md @@ -276,56 +276,92 @@ The brokers each register a route with the router, which load balances requests ### Create Manifest and Deploy -#### BOSH-lite - -1. Generate the manifest using a bosh-lite specific script and a stub provided for you, `bosh-lite/cf-mysql-stub-spiff.yml`. +#### Deploy on BOSH-lite +1. Run the following script to generate a working manifest for a bosh-lite on your local machine: ``` - $ ./bosh-lite/make_manifest + $ ./scripts/generate-bosh-lite-manifest ``` - The resulting file, `bosh-lite/manifests/cf-mysql-manifest.yml` is your deployment manifest. To modify the deployment configuration, you can edit the stub and regenerate the manifest or edit the manifest directly. -1. The `make_manifest` script will set the deployment to `bosh-lite/manifests/cf-mysql-manifest.yml` for you, so to deploy you only need to run: +1. Run the following command to deploy cf-mysql to the bosh-lite (assumes release has already been uploaded): ``` $ bosh deploy ``` - -#### vSphere - -1. Create a stub file called `cf-mysql-vsphere-stub.yml` by copying and modifying the [sample_vsphere_stub.yml](https://github.com/cloudfoundry/cf-mysql-release/blob/master/templates/sample_stubs/sample_vsphere_stub.yml) in `templates/sample_stubs`. The `sample_plans_stub.yml` can also be copied if values need changing. - -2. Generate the manifest: +1. To edit the manifest: ``` - $ ./generate_deployment_manifest \ - vsphere \ - plans_stub.yml \ - cf-mysql-vsphere-stub.yml > cf-mysql-vsphere.yml + $ bosh edit deployment ``` - The resulting file, `cf-mysql-vsphere.yml` is your deployment manifest. To modify the deployment configuration, you can edit the stub and regenerate the manifest or edit the manifest directly. -3. To deploy: - ``` - $ bosh deployment cf-mysql-vsphere.yml && bosh deploy - ``` +#### Deploy on AWS or vSphere - -#### AWS +For other environments, we provide a script called `generate-deployment-manifest`. -1. Create a stub file called `cf-mysql-aws-stub.yml` by copying and modifying the [sample_aws_stub.yml](https://github.com/cloudfoundry/cf-mysql-release/blob/master/templates/sample_stubs/sample_aws_stub.yml) in `templates/sample_stubs`. The `sample_plans_stub.yml` can also be copied if values need changing. +``` +Usage: +The script requires the following arguments- + -c CF Manifest + -p Property overrides stub file (Use this file to provide credentials and broker plans) + -i Infrastructure type stub file (AWS or vSphere) +The following arguments are optional- + -n Instance count overrides stub file (single node, 3 node) + -v Release versions stub file (Use this file to specify the cf-mysql release version, defaults to latest) +``` -1. Generate the manifest: +For example: + +``` +$ ./scripts/generate-deployment-manifest \ + -c /cf-deployment.yml \ + -p /cf-mysql/property-overrides.yml \ + -i /cf-mysql/iaas-settings.yml \ + > cf-mysql.yml +``` + +##### Provide a CF manifest + +The script must obtain some configuration options from an existing CF manifest (e.g. CF Admin credentials). +Add the following properties to your CF manifest prior to running this script: +``` +properties: + admin_username: REPLACE_WITH_ADMIN_USERNAME + admin_password: REPLACE_WITH_ADMIN_PASSWORD + skip_ssl_validation: # optional, defaults to false +``` +This admin user must have the `cloud_controller.admin` UAA permission. + +Note: This change to the CF manifest is temporary while we investigate better methods for sharing properties across deployments. + +##### Copy sample stubs and fill in values + +We have provided example stubs to serve as a starting point. +Copy these stubs to your config repository, and fill in the `REPLACE_WITH` text with values for your environment. + +For example, the following files can be used for an AWS deployment: +``` +$ cp cf-mysql-release/manifest-generation/examples/aws/iaas-settings.yml \ + cf-mysql-release/manifest-generation/examples/property-overrides.yml \ + /cf-mysql/ +``` + +Additional example stubs can be found under `cf-mysql-release/manifest-generation/examples/`. +These include: +- Deploying with a minimal number of VMs (`examples/minimal/`) +- Deploying without a running CF deployment (`examples/standalone/`) +- Replacing the arbitrator with a full MySQL node (`examples/no-arbitrator/`) + +##### Deploying + +Once the manifest has been generated, do the following to deploy cf-mysql on your environment. + +1. Deploy CF-MySQL (assumes release has already been uploaded): ``` - $ ./generate_deployment_manifest \ - aws \ - plans_stub.yml \ - cf-mysql-aws-stub.yml > cf-mysql-aws.yml + $ bosh deploy ``` - The resulting file, `cf-mysql-aws.yml` is your deployment manifest. To modify the deployment configuration, you can edit the stub and regenerate the manifest or edit the manifest directly. -1. To deploy: +1. To edit the manifest: ``` - $ bosh deployment cf-mysql-aws.yml && bosh deploy + $ bosh edit deployment ``` From 13245e9d394ffe9555d67fed958cbb820db77d20 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 4 Jan 2016 10:55:03 -0800 Subject: [PATCH 029/110] Include `--uuid` flag when documenting how to obtain director id - This command only prints the uuid, while `status` prints other unnecessary info Signed-off-by: Lyle Franklin --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9656a7fb..7c764484 100644 --- a/README.md +++ b/README.md @@ -369,7 +369,7 @@ Once the manifest has been generated, do the following to deploy cf-mysql on you Manifest properties are described in the `spec` file for each job; see [jobs](jobs). -You can find your `director_uuid` by running `bosh status`. +You can find your `director_uuid` by running `bosh status --uuid`. The MariaDB cluster nodes are configured by default with 100GB of persistent disk. This can be configured in your stub or manifest using `disk_pools.mysql-persistent-disk.disk_size`, however your deployment will fail if this is less than 3GB; we recommend allocating 10GB at a minimum. From 1d08c6a817f24c59e3d2a29b8352c1e602c04608 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Mon, 4 Jan 2016 11:40:46 -0800 Subject: [PATCH 030/110] Update bootstrap doc to show new arbitrator job [#109744234] [ci skip] Signed-off-by: Aditya Anchuri --- docs/bootstrapping.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/bootstrapping.md b/docs/bootstrapping.md index ce2db52b..866c488f 100644 --- a/docs/bootstrapping.md +++ b/docs/bootstrapping.md @@ -45,11 +45,11 @@ If the cluster was already healthy to begin with (i.e. quorum was never lost), t If one or more nodes are not reachable (i.e. the VM exists but in an unknown state), it will error out saying `nodes are not reachable`. In this situation, follow the steps below: -1. `bosh -n stop mysql_z1 && bosh -n stop mysql_z2 && bosh -n stop mysql_z3` +1. `bosh -n stop mysql_z1 && bosh -n stop mysql_z2 && bosh -n stop _z3` 1. `bosh edit deployment` 1. Set `update.canaries` to 0, `update.max_in_flight` to 3, and `update.serial` to false. 1. `bosh deploy` -1. `bosh -n start mysql_z1 ; bosh -n start mysql_z2 ; bosh -n start mysql_z3` (This will throw several errors, but it ensures that all the jobs are present on the VM) +1. `bosh -n start mysql_z1 ; bosh -n start mysql_z2 ; bosh -n start _z3` (This will throw several errors, but it ensures that all the jobs are present on the VM) 1. `bosh instances` to verify that all jobs report as failing. 1. Try running the errand again using `bosh -n run errand bootstrap` as above. 1. Once the errand succeeds, the cluster is synced, although some jobs might still report as failing. @@ -58,11 +58,12 @@ If one or more nodes are not reachable (i.e. the VM exists but in an unknown sta 1. Verify that deployment succeeds and all jobs are healthy. A healthy deployment should look like this: ``` -$ bosh vms cf-warden-mysql | egrep 'mysql_z./0' +$ bosh vms cf-warden-mysql' Acting as user 'admin' on deployment 'cf-warden-mysql' on 'Bosh Lite Director' | mysql_z1/0 | running | mysql_z1 | 10.244.7.2 | | mysql_z2/0 | running | mysql_z2 | 10.244.8.2 | -| mysql_z3/0 | running | mysql_z3 | 10.244.9.2 | +| arbitrator_z3/0 | running | arbitrator_z3 | 10.244.9.6 | +... ``` If these steps did not work for you, please refer to the [Manual Bootstrap Process](#manual-bootstrap-process) below. From b55979951d19d9b9aacb429299c1b6363eb28d5b Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Mon, 4 Jan 2016 11:47:43 -0800 Subject: [PATCH 031/110] Provide links to example stubs in README [#109744236] [ci skip] --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 7c764484..4ce0678f 100644 --- a/README.md +++ b/README.md @@ -344,11 +344,11 @@ $ cp cf-mysql-release/manifest-generation/examples/aws/iaas-settings.yml \ /cf-mysql/ ``` -Additional example stubs can be found under `cf-mysql-release/manifest-generation/examples/`. +Additional example stubs can be found under [cf-mysql-release/manifest-generation/examples/](manifest-generation/examples). These include: -- Deploying with a minimal number of VMs (`examples/minimal/`) -- Deploying without a running CF deployment (`examples/standalone/`) -- Replacing the arbitrator with a full MySQL node (`examples/no-arbitrator/`) +- Deploying with a minimal number of VMs ([examples/minimal/](manifest-generation/examples/minimal/)) +- Deploying without a running CF deployment ([examples/standalone/](manifest-generation/examples/standalone/)) +- Replacing the arbitrator with a full MySQL node ([examples/no-arbitrator/](manifest-generation/examples/no-arbitrator/)) ##### Deploying From d1a27687ec90627e0e711af73f3fba36487328e2 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Mon, 4 Jan 2016 16:29:21 -0800 Subject: [PATCH 032/110] Changed default value of skip_name_resolve to true - Last release added this feature, but disabled it by default; for the upcoming release we will enable it by default. [#106676160] Signed-off-by: Morgan Fine --- jobs/mysql/spec | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/mysql/spec b/jobs/mysql/spec index 9e9b4a19..95b911f1 100644 --- a/jobs/mysql/spec +++ b/jobs/mysql/spec @@ -67,7 +67,7 @@ properties: default: 1073741824 cf_mysql.mysql.skip_name_resolve: description: 'Do not restrict connections to database based on hostname' - default: false + default: true #these two properties are also used by the Broker cf_mysql.mysql.gcache_size: description: 'Cache size used by galera (maximum amount of data possible in an IST), in MB' From d587d2ee112701b3e52d5241eb0d4f67fe6c4149 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Mon, 4 Jan 2016 16:49:13 -0800 Subject: [PATCH 033/110] Updated mariadb to 10.0.23 [#110702918] Signed-off-by: Ben Calegari --- config/blobs.yml | 4 ++++ packages/mariadb/packaging | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/config/blobs.yml b/config/blobs.yml index cc5cba38..1bdcc56f 100644 --- a/config/blobs.yml +++ b/config/blobs.yml @@ -87,3 +87,7 @@ mariadb/mariadb-galera-10.0.22.tar.gz: object_id: 6276661c-5065-451a-8ef9-36e214ce29a6 sha: 3e79ccd1cb10dd4ef87d67a163f001b0edd05103 size: 56593286 +mariadb/mariadb-galera-10.0.23.tar.gz: + object_id: 6d3ba50a-c12e-4a4d-b992-10f7051a0a73 + sha: c920df0661770abb923a7af5e964927f7d02ff5f + size: 57726827 diff --git a/packages/mariadb/packaging b/packages/mariadb/packaging index 60d8e5b4..a2c26bf9 100644 --- a/packages/mariadb/packaging +++ b/packages/mariadb/packaging @@ -1,7 +1,7 @@ # abort script on any command that exits with a non zero value set -e -MARIADB_VERSION=10.0.22 +MARIADB_VERSION=10.0.23 tar xzf mariadb/mariadb-galera-${MARIADB_VERSION}.tar.gz ( From afbc023f668d35dd6d18128d036ce65e2012073c Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Mon, 4 Jan 2016 17:00:00 -0800 Subject: [PATCH 034/110] Default arbitrator instance size to t2.small or equivalent - Based on a benchmarking investigation, we saw no performance impact when going to a small instance size [#109744230] Signed-off-by: Aditya Anchuri --- manifest-generation/cf-mysql-template.yml | 6 +++++- manifest-generation/examples/aws/iaas-settings.yml | 2 +- manifest-generation/examples/vsphere/iaas-settings.yml | 2 +- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index d17b5950..1e37a7b5 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -48,6 +48,10 @@ resource_pools: network: mysql3 stemcell: (( iaas_settings.stemcell )) cloud_properties: (( iaas_settings.resource_pool_cloud_properties.mysql_z3.cloud_properties )) +- name: arbitrator_z3 + network: mysql3 + stemcell: (( iaas_settings.stemcell )) + cloud_properties: (( iaas_settings.resource_pool_cloud_properties.arbitrator_z3.cloud_properties )) - name: proxy_z1 network: mysql1 stemcell: (( iaas_settings.stemcell )) @@ -111,7 +115,7 @@ jobs: release: cf-mysql template: arbitrator instances: (( instance_count_overrides.arbitrator_z3.instances || 1 )) - resource_pool: mysql_z3 + resource_pool: arbitrator_z3 networks: - name: mysql3 static_ips: (( static_ips(1) )) diff --git a/manifest-generation/examples/aws/iaas-settings.yml b/manifest-generation/examples/aws/iaas-settings.yml index 3ff047bf..519f89e7 100644 --- a/manifest-generation/examples/aws/iaas-settings.yml +++ b/manifest-generation/examples/aws/iaas-settings.yml @@ -33,7 +33,7 @@ iaas_settings: - name: arbitrator_z3 cloud_properties: - instance_type: m3.medium + instance_type: t2.small availability_zone: (( properties.template_only.aws.availability_zone3 )) - name: proxy_z1 diff --git a/manifest-generation/examples/vsphere/iaas-settings.yml b/manifest-generation/examples/vsphere/iaas-settings.yml index a8e71987..300ce439 100644 --- a/manifest-generation/examples/vsphere/iaas-settings.yml +++ b/manifest-generation/examples/vsphere/iaas-settings.yml @@ -59,7 +59,7 @@ iaas_settings: cloud_properties: ram: 2048 disk: 4096 - cpu: 2 + cpu: 1 datacenters: - name: (( .properties.datacenters.z3.datacenter_name )) clusters: (( .properties.datacenters.z3.clusters )) From 52876e1f5af034d1ea8dfed32b963ccce0a204c5 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Wed, 6 Jan 2016 16:17:05 -0800 Subject: [PATCH 035/110] Extend cf-mysql-template and manifest generation script - These files are now able to handle extra jobs such as backups Signed-off-by: Morgan Fine --- manifest-generation/cf-mysql-template.yml | 23 ++++++++++++++++++----- scripts/generate-deployment-manifest | 12 +++++++++++- 2 files changed, 29 insertions(+), 6 deletions(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 1e37a7b5..c0928e68 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -5,6 +5,7 @@ director_uuid: (( config_from_cf.cf_director_uuid )) releases: - name: cf-mysql version: (( release_versions.cf-mysql.version || "latest" )) +- <<: (( merge )) compilation: workers: 4 @@ -14,8 +15,8 @@ compilation: update: canaries: 1 - canary_watch_time: (( "30000-" canary_end_in_sec )) - update_watch_time: (( "30000-" canary_end_in_sec )) + canary_watch_time: (( merge || "30000-" canary_end_in_sec )) + update_watch_time: (( merge || "30000-" canary_end_in_sec )) max_in_flight: 1 serial: true @@ -29,6 +30,7 @@ networks: - name: mysql3 type: manual subnets: (( iaas_settings.subnet_configs.mysql3.subnets )) + - <<: (( merge )) disk_pools: - name: mysql-persistent-disk @@ -72,11 +74,14 @@ resource_pools: network: mysql1 stemcell: (( iaas_settings.stemcell )) cloud_properties: (( iaas_settings.resource_pool_cloud_properties.errands_z1.cloud_properties )) +- <<: (( merge )) jobs: - name: mysql_z1 release: cf-mysql - template: mysql + templates: + - {release: cf-mysql, name: mysql} + - <<: (( merge )) instances: (( instance_count_overrides.mysql_z1.instances || 1 )) resource_pool: mysql_z1 persistent_disk_pool: mysql-persistent-disk @@ -88,7 +93,9 @@ jobs: - name: mysql_z2 release: cf-mysql - template: mysql + templates: + - {release: cf-mysql, name: mysql} + - <<: (( merge )) instances: (( instance_count_overrides.mysql_z2.instances || 1 )) resource_pool: mysql_z2 persistent_disk_pool: mysql-persistent-disk @@ -100,7 +107,9 @@ jobs: - name: mysql_z3 release: cf-mysql - template: mysql + templates: + - {release: cf-mysql, name: mysql} + - <<: (( merge )) # defaults to using an arbitrator in place of mysql node in z3 instances: (( instance_count_overrides.mysql_z3.instances || 0 )) resource_pool: mysql_z3 @@ -204,6 +213,8 @@ jobs: - name: mysql1 properties: {} +- <<: (( merge )) + properties: nats: machines: (( config_from_cf.nats.machines )) @@ -288,6 +299,8 @@ properties: standalone_tests_only: (( property_overrides.acceptance_tests.standalone_tests_only || nil )) timeout_scale: (( property_overrides.acceptance_tests.timeout_scale || nil )) + <<: (( merge )) + # The keys below should not be included in the final stub config_from_cf: (( merge )) iaas_settings: (( merge )) diff --git a/scripts/generate-deployment-manifest b/scripts/generate-deployment-manifest index a032c202..00a79a66 100755 --- a/scripts/generate-deployment-manifest +++ b/scripts/generate-deployment-manifest @@ -15,11 +15,12 @@ function usage(){ Optional arguments -n Instance count overrides stub file (single node, 3 node) -v Release versions stub file + -b Backups manifest stub file " exit 1 } -while getopts "c:p:i:n:v:" opt; do +while getopts "c:p:i:n:v:b:" opt; do case $opt in c) cf_manifest=$OPTARG @@ -36,6 +37,9 @@ while getopts "c:p:i:n:v:" opt; do v) release_versions=$OPTARG ;; + b) + backup_stub=$OPTARG + ;; *) echo "Unknown arguments" usage @@ -77,6 +81,11 @@ if [[ ! -z "${release_versions}" && ! -f "${release_versions}" ]]; then argument_error=true fi +if [[ ! -z "${backup_stub}" && ! -f "${backup_stub}" ]]; then + >&2 echo "Backup Stub file '${backup_stub}' is not a regular file" + argument_error=true +fi + if [ "$argument_error" = "true" ]; then usage fi @@ -92,6 +101,7 @@ spiff merge \ spiff merge \ ${MANIFEST_DIR}/cf-mysql-template.yml \ + ${backup_stub} \ ${tmpdir}/config-from-cf.yml \ ${infrastructure_type} \ ${instance_counts} \ From 16d17ca646fddae9835e7f30037f1f68050e4eaa Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 7 Jan 2016 15:56:30 -0800 Subject: [PATCH 036/110] Add instance-count-overrides for upgrading to an arbitrator - As of yet we have not been able to make this deploy succeed properly. It reports as successful but MySQL does not accept connections, so consider this a WIP. [#110772724] --- manifest-generation/cf-mysql-template.yml | 3 ++- .../deploy-arbitrator/instance-count-overrides.yml | 5 +++++ .../remove-mysql-node/instance-count-overrides.yml | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml create mode 100644 manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 1e37a7b5..f09a8842 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -74,6 +74,7 @@ resource_pools: cloud_properties: (( iaas_settings.resource_pool_cloud_properties.errands_z1.cloud_properties )) jobs: + - name: mysql_z1 release: cf-mysql template: mysql @@ -298,7 +299,7 @@ empty_hash: {} default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) -- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +- (( jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) default_proxy_ips: - (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) - (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) diff --git a/manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml b/manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml new file mode 100644 index 00000000..8dccb1a5 --- /dev/null +++ b/manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml @@ -0,0 +1,5 @@ +instance_count_overrides: + - name: mysql_z3 + instances: 1 + - name: arbitrator_z3 + instances: 1 diff --git a/manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml b/manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml new file mode 100644 index 00000000..1e9fd192 --- /dev/null +++ b/manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml @@ -0,0 +1,3 @@ +instance_count_overrides: + - name: mysql_z3 + instances: 0 From 657f482d699087e949f0ee096a002a758bf185b9 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 7 Jan 2016 17:21:23 -0800 Subject: [PATCH 037/110] Added back arbitrator IP to cluster IPs [#110369648] --- manifest-generation/cf-mysql-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index f09a8842..2bf3bc4d 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -299,7 +299,7 @@ empty_hash: {} default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) -- (( jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) default_proxy_ips: - (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) - (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) From aded2da9d370e49ec799933a04e46bdcc23d4ba8 Mon Sep 17 00:00:00 2001 From: test Date: Thu, 7 Jan 2016 17:23:39 -0800 Subject: [PATCH 038/110] Revert "Added back arbitrator IP to cluster IPs" This reverts commit 657f482d699087e949f0ee096a002a758bf185b9. --- manifest-generation/cf-mysql-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 2bf3bc4d..f09a8842 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -299,7 +299,7 @@ empty_hash: {} default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) -- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +- (( jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) default_proxy_ips: - (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) - (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) From 94978f1b4087b456e9c60a6e265efb71a9ce29ea Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Thu, 7 Jan 2016 17:24:51 -0800 Subject: [PATCH 039/110] Added back arbitrator to cluster IPs - Replica of the commit that was just reverted, but with the correct story ID. [#110772724] --- manifest-generation/cf-mysql-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index f09a8842..2bf3bc4d 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -299,7 +299,7 @@ empty_hash: {} default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) -- (( jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) default_proxy_ips: - (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) - (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) From eaa1fbb9bf6c00ed769a05cc3b58c320d9d194db Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Thu, 7 Jan 2016 17:28:14 -0800 Subject: [PATCH 040/110] Allow additional jobs and properties to be merged in manifest - Allows easily colocating jobs, adding new standalone jobs, creating new top-level properties, and specifying multiple IaaS stubs [#111004656] Signed-off-by: Lyle Franklin --- manifest-generation/cf-mysql-template.yml | 301 +++++++++--------- .../examples/aws/iaas-settings.yml | 1 + scripts/generate-deployment-manifest | 53 +-- 3 files changed, 184 insertions(+), 171 deletions(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index c0928e68..0d2ee4fb 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -2,10 +2,7 @@ name: (( config_from_cf.cf_deployment_name "-mysql" )) director_uuid: (( config_from_cf.cf_director_uuid )) -releases: -- name: cf-mysql - version: (( release_versions.cf-mysql.version || "latest" )) -- <<: (( merge )) +releases: (( base_releases additional_releases )) compilation: workers: 4 @@ -30,14 +27,133 @@ networks: - name: mysql3 type: manual subnets: (( iaas_settings.subnet_configs.mysql3.subnets )) - - <<: (( merge )) -disk_pools: +disk_pools: (( base_disk_pools additional_disk_pools )) + +resource_pools: (( base_resource_pools additional_resource_pools )) + +jobs: (( base_jobs additional_jobs )) + +properties: + <<: (( merge || nil )) + nats: + machines: (( config_from_cf.nats.machines )) + user: (( config_from_cf.nats.user )) + password: (( config_from_cf.nats.password )) + port: (( config_from_cf.nats.port )) + app_domains: (( config_from_cf.app_domains )) + domain: (( config_from_cf.domain )) + cf: + api_url: (( config_from_cf.api_url )) + app_domains: (( config_from_cf.app_domains )) + admin_username: (( config_from_cf.admin_username )) + admin_password: (( config_from_cf.admin_password )) + skip_ssl_validation: (( config_from_cf.skip_ssl_validation || nil )) + smoke_tests: + use_existing_org: (( config_from_cf.smoke_tests.use_existing_org || nil )) + org: (( config_from_cf.smoke_tests.org || nil )) + syslog_aggregator: (( property_overrides.syslog_aggregator || nil )) + cf_mysql: + standalone: (( property_overrides.standalone || false )) + external_host: (( "p-mysql." .properties.domain )) + host: (( property_overrides.host || jobs.proxy_z1.networks.mysql1.static_ips.[0] )) + mysql: + admin_username: (( property_overrides.mysql.admin_username || nil )) + admin_password: (( property_overrides.mysql.admin_password )) + port: (( property_overrides.mysql.port || nil )) + galera_port: (( property_overrides.mysql.galera_port || nil )) + healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) + bootstrap_endpoint: (( property_overrides.mysql.bootstrap_endpoint )) + cluster_ips: (( property_overrides.mysql.cluster_ips || default_mysql_cluster_ips )) + seeded_databases: (( property_overrides.mysql.seeded_databases || nil )) + database_startup_timeout: (( property_overrides.mysql.database_startup_timeout || 600 )) + max_heap_table_size: (( property_overrides.mysql.max_heap_table_size || nil )) + tmp_table_size: (( property_overrides.mysql.tmp_table_size || nil )) + wsrep_max_ws_rows: (( property_overrides.mysql.wsrep_max_ws_rows || nil )) + wsrep_max_ws_size: (( property_overrides.mysql.wsrep_max_ws_size || nil )) + skip_name_resolve: (( property_overrides.mysql.skip_name_resolve || nil )) + gcache_size: (( property_overrides.mysql.gcache_size || nil )) + ib_log_file_size: (( property_overrides.mysql.ib_log_file_size || nil )) + innodb_buffer_pool_size: (( property_overrides.mysql.innodb_buffer_pool_size || nil )) + max_connections: (( property_overrides.mysql.max_connections || nil )) + persistent_disk: (( disk_pools.mysql-persistent-disk.disk_size )) + proxy: + api_username: (( property_overrides.proxy.api_username )) + api_password: (( property_overrides.proxy.api_password )) + health_port: (( property_overrides.proxy.health_port || nil )) + api_force_https: (( property_overrides.proxy.api_force_https || nil )) + proxy_ips: (( property_overrides.proxy.proxy_ips || default_proxy_ips )) + arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) + port: (( property_overrides.proxy.port || nil )) + healthcheck_timeout_millis: (( property_overrides.proxy.healthcheck_timeout_millis || nil )) + api_port: (( property_overrides.proxy.api_port || nil )) + broker: + max_user_connections_default: (( property_overrides.broker.max_user_connections_default || nil )) + ssl_enabled: (( property_overrides.broker.ssl_enabled || nil )) + auth_username: (( property_overrides.broker.auth_username )) + auth_password: (( property_overrides.broker.auth_password )) + cookie_secret: (( property_overrides.broker.cookie_secret )) + external_host: (( property_overrides.broker.external_domain || .properties.cf_mysql.external_host )) + protocol: (( property_overrides.broker.protocol || nil )) + port: (( property_overrides.broker.port || nil )) + services: + - name: p-mysql + plan_updateable: true + id: (( property_overrides.broker.service_id || "44b26033-1f54-4087-b7bc-da9652c2a539" )) + description: MySQL databases on demand + tags: + - mysql + metadata: + displayName: "MySQL for Pivotal Cloud Foundry" + imageUrl: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAJo5JREFUeNrsXQtsVNeZvvPwzBiD3zyNwxgWhEKqmEibR9skdhQtaao2Rqtks4+qthKp1e5KxKhVCg3BbrKOst3lIW2rrUQVR5W2adgKp92ikG1ik2RbkpWCs4WWQomHVyAY8NhgPDOex97v+F4zHs97zrlz7z3/Jw1j/Ji5c+/9vv9x/v8/DoVgK9Rt7/arT37tv7XqozXT7zorK1clotFgYnp6PMtLDquPoPZ1YKxvd4DOsn3goFNgOYK3asRuUx81SQRvM/hQhpIEYlz7f1AViGG6SiQABH6WHMS+U/u61SKHDxGAp/CxJgzDqjAE6aqSABAyE75NI/iD2rPfZh8xoAnDYdfCquEr218coqtOAiC7K9+hEb5N0tMw5PBUHE5EpgcodCABsDvha5MI36HF8YRbQIgw4PR5D8dD4QEKGUgA7BLHg+yPSWzli7sx3e4hxeV8w11bO3B5y7YAnRESAKtZ+i2KdZJ2pobT6x2Oh8N74SGQZ0ACYFbig/Rf18hPEHGzulzqP44BR0XFq1d39A3QGSEBMIOLD0vfSTG94WIQdHi9/a5FC/dSiEACYCgaXtjeEZ+aImtvnnzBQCIa3auGB0N0NkgARFp8WPqdiv3W6O1xM3s9gUQ40qsKQT+dDRIAXqSHa/+M5uqTm2+R8MBZ6dvrXe3fc+HJpylpSAJAxCchICEgASDikxCQEJAAZCF/DxHf/kJwZfuLPXQ2SACSid+pUHJPnpve6wk4fb7eK8/29pMAyE38No34bUQLCW9+t3vI3VDfe3nLtiESAPni/N3KTAEPQe6wQHF4PP2+tau7ZcwPOCQkf6dGforzCXPzAwurumULCxwSER/x/Ssyufuwbs5K35zvOauqsv5NfHJy7v+nQkoiFpOGEE6fd8hVU9MlS3mxQxLy9yg2zO67Fqpkdjox3FMlu3OG7M75pOcFXRxmRCGuPk8pielp9n+bIeiuq907+u2dPSQA1rf6BxSLt+UyS161gBEd5Hb6fGrcWmGqY2SioIkBhMIOnoOWJLS1N+CwMflRzLPTilYf5Haprroak7Jns5G9EFGIh9THjUklpopCIjJtydyAq66md3Trjj0kANYgfq0W63dY6CZTXNWLLE/4XIAAxCYmlBgEYeK6xXIDvgHfujVddlspcNiM/G2ay296q8+sfHW14q5ZlDMxZ1dABGLjE+zZCuGCw+NBbmCzneoGHDYif4/m8pvb0tfVKupNJCxRR2JgiLfWa5cEocMGxDe9yw/33l1fx54JuREdC86KgWmv6aKFA96WVZYPCRwWJ3+rRn7TZfnh4rvr6hR3Y/3MLDtCUTkDiEF0bMyUCUSH2zXsaVrRdekbW4dJAIwnf4dGflPF+1ib1918Al+vIHrlqulqDpAXUK951+i3nh8gATCO/Fji222mYwLh3fW10ib0jAJqDKY/G2UrCabKC9TVdFtxqdBhQfLD6neaifgVS5fYdunOzOHB9GeXmWdgmnuhvq5f9QS6SADEEN9UyT4ivnmEIHL+gmk8AqvVCzgsRP5BxQTJPsT4FUsXk6tPoUFmEahaMOxb09JuBRFwEPnzPFGqpfc2NxHxTQ4sHUY+vVj2VQOIQHzyZrvZtzJzmJz8ID0q+/xlO0Eul+JubGBWn2AdwBvAqkE5i4rUeyfgWblis5mXCR0mJz8sf9nW01C441mxnOJ8K+cHVG+gnAVFWCb0LF/ablYRcBD507v7ID5V7tkoLDh3oWzegJlFwEHknwvd3afqPZt5Ayr59bCARMCkAlBO8oPwnuYmsvrkDUglAg4TkR+kP6qUIeHHYn2V/GT15fEGIAJlyg2A/KZZHXCaiPyD5SC/Z8Uyxeu/jcgvEXCtcc1x7cuAVmfVgsGm1/aZolnEYSLyG7rOz9b1V91GffmSA81F4TNnDa8bMEuxkBk8AMPbeVHN51u7hshPYPcA7gWjcz/xyZutoZOnXyn35y+r36s19jxp5Hsiy++9baXicDrp7ifMeIPqveCurWEOceq+CEJzEdHo+kWPPeq/+dbgG9IJgNbS+x1D4/3mJqViSSPd8YSMnqHD44F1VtmZMCoEaV20+cvjNw8NHpEmB6AN8zhg2IdE0me1n1x+Qv55gU8Chi0VsnkCNdWbyzFUxFEG8hu61g/Se1Y2EfkJhbnnkWmWHDRqAlG5agScBpNf7+k3jPxk+QnFEbLC0HsnEYnURi58+orRy4NGZ8IMy/jr5Kf1fUKpoSPbg9EIEYjGWsMjZwxdGTCMHdrc/m8a8V6Y1uNBpp/ITyhVBLBCoN5PCAmwzZkBnsD6RV/9kuPmfw8N2SYHoO3YM2gY+Zub6M4lcAfKh42YQajNoGg3YgcihwHkR0wzYkTcT+Qn2EYEPJ5g5fq1LaIrBY3IARiS9NMbeggEkcA9ZsSeD0gKGlEpKDRI1op9nhH9IVjCDw09VN1HMACummpWLCS6f4BVCgouEnIIJL9fmWnvrRVOfsr2F4zmqoXKyqrs9e8TkYhyPHiVTlY6csZirFhIdJ2Ael8H3Y0NGy9v2RYQEjYLPPYDRH7jUV3hUTbUNcwSfENtPftetUf9fm1DSa99fvKGcm5ypof+t6OXVIEIqwJxTf3+dfX7N6Q6z/oSoWgRUIWmNnr1GkKBdst4AEZs1U3lvTO4b8nymcfiZSrpF6mkX1iW45iYVr2FsatMGNjz5Yvse7b3BNQwIHTqtPCyYXddrZAtyR0CyG+I6y9rOy8s+6amVYz0eDYzED4cunBWOXT+jK1DCYN6B4IVS5dwDwVEhADCs/7IxMpG/ida1iqP+9cy4lsFCDnw2LphIwsf9p08prw+csp2ngHuxYoVy9gSoUDUxsbHuYcCXD0A1fp3agIgzhVqbCjXKKeyEX/rhrvK5tqLCBX2nTyu7PvjMdsJAaYOY8NSkXDVVHddeba333QCYETBD2qyEffL4urvuvsBS1n8QgCPoPvDd1muwE4IB84KHTaKVYHKDeu5FQjxXDjfLZL8bLOOVc1SkB9u86FNm21LfgAezf72R5mHYyew6dICd5LCqkDo1Ce7uYUvnKx/m/rUKfLEYoCnLMt9P/7iw2zpTgbAy2m2SXijWWh2r4pEIhLpXLL3pTYzeQBCl/wQ88uS9INFXGkjQuQD5DjsBDaERmCeCqsN0avXdppCALTEX5uoD4u4H4k/WZCrOs+u4YDdgHtW5KThRDTa1vjyzs6yC4BI68+265Ik7ifYD6J3m4qHQjvLKgBaxZ/fqieQQBCdDxDZoZoIR/yNfc/1lEUAtGW/LcJcf9V9oo06CVYH7mGRIWx8KrSllDmCpXgAaPMVsuwnWjkJBCMhcrt5LAuGPwk8Y6gAiLb+5PoTKBQwxgso1gMQZv2R9ZfZ9UcnnXSfOXhNilBA1H1dihdQsACItP5MKVfK7fofunDGduWx2YB+gF3HPpLis3pWLBfm2RbrBRTjAQiz/kiWiCyjtAqeev/XUkziAfkff+egFHMDmIFT721RCcFivYBiBGCLqJODZAnhFjHQNWdXwMvZdOiAdCPHWEJQkJGDF1BwaFKg+9+pCNrO29u8UnF6vcR+DeF4TBm6dJ5N2EGtfLNNKgTRBdgzfETpOfqBNJZ/ntX1eJRYcFyAG5DwVd5/35nQe0fy3l+w0IEgQqr+ZE/85bKUj6sPiMBT6+5gvQJWbBTaP3JKeT1wSqr8Rsb7HQlB9Z6P3Zjk70l7PeBof96/X4D1F7alt2+NX3FWVRHb84Q+Duy+JctKHvQp0tKD7G9qSU1ZrX1Gd31yUgmdDojxMCorN1/d0ZfXVuOFeABfF2X9ify3AAs/Homw1YBsXoFuSeENQBAwCRiDQfFcDg8Bx4PlvN8HZwaC5jslGAK2aeUqaVYCZkmq3vOivIDE9DS4mpcA5OUBaIM+R4RYf0mHe2bC1jvuYjP0QKJdx48W7TJDFPQR4ez/i5fN+VkhQKIOewTMfH2NWXN9FHixI8ER0qAN+HFV8FiYM3hQumuNicJTJ04Kee2KpUta8hkgmq8HICTzjy2WiPyZCbxffYAc+9XYGcM0C7XIQDZPolyf6wn/WkZ82cGWBVUOiNhrMHb9BjjbzUsAOoUIgER9/qUQBo+ejfey8dqIqc1G6nzcfBD+kaZV0g07ycNSCxGARDjcyUUAtOQf98IfFvuT9c8bcOdBIt1ysjzA6KU5+QAzER4JSl28ZBlvVqwXICIXgMKghhe2d+RKBubjAQhJ/lHRDx/PQNmwcTZOR+YdMTr6CRCnixYGEB1bjuE4mhcsZNbdzoNMxXkBi8uWDHTnsP6w/B28DwyWnzL//MnIMuppdgtKTuLBayhKcEpIIhJy8KFqxhvmvsdgItEBDo/17Q4W6wF0iPjAFPtnBjbcFCEOZiYv1QjMcIL3zkLaVmXgcH9G8cnxGtyz/3rmk5Aedtw6Kxfs3POQtwConBDRI+D0erNy2JnF/ferT638P2gdsTyHNew5ekSaz4sSYSoPFseNeDjcumTvS/5iPAAx7j9Z/7y8ALQE290TQKETtgcjiOVGNBjsKEYAHuN9IGiCoH7//IC1/nt/+TNmIe2GmVbgAenKf/MJj4U0xcXiGbnsyOD+Q4rGeB8H5qKRB1A49LJZ1MxbeU0dooZ4n1z+LNZ6LChqm/G6dKsBbqPcf4xCIvIXh3PaTrrVRz3KI6oIYKkv3XKfGYElSHgxIH8xPQMyhgHTn17SM/jc4PR5064GZBKAB0W4/4T8rH0moiAngPyA3hegFwNhjd4sy3vHWTfgrQrFfPIY2T6zjABXeJcHx0PhBwsRAO4egKummq5sHni8ZR1rloHFz+Uqp5YBY70f1XjoANxQWz/bDSgibNDfF4VFekdgoa49iI/dgdnnlrAbMBtXBPQHgNNdOXMAavyPpb+jvN3/yg3r6crmAb0dWCdZMZ2A6ZDcGjwjFqpAeLx5Ex0otvU3FakdgbK2A2fD1PET3MMAFRvH+nYP5/IA+Ft/cv+LJgoeSACiCxCxdLFDNFN7A4xOxMHaI28B0pt1ipHdwwCHpwLczikA/ON/cv9LAtz6p9dtYA80/Ohjtsw+aksXsE1NtxHpTRAGJCLTD+aTA2jj7f6TByBGDAB4BL8fu8ae0QlYriU21oxUV6/cXjuTf6CGIVN6zW1ZBUCN/9t4v6OzagFdTdHEw8ANZe0cdx8twfAWzt28Mc/l19uFC7HkqbkDllNQv8a4chryIU4EYhPXub5mY99zbVe2vziUyQPgXvtP7r/x0AeFzlWKjXRirCYAGBTCWQBiNybB8VkBcAqP/6nvvyDIWCVX7IwC+3sAQoznHI47RXoAqG2m2v/CBcBqM/9KAcKUfX88RhfeOP60phUArf3XT9a//Oj+4F0pRACJSxm6Hk3GIb/W6zPPA/DzfifnQhKAYgBCgBiYC2BXcqApCBugyrY5qEk41JpOANoo/jcfSfSWYLsIATwbtALbWdy4CoBPyOTsWa4nrwLcyTV+cbko/ufkDSR3Aj61boPlimoQ52Nj0P0jJ6npp1ABqPQxLnEuC74znQD4eR84ga8Q6J2Aelmtmefu6/sWYDMTcvNL5xLnseH+dALAdQWAxn4XB73CL9ugTFhR/Fz/ndQqPKM3CAXZZ/YkKLwaUW8MotFg2bnEWQBa5wiA1gHIXbUIhQNVdugGfHrdHcqu4x/l1Qk4QzxY2VPzyJX8rG/eke530pE61QMBudn7aZWEhVYUpr4vPud92v6HBGO5BM6jM1D3ALiP6hGUvJAGK7VeeXQCFhs/z/bsm4Rg8EqsmscoJxwVQnJptckhQBv3g6YEIDchgKXEw4rjtXTSW2mMmQwegMb5IV0Aani+sovW/4WAxfobG9hOwee1CTy/Hb2ouuLXTJNoQ4LydrY56HK2QShZen4iwHnrsJpkD4BvDsDppCtmgGeQvFuw7urrHYA84vRcMbz+rHcGGp18lC4M4CsArckCwFmtKumKlQHZevB1MZjzfy2pl9GapyQNqfW3nB5AJffOQGE5AIeLPIBigbl7omLxVIEwS0x+noqDysEpxnkhTKUlwOLx5vkz0pXIYpWDUB5OObUuQM6v6qIrViRAfjTJyCICW/MYf04QA3AfHgB3ASAPoDQgo2/3Tjm94/H1EbL+eXFKTGWtn4J1E4sA20Dz+FHbeQOoZUCXo0yDT8wKJAG5VgGic4nAD9hBFxNznmhZy8qDrZqF15uZfnzyGHUElsAtzl2BtRAAvk1A5P4LIY/e/MMmAKti8EjTKtOLAY4b3YDYx4CsPR9u8W4KctNptV5ocPzoVTZQA1V3M5uDLldur6sve9Wd3hE4s2nJJWoDtkgIQLAo4EqfG5m7d2DyBqF6IQ/PAp7k6kIUEhW7MSiBBIAgykNQH5lcbngNK6sK23VmIhIha04CkGecQoNAzO81UBLOmjkA/oNBFFoGJBBkFhU6BQSC3AJAUxoIBDlRQx4AgSC5B0AVGgSCnBgnD4BAkNwDIBAIJAB8EJ+cpLNKIAiACG6RB0AgkAdAIBBkBPUCWAzYOxDbh6UiWzMO1fITsgnAMNc4he/sckISdt/9wJx9AOZgw8aiX5ftJZBlGrG+10A6YGAJwaAcAH9usb0BgzxfkfPEEoIyM9J7/0OPCuv3X1m1sOh2YaO2/27Wjk/mRiYB3ApSCCA5+UsFPBIMKBEJjEPDRqm6N4IJQwh5aMpQ6XBoY8FHeL6ob42f2oIlIL9OyHv/62fCXh/7ICLvkQkQAgxOtftAEiwBhk4HeL9si3Osb3eAqEbkLyV8EHGM+Pw//uLDWckPYCQakpyEwgHuC1kGpERgaQCh3tq02TI76/IeTorPDfHLZ+syWH8ZVjhEcUrPAQwpHPcHTMTixOISb36r7LKLEIBnLA6LDsufz+cH8WVZhRDAqaFkAeCsVlPEZAnID/Dc1y852ZcLGEr69Pu/lubeEFVirwvAMFcPYHqa2CwB+QFsWsIDWWsc0rn+quWXaUlQwBLgcLIAjFMOoHyA5UO220rkZ7P/Ry+VvG1ZMclOvDc2SZEJAjg1npoD2Mn7gGmXIL5ur2hgzz7dpTdiWQ2kR7xfSBIRgoMdhYn8fHMAQd6vzsIAEgDLkJ+RfvSiYevpyPDvvueBgr0e2Vx/gSE14zxbBhzr2z3M+9UpDLAW+QGjiIW1/Xwz/cnAaoNsrr8oLumcd6ckBbhtFDqTtVxMTE+DrXfcpWwtoXlHFHo23qPt73ctZwyeDfrWYeni/d6N9xaU7Et2/bs/eFfK+0XACsCswU8WgABXASAPIC0KzXYbCcTkeOQswCmDeIH8pSYcyQOYw3VlNgTQ8DHXuCUWIxGwEPnNjEMSby8ODglYAvw4nQAMcT/4EAkAkb80yOz6C+TQ0DwBqFi6JMD94G/QgFDEvUR+cv1NxqHheQJwecu2QHJswAMxySYEI7mX2r2G2nYif3FAxl/2nn8BHAqM9e2eXfZ3p1EGP7c8QGSaPRyeCqlcfFis10dOzcavKLAhESgMWI2QfdyYzh9R1j81BwAc5q5gExO2d/EPbeqYQ3Cs7yeXtnZ/+K7tB1Zwd/0/lNv1F8idwxkFwLWwintBUMzGeQC9gSddHXvq9596/9c0mbcA158EUxh3MnsAV7a/OMRfxa7b8uIgts/WxALPYNc9989Wu+ntq7JbNXL9y8sdNf4fyhYCACQCOYAy3v3tuVt3dQ9BB0ptH3/nIIlAFjxFIimSM/O4PU8AHJ4K/nmAcfvkAdC2W0gNP0Rgd9LvIwzoOXqEmJ4Gsoz3KiNnDucUgERkeoA8gPTAEl+uIZXpgARh8t9hhYBEYC5kGu9VRs7M47Yr9Ruh945cqrz/vmfUL/n18iYSirOyUnF6vZa+KH+aGFfalq9UlvgWFPy3+LvzN2812nx0dVRprlqkbKhroLtdxdcOv6WMhmiUnE7+6Bj3Dv2gGv935/QAMikFhQHaMIoP3is6RkX4kLo8SC4vuf4GcSUtp9MOBXX6vIfjoXAnhQHpXVUk8rD2Xyj08VebDh2Y7b3Ha1lh/n+yCKLdt7RzeG2OiJLrbwhX0ub2HOm+Wbe9u1Z9GuN9BJ7mJsVdV2uLi1TKQA9dRHQSFDsQFHmEXL37qTg/eV3q/fXMDrj+kXMXRLx0XXIJcFYBAOqf//ZgIhpt43kErupFitd/m20uVimDPVDo8vjgwdn/QwTy9Sr0DjnaG89+CAfOivAAhlTyt6f19jOz1fmGCNdGQG1z+WJX1XXdP1LcXHwUEqUuD+Yz7BLkZyEIkd92ADcEuf8ZuZxRANy1tQMijkRAdrOsKCWRl255MNvMO7zPvb/8GSXMbOz+C0JGLrsy/WDyzbeDVQ+3dSRisWVcVS4UUiqWNNrqwv3i7CclLQ8ijj99fWZrhqFL59MuD8LiU5WcvRE5c44tmXPGsOr+v1ywBwDEw+G93N2cWMx2XkCpy4MYj528CrCTJfduWXmEGUR++1t/AaO/gKwczrU78IDD5eL/Ya/Yz4UFYZ/Kc686NLwgCXhey8bry4PJjUOI8/Fz5AW6P3yXGGJ3ARDHiayhfFZ2h947Eqps+0KrEo+v5+oFRKNoPVYcHo+tLiIIi2q/bFN12XCQwYPK/sBMvA+yIwzwqkKLZ4QT4XiMPfDzQpf5CNYDxn5PX74ihPyq+/9qKR6A4qioeFXEkU1/NmrLi5ktkQcvIdWaJ/e+s8ahex4gRkgGgVzIyd2cAnB1Rx/CAO5BO4Yd2HVsOAp00i0PHrpwNu3vY5NNHWzLrLtJBKSx/ioHBA3+QO3/QMkCwLwAr7ffYnFP2ZGayNNj/XRI/T6WB5+gGYIU+5eGvDiblwC4Fi3cK+TDI/Npo8KgZCQn8nSA1Kk1/yB/OrKjzBjFQgR7W3+BK2J5cTavFD9qAha03889GQigLsAu/QGpQCLvt5cvKY+tWs2SfHjga7S9Vnu8yudVgv/wvoeUxb7KeX/LtupWH3gNgj0ROXdelAFE8u9H+fyiO99XRDIwEY128D5SlguYnFScVVW2vMgs8ffBu2w3XN3i52oiQmKQBobY3Pqr97zAgbl5e+x5L/JPDb53YsHDD3YqsRh3c439z+3qBQCo8su1PKgD6/4/+MP/EUPsbv3PfyrK+gfSDf4oKQcwS9RwpFfEEUMJ7To9WAfc+WyNQyxnMHhwdkMRgn2Be12g9S+Io45CX71+x7fGEgK8AOweVLl+ne0vPkKBVE8AiULaN0AeTJ04Kcr6Y+mvrpA/cBb6Ds5Kn5AVAZwQuxYHJQP5gGSi4+u/OHSAyC8JcI8LXPkqmJsFC4B3tX+PiMIgAGuidl0WnOPqa9OAEBJsOjRATT6SAPe2wHV/cHJPoX9UcKfP9f/8RajqkYcq1Q/Txv8MJdSTFFHctTW2vhGwtIea/9cDFO/LBIz6ioeEVb++rLr/bwr3AER7ASxBYvOEIEBz+eSC4Pu6KOtftABcePLpoKhcgK6UgnqjCQTjXX/1XhY06HM29k838FOYAIj2Agw4YQSCoa6/QINWtPUvSQBEewGyhAIEcv3LZf0BR6nvXt/77EgiHPGL+GSYRuRbv1YRMZWIQDDC9Q+dOCXS+qPqr6WUF3CWegROn69X5AmkUIBArn9GlMw9B4+jELGJSDI8K5Yp7kbaRJNgHWC9P/LpJZFvkXGzD0M9AMDdUN8r0k3HibTr9CCC/YB7VTD5uVh/bgJwecu2IYfH0y/y04bPnKWlQYIl4n7cq4LRr1r/IdMIAOBbu7pb1LIgO7GR6ZmNEwgEM8f96j0quJwdHOvm9WLcBIAtCy6s6hb5ydFCaYBrRSAUHaoKbPPV0V3Ksl8qHLyPruF73xmMh8JtIs+AnbYZJ9gDArf1TgaXxF8y3LyP0FVT0xUPXT6qfimMoTjRTp8Prcl056UBxk3diknjRSdQMaPBqT5m3UWbjm0r+Xwj6See/LD6Xbxf1CHiSBd/v7dHVcSdIs8GVh28q/3SiQDiy8R0hN10M+SeUu9A9TkSMayVekYYPKoiONXzX6leCye7Do4KD/uZbOQPfxIwIkHdq1r/HksIACC6NoBZJPWmgwjYtVIQlhw3GCP79LQR8SUfLxDbvlVUsOvDHjb1HFjGXyW/AUvU3F1/YSHA7As31HdFr1w9KmJ8WKr62kEEGNFD6gNTkkMhS9c9zArV2FyxZmGbKg52CN8MJL8Q11+4B8BCgV0vPBO9cm236DNkRU8A7npsYkIbi35TuhoHXCtn1QLmLbiqqy0VOhhIfgBZ/z2WFACg4XvbDqgWrUP0+2BVAKsDpraM6AwbV0mvuvZ2H31WTF7BpYYKrppqVRAWmfpYkfATuKNPMrDBx2ahvBH9CXzr1nRNnTjVlohEhK7b6RfETCIASzFLemptzukRRSPB2esIEdDFwEyenYHkF+r6G+YBAEv2vtQWvXJ10Ag3t9yeAJGeP8wiBgaSH2jnVe6b9dwa8Ukm33w7sPArmxxqKNAm+r2QQIM1QbLJ4XQadnMgY4+Rz9MXLiqx4Dg2USHm8hJV9VwyUb02pn4dVhxulxoyeAwVdVxXA8mPJb9+Q0IvIy9k40s7DsSu3+gw4r2MSAwya6/eFNMSjDM3Y86gorFBcaken+hrbGDCz5C4fw5PjLxo3pZVXap6DxtikQUWaMDawx2cOn6C1X8T+cuTM8C5Z9cA47YnJ+1A/mEj4v6yeQDAsh/tao1c/GxQdFIw2RPwrGzisu4MFxAW3yoFOdLlCrCkqHoEPPpEWHnv+QtGkj+oxf3DthYAYPG/fK8jNj5xwKi171LLhkH86c8uk6W3UniwdEnRQmBgeW8yNqvkHzBcNMtxgW6+NXhiYceXxuM3px4xxl9MKNFrYyxxlK8I4OJHR68yKwCrr8TixCyrQL1WSBpGtevG+hTyTAjjb8IjZ9g9YyC6jUr6mcIDSPIEXlGJ2Wnke2K2IGYMZr0Jrlyd2cTRYtV5n1vepNyxfIVyW139vJ/97uIF5djFT5WzY9fk8whUD7Bi6eKccyWRUxC4d18mYLpPV9nOTbkvjlGVgqmxomdV87zssRVdfZD+m59/QHn09juUGl9lzt+HAPzPyGnlpx/9r/L+yJ/m/fyLLX+m/OLpv+dybPXf3WqJ0IBNnz5zrhy5HUMz/mkNYrkvCioFQ6dH/PHJm62GeYjqhQ6dOq14V93G3ENkkMMY4Wwh4sPK9z3awYhf6N/h8dd3/TkTgn/4+U+l8QrYyoF6nSHy3uYm1qXI4n3MmzT+2hue8TelAGCUWNNr+9pVERg0UgRwwSECTAAs1nkHK/2Tv+vKy+JnA8IFGUMCdu1PB8p57UH+dp6jvYqF0wwXBCKgkr9dOzGGwmrkh+WGi14q+YGDvz+myAzZyW8KD0AHTsiyH+3qMrJGwGpAvN/35czpkvHQFCO1nvDT8YXVa5TPLWtiz8nC8e+/eZdOqrFgDT5mIb+pBAC49I2tw6oItJMIpMe//eWTGS0/yPzy24eYCKRCT/bhb5EzePahTez/EIp88dV9P0ybNCQURH7DC30sJQAkAtldf3gA6YBEHrL6uQBxwO/BS8j0WgR5yG+aHEA6EfAsX9ru8HiCdO/MQLfaqdj+q4G8yJ8qBGTNifymFYBkEVDKkBg0Y+yfrrgHy3gUx5saw2Ymv6kFQBcBnEBn1QKpRSDTWv9/fPQhUYzIb18BAJAx9a1pkVoEsO6fDoW6/gTDyW/6ENZthbM5Wyx08vQrRpcNmwHNtXVp3X8jgaKhfHA2eE3K4qIkoKOvywrkt4wA6CKgPm0uRwNRuZEu/h+fmjL0GLLVHyTj5XcOseVISVHWxh5bhgCpGP3W813uxvpuu+4GlC9+d+mCQjAVuq1GfksKABOBrTv2uGqqN8u8TMijFJjABcwzFbl5BwlAek9ggNUKGDRj0GygQh5TQE/2DVj1A7itfPaxTIjkYHjkzCtGTRsui7t/8cI8wqdLDIo+holQ7uYZiRKAlkr22VIAgNnk4Pd7e2IT13facY+9c2Nj8wQAiUF8r5B6/lLw3V+9QdWDGhyeit5rPf/cY4fP4rTLRRn99s4ed2ODLcuHMxEP/QEEw+P9druQ31YCAFzesm2ocv3aFqfPN2Cnz3XwD8cyCkC6JUKCMJe/xYjtukgASgwJrj7/0mZtqdAW3oA+xy8VWAn4yd920YqAeKuPJb7NVo/3pRCA2ZBg6449akiw0eF220KxMxXXIA9Q6IQgeA4jO/6JhCM3cO9stOoSXz6wdTXN5JtvB6cO/+bVRV/9EjYmxbxBn1U/C0psQfa1i5fM+9nSRdVK1z2fV3wVFcq54FjaoSD6INB9f/U15W/uulvxuSuUd079kb1uut9LBfoOUn/X5lb/ZRT2hN47YutaE4csV3TJ3pf8sfHxV+KhcJtVPwMsNqx9rhoAhAwQAh1YMkyXK0hXtptpLLhEE4Fg9bG8F5Dhw7plEYDLW7bhgrY3vryzM35jcnciFrPctCFY9n/8+Wss7s+W/NNHf+cC5gQS5sX6/TJ9aKdsV/nKs739lRvWtzgrK/ut2E+Adf8H/+1fuXQD5tvhJwFA+hbZyC+lAABspWBHXxerG7BgkhCewFf2/aCkTT0gIPAmJAeuPUp5u+yY4accQJ5gYUEotDMRjvitePyI2zE1CLmBL7SsyZgXwKhwxPGoK8gkHFmTgPYp80U42CujxScByCYEfc/1xKdCW6yYH5gX36tigKQhle/Oi/P3qsTvoVNBApAWTa/tqw1/EnjGLkJAuEV89bFHVlefBICEgIhPIAEgISDiE0gAikLd9u5Oh9dj2WShBAgolNwjATBACNocbveWRDTaQWfDFECn3l67deqRAJgcrLz4+o0tiXC4k8KDsrj5/RrxA3Q6SADKioYXtnckpqe/riQSHXacSmQya/+qlefwkQDYOzyAJ9Dh9Hq3xMPhVjojXIDhm0jqDVBSjwTAUiFCNBjsUGLxxxLRaBudkYKAeP4NjfTk4pMAWN8zcPq8HfFQ+EF4COqDcgbzY3q49YfJ0pMAyCAIrQ5PRUciMg1BkNU7GEoi/DDdFSQA0qKx77m22I1J5AwgCHj22+wjBrRYHoQfpiU7EgBCjpBBEwJ4B3dqgmCVpOKwRviPNSs/TC49CQCBU+ig5Q8gDDVJomB0GDGURPZx7f9BcuVJAAjlFQh/UvhQy8FrAKF16x2gjLy98P8CDADMneNqQ66ddAAAAABJRU5ErkJggg==" + longDescription: "Creating a service instance provisions a database. Binding applications provisions unique credentials for each application to access the database." + providerDisplayName: "Pivotal Software" + documentationUrl: "https://github.com/cloudfoundry/cf-mysql-release/blob/master/README.md" + supportUrl: "https://support.pivotal.io" + dashboard_client: + id: p-mysql + secret: (( property_overrides.broker.dashboard_secret )) + plans: (( property_overrides.broker.plans )) + acceptance_tests: + smoke_tests_only: (( property_overrides.acceptance_tests.smoke_tests_only || nil )) + standalone_tests_only: (( property_overrides.acceptance_tests.standalone_tests_only || nil )) + timeout_scale: (( property_overrides.acceptance_tests.timeout_scale || nil )) + +# The keys below should not be included in the final stub +config_from_cf: (( merge )) +iaas_settings: (( merge )) +property_overrides: (( merge )) +instance_count_overrides: (( merge || nil )) +release_versions: (( merge || nil )) +job_overrides: (( merge || nil )) + +default_mysql_cluster_ips: +- (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) +- (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) +- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +default_proxy_ips: +- (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) +- (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) +# spiff does not allow concating multiple fields of different types +# without adding new temporary properties +database_startup_timeout_str: (( "" .properties.cf_mysql.mysql.database_startup_timeout )) +canary_end_in_sec: (( database_startup_timeout_str "000" )) + +additional_disk_pools: (( iaas_settings.additional_disk_pools || empty_list )) +additional_releases: (( job_overrides.additional_releases || empty_list )) +additional_resource_pools: (( iaas_settings.additional_resource_pools || empty_list)) +additional_jobs: (( job_overrides.additional_jobs || empty_list )) +empty_hash: {} +empty_list: [] + +base_releases: +- name: cf-mysql + version: (( release_versions.cf-mysql.version || "latest" )) +base_disk_pools: - name: mysql-persistent-disk disk_size: (( iaas_settings.disk_pools.mysql-persistent-disk.disk_size || 100000 )) cloud_properties: (( iaas_settings.disk_pools.mysql-persistent-disk.cloud_properties || empty_hash )) - -resource_pools: +base_resource_pools: - name: mysql_z1 network: mysql1 stemcell: (( iaas_settings.stemcell )) @@ -74,14 +190,28 @@ resource_pools: network: mysql1 stemcell: (( iaas_settings.stemcell )) cloud_properties: (( iaas_settings.resource_pool_cloud_properties.errands_z1.cloud_properties )) -- <<: (( merge )) +base_templates: + mysql: + - {release: cf-mysql, name: mysql} + proxy: + - {release: cf-mysql, name: proxy} + arbitrator: + - {release: cf-mysql, name: arbitrator} + broker: + - {release: cf-mysql, name: cf-mysql-broker} +colocated_jobs: + mysql_z1: (( job_overrides.colocated_jobs.mysql_z1.additional_templates || empty_list )) + mysql_z2: (( job_overrides.colocated_jobs.mysql_z2.additional_templates || empty_list )) + mysql_z3: (( job_overrides.colocated_jobs.mysql_z3.additional_templates || empty_list )) + arbitrator_z3: (( job_overrides.colocated_jobs.arbitrator_z3.additional_templates || empty_list )) + proxy_z1: (( job_overrides.colocated_jobs.proxy_z1.additional_templates || empty_list )) + proxy_z2: (( job_overrides.colocated_jobs.proxy_z2.additional_templates || empty_list )) + cf-mysql-broker_z1: (( job_overrides.colocated_jobs.cf-mysql-broker_z1.additional_templates || empty_list )) + cf-mysql-broker_z2: (( job_overrides.colocated_jobs.cf-mysql-broker_z2.additional_templates || empty_list )) -jobs: +base_jobs: - name: mysql_z1 - release: cf-mysql - templates: - - {release: cf-mysql, name: mysql} - - <<: (( merge )) + templates: (( base_templates.mysql colocated_jobs.mysql_z1 )) instances: (( instance_count_overrides.mysql_z1.instances || 1 )) resource_pool: mysql_z1 persistent_disk_pool: mysql-persistent-disk @@ -90,12 +220,8 @@ jobs: static_ips: (( static_ips(0) )) properties: network_name: mysql1 - - name: mysql_z2 - release: cf-mysql - templates: - - {release: cf-mysql, name: mysql} - - <<: (( merge )) + templates: (( base_templates.mysql colocated_jobs.mysql_z2 )) instances: (( instance_count_overrides.mysql_z2.instances || 1 )) resource_pool: mysql_z2 persistent_disk_pool: mysql-persistent-disk @@ -104,12 +230,8 @@ jobs: static_ips: (( static_ips(0) )) properties: network_name: mysql2 - - name: mysql_z3 - release: cf-mysql - templates: - - {release: cf-mysql, name: mysql} - - <<: (( merge )) + templates: (( base_templates.mysql colocated_jobs.mysql_z3 )) # defaults to using an arbitrator in place of mysql node in z3 instances: (( instance_count_overrides.mysql_z3.instances || 0 )) resource_pool: mysql_z3 @@ -119,10 +241,8 @@ jobs: static_ips: (( static_ips(0) )) properties: network_name: mysql3 - - name: arbitrator_z3 - release: cf-mysql - template: arbitrator + templates: (( base_templates.arbitrator colocated_jobs.arbitrator_z3 )) instances: (( instance_count_overrides.arbitrator_z3.instances || 1 )) resource_pool: arbitrator_z3 networks: @@ -130,10 +250,8 @@ jobs: static_ips: (( static_ips(1) )) properties: network_name: mysql3 - - name: proxy_z1 - release: cf-mysql - template: proxy + templates: (( base_templates.proxy colocated_jobs.proxy_z1 )) instances: (( instance_count_overrides.proxy_z1.instances || 1 )) resource_pool: proxy_z1 networks: @@ -141,10 +259,8 @@ jobs: static_ips: (( static_ips(1) )) properties: network_name: mysql1 - - name: proxy_z2 - release: cf-mysql - template: proxy + templates: (( base_templates.proxy colocated_jobs.proxy_z2 )) instances: (( instance_count_overrides.proxy_z2.instances || 1 )) resource_pool: proxy_z2 networks: @@ -152,27 +268,22 @@ jobs: static_ips: (( static_ips(1) )) properties: network_name: mysql2 - - name: cf-mysql-broker_z1 - release: cf-mysql - template: cf-mysql-broker + templates: (( base_templates.broker colocated_jobs.cf-mysql-broker_z1 )) instances: (( instance_count_overrides.cf-mysql-broker_z1.instances || 1 )) resource_pool: cf-mysql-broker_z1 networks: - name: mysql1 properties: network_name: mysql1 - - name: cf-mysql-broker_z2 - release: cf-mysql - template: cf-mysql-broker + templates: (( base_templates.broker colocated_jobs.cf-mysql-broker_z2 )) instances: (( instance_count_overrides.cf-mysql-broker_z2.instances || 1 )) resource_pool: cf-mysql-broker_z2 networks: - name: mysql2 properties: network_name: mysql2 - - name: broker-registrar template: broker-registrar release: cf-mysql @@ -182,7 +293,6 @@ jobs: networks: - name: mysql1 properties: {} - - name: broker-deregistrar template: broker-deregistrar release: cf-mysql @@ -192,7 +302,6 @@ jobs: networks: - name: mysql1 properties: {} - - name: acceptance-tests template: acceptance-tests release: cf-mysql @@ -202,7 +311,6 @@ jobs: networks: - name: mysql1 properties: {} - - name: bootstrap template: bootstrap release: cf-mysql @@ -212,110 +320,3 @@ jobs: networks: - name: mysql1 properties: {} - -- <<: (( merge )) - -properties: - nats: - machines: (( config_from_cf.nats.machines )) - user: (( config_from_cf.nats.user )) - password: (( config_from_cf.nats.password )) - port: (( config_from_cf.nats.port )) - app_domains: (( config_from_cf.app_domains )) - domain: (( config_from_cf.domain )) - cf: - api_url: (( config_from_cf.api_url )) - app_domains: (( config_from_cf.app_domains )) - admin_username: (( config_from_cf.admin_username )) - admin_password: (( config_from_cf.admin_password )) - skip_ssl_validation: (( config_from_cf.skip_ssl_validation || nil )) - smoke_tests: - use_existing_org: (( config_from_cf.smoke_tests.use_existing_org || nil )) - org: (( config_from_cf.smoke_tests.org || nil )) - syslog_aggregator: (( property_overrides.syslog_aggregator || nil )) - cf_mysql: - standalone: (( property_overrides.standalone || false )) - external_host: (( "p-mysql." .properties.domain )) - host: (( property_overrides.host || jobs.proxy_z1.networks.mysql1.static_ips.[0] )) - mysql: - admin_username: (( property_overrides.mysql.admin_username || nil )) - admin_password: (( property_overrides.mysql.admin_password )) - port: (( property_overrides.mysql.port || nil )) - galera_port: (( property_overrides.mysql.galera_port || nil )) - healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) - bootstrap_endpoint: (( property_overrides.mysql.bootstrap_endpoint )) - cluster_ips: (( property_overrides.mysql.cluster_ips || default_mysql_cluster_ips )) - seeded_databases: (( property_overrides.mysql.seeded_databases || nil )) - database_startup_timeout: (( property_overrides.mysql.database_startup_timeout || 600 )) - max_heap_table_size: (( property_overrides.mysql.max_heap_table_size || nil )) - tmp_table_size: (( property_overrides.mysql.tmp_table_size || nil )) - wsrep_max_ws_rows: (( property_overrides.mysql.wsrep_max_ws_rows || nil )) - wsrep_max_ws_size: (( property_overrides.mysql.wsrep_max_ws_size || nil )) - skip_name_resolve: (( property_overrides.mysql.skip_name_resolve || nil )) - gcache_size: (( property_overrides.mysql.gcache_size || nil )) - ib_log_file_size: (( property_overrides.mysql.ib_log_file_size || nil )) - innodb_buffer_pool_size: (( property_overrides.mysql.innodb_buffer_pool_size || nil )) - max_connections: (( property_overrides.mysql.max_connections || nil )) - persistent_disk: (( disk_pools.mysql-persistent-disk.disk_size )) - proxy: - api_username: (( property_overrides.proxy.api_username )) - api_password: (( property_overrides.proxy.api_password )) - health_port: (( property_overrides.proxy.health_port || nil )) - api_force_https: (( property_overrides.proxy.api_force_https || nil )) - proxy_ips: (( property_overrides.proxy.proxy_ips || default_proxy_ips )) - arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) - port: (( property_overrides.proxy.port || nil )) - healthcheck_timeout_millis: (( property_overrides.proxy.healthcheck_timeout_millis || nil )) - api_port: (( property_overrides.proxy.api_port || nil )) - broker: - max_user_connections_default: (( property_overrides.broker.max_user_connections_default || nil )) - ssl_enabled: (( property_overrides.broker.ssl_enabled || nil )) - auth_username: (( property_overrides.broker.auth_username )) - auth_password: (( property_overrides.broker.auth_password )) - cookie_secret: (( property_overrides.broker.cookie_secret )) - external_host: (( property_overrides.broker.external_domain || .properties.cf_mysql.external_host )) - protocol: (( property_overrides.broker.protocol || nil )) - port: (( property_overrides.broker.port || nil )) - services: - - name: p-mysql - plan_updateable: true - id: (( property_overrides.broker.service_id || "44b26033-1f54-4087-b7bc-da9652c2a539" )) - description: MySQL databases on demand - tags: - - mysql - metadata: - displayName: "MySQL for Pivotal Cloud Foundry" - imageUrl: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAJo5JREFUeNrsXQtsVNeZvvPwzBiD3zyNwxgWhEKqmEibR9skdhQtaao2Rqtks4+qthKp1e5KxKhVCg3BbrKOst3lIW2rrUQVR5W2adgKp92ikG1ik2RbkpWCs4WWQomHVyAY8NhgPDOex97v+F4zHs97zrlz7z3/Jw1j/Ji5c+/9vv9x/v8/DoVgK9Rt7/arT37tv7XqozXT7zorK1clotFgYnp6PMtLDquPoPZ1YKxvd4DOsn3goFNgOYK3asRuUx81SQRvM/hQhpIEYlz7f1AViGG6SiQABH6WHMS+U/u61SKHDxGAp/CxJgzDqjAE6aqSABAyE75NI/iD2rPfZh8xoAnDYdfCquEr218coqtOAiC7K9+hEb5N0tMw5PBUHE5EpgcodCABsDvha5MI36HF8YRbQIgw4PR5D8dD4QEKGUgA7BLHg+yPSWzli7sx3e4hxeV8w11bO3B5y7YAnRESAKtZ+i2KdZJ2pobT6x2Oh8N74SGQZ0ACYFbig/Rf18hPEHGzulzqP44BR0XFq1d39A3QGSEBMIOLD0vfSTG94WIQdHi9/a5FC/dSiEACYCgaXtjeEZ+aImtvnnzBQCIa3auGB0N0NkgARFp8WPqdiv3W6O1xM3s9gUQ40qsKQT+dDRIAXqSHa/+M5uqTm2+R8MBZ6dvrXe3fc+HJpylpSAJAxCchICEgASDikxCQEJAAZCF/DxHf/kJwZfuLPXQ2SACSid+pUHJPnpve6wk4fb7eK8/29pMAyE38No34bUQLCW9+t3vI3VDfe3nLtiESAPni/N3KTAEPQe6wQHF4PP2+tau7ZcwPOCQkf6dGforzCXPzAwurumULCxwSER/x/Ssyufuwbs5K35zvOauqsv5NfHJy7v+nQkoiFpOGEE6fd8hVU9MlS3mxQxLy9yg2zO67Fqpkdjox3FMlu3OG7M75pOcFXRxmRCGuPk8pielp9n+bIeiuq907+u2dPSQA1rf6BxSLt+UyS161gBEd5Hb6fGrcWmGqY2SioIkBhMIOnoOWJLS1N+CwMflRzLPTilYf5Haprroak7Jns5G9EFGIh9THjUklpopCIjJtydyAq66md3Trjj0kANYgfq0W63dY6CZTXNWLLE/4XIAAxCYmlBgEYeK6xXIDvgHfujVddlspcNiM/G2ay296q8+sfHW14q5ZlDMxZ1dABGLjE+zZCuGCw+NBbmCzneoGHDYif4/m8pvb0tfVKupNJCxRR2JgiLfWa5cEocMGxDe9yw/33l1fx54JuREdC86KgWmv6aKFA96WVZYPCRwWJ3+rRn7TZfnh4rvr6hR3Y/3MLDtCUTkDiEF0bMyUCUSH2zXsaVrRdekbW4dJAIwnf4dGflPF+1ib1918Al+vIHrlqulqDpAXUK951+i3nh8gATCO/Fji222mYwLh3fW10ib0jAJqDKY/G2UrCabKC9TVdFtxqdBhQfLD6neaifgVS5fYdunOzOHB9GeXmWdgmnuhvq5f9QS6SADEEN9UyT4ivnmEIHL+gmk8AqvVCzgsRP5BxQTJPsT4FUsXk6tPoUFmEahaMOxb09JuBRFwEPnzPFGqpfc2NxHxTQ4sHUY+vVj2VQOIQHzyZrvZtzJzmJz8ID0q+/xlO0Eul+JubGBWn2AdwBvAqkE5i4rUeyfgWblis5mXCR0mJz8sf9nW01C441mxnOJ8K+cHVG+gnAVFWCb0LF/ablYRcBD507v7ID5V7tkoLDh3oWzegJlFwEHknwvd3afqPZt5Ayr59bCARMCkAlBO8oPwnuYmsvrkDUglAg4TkR+kP6qUIeHHYn2V/GT15fEGIAJlyg2A/KZZHXCaiPyD5SC/Z8Uyxeu/jcgvEXCtcc1x7cuAVmfVgsGm1/aZolnEYSLyG7rOz9b1V91GffmSA81F4TNnDa8bMEuxkBk8AMPbeVHN51u7hshPYPcA7gWjcz/xyZutoZOnXyn35y+r36s19jxp5Hsiy++9baXicDrp7ifMeIPqveCurWEOceq+CEJzEdHo+kWPPeq/+dbgG9IJgNbS+x1D4/3mJqViSSPd8YSMnqHD44F1VtmZMCoEaV20+cvjNw8NHpEmB6AN8zhg2IdE0me1n1x+Qv55gU8Chi0VsnkCNdWbyzFUxFEG8hu61g/Se1Y2EfkJhbnnkWmWHDRqAlG5agScBpNf7+k3jPxk+QnFEbLC0HsnEYnURi58+orRy4NGZ8IMy/jr5Kf1fUKpoSPbg9EIEYjGWsMjZwxdGTCMHdrc/m8a8V6Y1uNBpp/ITyhVBLBCoN5PCAmwzZkBnsD6RV/9kuPmfw8N2SYHoO3YM2gY+Zub6M4lcAfKh42YQajNoGg3YgcihwHkR0wzYkTcT+Qn2EYEPJ5g5fq1LaIrBY3IARiS9NMbeggEkcA9ZsSeD0gKGlEpKDRI1op9nhH9IVjCDw09VN1HMACummpWLCS6f4BVCgouEnIIJL9fmWnvrRVOfsr2F4zmqoXKyqrs9e8TkYhyPHiVTlY6csZirFhIdJ2Ael8H3Y0NGy9v2RYQEjYLPPYDRH7jUV3hUTbUNcwSfENtPftetUf9fm1DSa99fvKGcm5ypof+t6OXVIEIqwJxTf3+dfX7N6Q6z/oSoWgRUIWmNnr1GkKBdst4AEZs1U3lvTO4b8nymcfiZSrpF6mkX1iW45iYVr2FsatMGNjz5Yvse7b3BNQwIHTqtPCyYXddrZAtyR0CyG+I6y9rOy8s+6amVYz0eDYzED4cunBWOXT+jK1DCYN6B4IVS5dwDwVEhADCs/7IxMpG/ida1iqP+9cy4lsFCDnw2LphIwsf9p08prw+csp2ngHuxYoVy9gSoUDUxsbHuYcCXD0A1fp3agIgzhVqbCjXKKeyEX/rhrvK5tqLCBX2nTyu7PvjMdsJAaYOY8NSkXDVVHddeba333QCYETBD2qyEffL4urvuvsBS1n8QgCPoPvDd1muwE4IB84KHTaKVYHKDeu5FQjxXDjfLZL8bLOOVc1SkB9u86FNm21LfgAezf72R5mHYyew6dICd5LCqkDo1Ce7uYUvnKx/m/rUKfLEYoCnLMt9P/7iw2zpTgbAy2m2SXijWWh2r4pEIhLpXLL3pTYzeQBCl/wQ88uS9INFXGkjQuQD5DjsBDaERmCeCqsN0avXdppCALTEX5uoD4u4H4k/WZCrOs+u4YDdgHtW5KThRDTa1vjyzs6yC4BI68+265Ik7ifYD6J3m4qHQjvLKgBaxZ/fqieQQBCdDxDZoZoIR/yNfc/1lEUAtGW/LcJcf9V9oo06CVYH7mGRIWx8KrSllDmCpXgAaPMVsuwnWjkJBCMhcrt5LAuGPwk8Y6gAiLb+5PoTKBQwxgso1gMQZv2R9ZfZ9UcnnXSfOXhNilBA1H1dihdQsACItP5MKVfK7fofunDGduWx2YB+gF3HPpLis3pWLBfm2RbrBRTjAQiz/kiWiCyjtAqeev/XUkziAfkff+egFHMDmIFT721RCcFivYBiBGCLqJODZAnhFjHQNWdXwMvZdOiAdCPHWEJQkJGDF1BwaFKg+9+pCNrO29u8UnF6vcR+DeF4TBm6dJ5N2EGtfLNNKgTRBdgzfETpOfqBNJZ/ntX1eJRYcFyAG5DwVd5/35nQe0fy3l+w0IEgQqr+ZE/85bKUj6sPiMBT6+5gvQJWbBTaP3JKeT1wSqr8Rsb7HQlB9Z6P3Zjk70l7PeBof96/X4D1F7alt2+NX3FWVRHb84Q+Duy+JctKHvQp0tKD7G9qSU1ZrX1Gd31yUgmdDojxMCorN1/d0ZfXVuOFeABfF2X9ify3AAs/Homw1YBsXoFuSeENQBAwCRiDQfFcDg8Bx4PlvN8HZwaC5jslGAK2aeUqaVYCZkmq3vOivIDE9DS4mpcA5OUBaIM+R4RYf0mHe2bC1jvuYjP0QKJdx48W7TJDFPQR4ez/i5fN+VkhQKIOewTMfH2NWXN9FHixI8ER0qAN+HFV8FiYM3hQumuNicJTJ04Kee2KpUta8hkgmq8HICTzjy2WiPyZCbxffYAc+9XYGcM0C7XIQDZPolyf6wn/WkZ82cGWBVUOiNhrMHb9BjjbzUsAOoUIgER9/qUQBo+ejfey8dqIqc1G6nzcfBD+kaZV0g07ycNSCxGARDjcyUUAtOQf98IfFvuT9c8bcOdBIt1ysjzA6KU5+QAzER4JSl28ZBlvVqwXICIXgMKghhe2d+RKBubjAQhJ/lHRDx/PQNmwcTZOR+YdMTr6CRCnixYGEB1bjuE4mhcsZNbdzoNMxXkBi8uWDHTnsP6w/B28DwyWnzL//MnIMuppdgtKTuLBayhKcEpIIhJy8KFqxhvmvsdgItEBDo/17Q4W6wF0iPjAFPtnBjbcFCEOZiYv1QjMcIL3zkLaVmXgcH9G8cnxGtyz/3rmk5Aedtw6Kxfs3POQtwConBDRI+D0erNy2JnF/ferT638P2gdsTyHNew5ekSaz4sSYSoPFseNeDjcumTvS/5iPAAx7j9Z/7y8ALQE290TQKETtgcjiOVGNBjsKEYAHuN9IGiCoH7//IC1/nt/+TNmIe2GmVbgAenKf/MJj4U0xcXiGbnsyOD+Q4rGeB8H5qKRB1A49LJZ1MxbeU0dooZ4n1z+LNZ6LChqm/G6dKsBbqPcf4xCIvIXh3PaTrrVRz3KI6oIYKkv3XKfGYElSHgxIH8xPQMyhgHTn17SM/jc4PR5064GZBKAB0W4/4T8rH0moiAngPyA3hegFwNhjd4sy3vHWTfgrQrFfPIY2T6zjABXeJcHx0PhBwsRAO4egKummq5sHni8ZR1rloHFz+Uqp5YBY70f1XjoANxQWz/bDSgibNDfF4VFekdgoa49iI/dgdnnlrAbMBtXBPQHgNNdOXMAavyPpb+jvN3/yg3r6crmAb0dWCdZMZ2A6ZDcGjwjFqpAeLx5Ex0otvU3FakdgbK2A2fD1PET3MMAFRvH+nYP5/IA+Ft/cv+LJgoeSACiCxCxdLFDNFN7A4xOxMHaI28B0pt1ipHdwwCHpwLczikA/ON/cv9LAtz6p9dtYA80/Ohjtsw+aksXsE1NtxHpTRAGJCLTD+aTA2jj7f6TByBGDAB4BL8fu8ae0QlYriU21oxUV6/cXjuTf6CGIVN6zW1ZBUCN/9t4v6OzagFdTdHEw8ANZe0cdx8twfAWzt28Mc/l19uFC7HkqbkDllNQv8a4chryIU4EYhPXub5mY99zbVe2vziUyQPgXvtP7r/x0AeFzlWKjXRirCYAGBTCWQBiNybB8VkBcAqP/6nvvyDIWCVX7IwC+3sAQoznHI47RXoAqG2m2v/CBcBqM/9KAcKUfX88RhfeOP60phUArf3XT9a//Oj+4F0pRACJSxm6Hk3GIb/W6zPPA/DzfifnQhKAYgBCgBiYC2BXcqApCBugyrY5qEk41JpOANoo/jcfSfSWYLsIATwbtALbWdy4CoBPyOTsWa4nrwLcyTV+cbko/ufkDSR3Aj61boPlimoQ52Nj0P0jJ6npp1ABqPQxLnEuC74znQD4eR84ga8Q6J2Aelmtmefu6/sWYDMTcvNL5xLnseH+dALAdQWAxn4XB73CL9ugTFhR/Fz/ndQqPKM3CAXZZ/YkKLwaUW8MotFg2bnEWQBa5wiA1gHIXbUIhQNVdugGfHrdHcqu4x/l1Qk4QzxY2VPzyJX8rG/eke530pE61QMBudn7aZWEhVYUpr4vPud92v6HBGO5BM6jM1D3ALiP6hGUvJAGK7VeeXQCFhs/z/bsm4Rg8EqsmscoJxwVQnJptckhQBv3g6YEIDchgKXEw4rjtXTSW2mMmQwegMb5IV0Aani+sovW/4WAxfobG9hOwee1CTy/Hb2ouuLXTJNoQ4LydrY56HK2QShZen4iwHnrsJpkD4BvDsDppCtmgGeQvFuw7urrHYA84vRcMbz+rHcGGp18lC4M4CsArckCwFmtKumKlQHZevB1MZjzfy2pl9GapyQNqfW3nB5AJffOQGE5AIeLPIBigbl7omLxVIEwS0x+noqDysEpxnkhTKUlwOLx5vkz0pXIYpWDUB5OObUuQM6v6qIrViRAfjTJyCICW/MYf04QA3AfHgB3ASAPoDQgo2/3Tjm94/H1EbL+eXFKTGWtn4J1E4sA20Dz+FHbeQOoZUCXo0yDT8wKJAG5VgGic4nAD9hBFxNznmhZy8qDrZqF15uZfnzyGHUElsAtzl2BtRAAvk1A5P4LIY/e/MMmAKti8EjTKtOLAY4b3YDYx4CsPR9u8W4KctNptV5ocPzoVTZQA1V3M5uDLldur6sve9Wd3hE4s2nJJWoDtkgIQLAo4EqfG5m7d2DyBqF6IQ/PAp7k6kIUEhW7MSiBBIAgykNQH5lcbngNK6sK23VmIhIha04CkGecQoNAzO81UBLOmjkA/oNBFFoGJBBkFhU6BQSC3AJAUxoIBDlRQx4AgSC5B0AVGgSCnBgnD4BAkNwDIBAIJAB8EJ+cpLNKIAiACG6RB0AgkAdAIBBkBPUCWAzYOxDbh6UiWzMO1fITsgnAMNc4he/sckISdt/9wJx9AOZgw8aiX5ftJZBlGrG+10A6YGAJwaAcAH9usb0BgzxfkfPEEoIyM9J7/0OPCuv3X1m1sOh2YaO2/27Wjk/mRiYB3ApSCCA5+UsFPBIMKBEJjEPDRqm6N4IJQwh5aMpQ6XBoY8FHeL6ob42f2oIlIL9OyHv/62fCXh/7ICLvkQkQAgxOtftAEiwBhk4HeL9si3Osb3eAqEbkLyV8EHGM+Pw//uLDWckPYCQakpyEwgHuC1kGpERgaQCh3tq02TI76/IeTorPDfHLZ+syWH8ZVjhEcUrPAQwpHPcHTMTixOISb36r7LKLEIBnLA6LDsufz+cH8WVZhRDAqaFkAeCsVlPEZAnID/Dc1y852ZcLGEr69Pu/lubeEFVirwvAMFcPYHqa2CwB+QFsWsIDWWsc0rn+quWXaUlQwBLgcLIAjFMOoHyA5UO220rkZ7P/Ry+VvG1ZMclOvDc2SZEJAjg1npoD2Mn7gGmXIL5ur2hgzz7dpTdiWQ2kR7xfSBIRgoMdhYn8fHMAQd6vzsIAEgDLkJ+RfvSiYevpyPDvvueBgr0e2Vx/gSE14zxbBhzr2z3M+9UpDLAW+QGjiIW1/Xwz/cnAaoNsrr8oLumcd6ckBbhtFDqTtVxMTE+DrXfcpWwtoXlHFHo23qPt73ctZwyeDfrWYeni/d6N9xaU7Et2/bs/eFfK+0XACsCswU8WgABXASAPIC0KzXYbCcTkeOQswCmDeIH8pSYcyQOYw3VlNgTQ8DHXuCUWIxGwEPnNjEMSby8ODglYAvw4nQAMcT/4EAkAkb80yOz6C+TQ0DwBqFi6JMD94G/QgFDEvUR+cv1NxqHheQJwecu2QHJswAMxySYEI7mX2r2G2nYif3FAxl/2nn8BHAqM9e2eXfZ3p1EGP7c8QGSaPRyeCqlcfFis10dOzcavKLAhESgMWI2QfdyYzh9R1j81BwAc5q5gExO2d/EPbeqYQ3Cs7yeXtnZ/+K7tB1Zwd/0/lNv1F8idwxkFwLWwintBUMzGeQC9gSddHXvq9596/9c0mbcA158EUxh3MnsAV7a/OMRfxa7b8uIgts/WxALPYNc9989Wu+ntq7JbNXL9y8sdNf4fyhYCACQCOYAy3v3tuVt3dQ9BB0ptH3/nIIlAFjxFIimSM/O4PU8AHJ4K/nmAcfvkAdC2W0gNP0Rgd9LvIwzoOXqEmJ4Gsoz3KiNnDucUgERkeoA8gPTAEl+uIZXpgARh8t9hhYBEYC5kGu9VRs7M47Yr9Ruh945cqrz/vmfUL/n18iYSirOyUnF6vZa+KH+aGFfalq9UlvgWFPy3+LvzN2812nx0dVRprlqkbKhroLtdxdcOv6WMhmiUnE7+6Bj3Dv2gGv935/QAMikFhQHaMIoP3is6RkX4kLo8SC4vuf4GcSUtp9MOBXX6vIfjoXAnhQHpXVUk8rD2Xyj08VebDh2Y7b3Ha1lh/n+yCKLdt7RzeG2OiJLrbwhX0ub2HOm+Wbe9u1Z9GuN9BJ7mJsVdV2uLi1TKQA9dRHQSFDsQFHmEXL37qTg/eV3q/fXMDrj+kXMXRLx0XXIJcFYBAOqf//ZgIhpt43kErupFitd/m20uVimDPVDo8vjgwdn/QwTy9Sr0DjnaG89+CAfOivAAhlTyt6f19jOz1fmGCNdGQG1z+WJX1XXdP1LcXHwUEqUuD+Yz7BLkZyEIkd92ADcEuf8ZuZxRANy1tQMijkRAdrOsKCWRl255MNvMO7zPvb/8GSXMbOz+C0JGLrsy/WDyzbeDVQ+3dSRisWVcVS4UUiqWNNrqwv3i7CclLQ8ijj99fWZrhqFL59MuD8LiU5WcvRE5c44tmXPGsOr+v1ywBwDEw+G93N2cWMx2XkCpy4MYj528CrCTJfduWXmEGUR++1t/AaO/gKwczrU78IDD5eL/Ya/Yz4UFYZ/Kc686NLwgCXhey8bry4PJjUOI8/Fz5AW6P3yXGGJ3ARDHiayhfFZ2h947Eqps+0KrEo+v5+oFRKNoPVYcHo+tLiIIi2q/bFN12XCQwYPK/sBMvA+yIwzwqkKLZ4QT4XiMPfDzQpf5CNYDxn5PX74ihPyq+/9qKR6A4qioeFXEkU1/NmrLi5ktkQcvIdWaJ/e+s8ahex4gRkgGgVzIyd2cAnB1Rx/CAO5BO4Yd2HVsOAp00i0PHrpwNu3vY5NNHWzLrLtJBKSx/ioHBA3+QO3/QMkCwLwAr7ffYnFP2ZGayNNj/XRI/T6WB5+gGYIU+5eGvDiblwC4Fi3cK+TDI/Npo8KgZCQn8nSA1Kk1/yB/OrKjzBjFQgR7W3+BK2J5cTavFD9qAha03889GQigLsAu/QGpQCLvt5cvKY+tWs2SfHjga7S9Vnu8yudVgv/wvoeUxb7KeX/LtupWH3gNgj0ROXdelAFE8u9H+fyiO99XRDIwEY128D5SlguYnFScVVW2vMgs8ffBu2w3XN3i52oiQmKQBobY3Pqr97zAgbl5e+x5L/JPDb53YsHDD3YqsRh3c439z+3qBQCo8su1PKgD6/4/+MP/EUPsbv3PfyrK+gfSDf4oKQcwS9RwpFfEEUMJ7To9WAfc+WyNQyxnMHhwdkMRgn2Be12g9S+Io45CX71+x7fGEgK8AOweVLl+ne0vPkKBVE8AiULaN0AeTJ04Kcr6Y+mvrpA/cBb6Ds5Kn5AVAZwQuxYHJQP5gGSi4+u/OHSAyC8JcI8LXPkqmJsFC4B3tX+PiMIgAGuidl0WnOPqa9OAEBJsOjRATT6SAPe2wHV/cHJPoX9UcKfP9f/8RajqkYcq1Q/Txv8MJdSTFFHctTW2vhGwtIea/9cDFO/LBIz6ioeEVb++rLr/bwr3AER7ASxBYvOEIEBz+eSC4Pu6KOtftABcePLpoKhcgK6UgnqjCQTjXX/1XhY06HM29k838FOYAIj2Agw4YQSCoa6/QINWtPUvSQBEewGyhAIEcv3LZf0BR6nvXt/77EgiHPGL+GSYRuRbv1YRMZWIQDDC9Q+dOCXS+qPqr6WUF3CWegROn69X5AmkUIBArn9GlMw9B4+jELGJSDI8K5Yp7kbaRJNgHWC9P/LpJZFvkXGzD0M9AMDdUN8r0k3HibTr9CCC/YB7VTD5uVh/bgJwecu2IYfH0y/y04bPnKWlQYIl4n7cq4LRr1r/IdMIAOBbu7pb1LIgO7GR6ZmNEwgEM8f96j0quJwdHOvm9WLcBIAtCy6s6hb5ydFCaYBrRSAUHaoKbPPV0V3Ksl8qHLyPruF73xmMh8JtIs+AnbYZJ9gDArf1TgaXxF8y3LyP0FVT0xUPXT6qfimMoTjRTp8Prcl056UBxk3diknjRSdQMaPBqT5m3UWbjm0r+Xwj6See/LD6Xbxf1CHiSBd/v7dHVcSdIs8GVh28q/3SiQDiy8R0hN10M+SeUu9A9TkSMayVekYYPKoiONXzX6leCye7Do4KD/uZbOQPfxIwIkHdq1r/HksIACC6NoBZJPWmgwjYtVIQlhw3GCP79LQR8SUfLxDbvlVUsOvDHjb1HFjGXyW/AUvU3F1/YSHA7As31HdFr1w9KmJ8WKr62kEEGNFD6gNTkkMhS9c9zArV2FyxZmGbKg52CN8MJL8Q11+4B8BCgV0vPBO9cm236DNkRU8A7npsYkIbi35TuhoHXCtn1QLmLbiqqy0VOhhIfgBZ/z2WFACg4XvbDqgWrUP0+2BVAKsDpraM6AwbV0mvuvZ2H31WTF7BpYYKrppqVRAWmfpYkfATuKNPMrDBx2ahvBH9CXzr1nRNnTjVlohEhK7b6RfETCIASzFLemptzukRRSPB2esIEdDFwEyenYHkF+r6G+YBAEv2vtQWvXJ10Ag3t9yeAJGeP8wiBgaSH2jnVe6b9dwa8Ukm33w7sPArmxxqKNAm+r2QQIM1QbLJ4XQadnMgY4+Rz9MXLiqx4Dg2USHm8hJV9VwyUb02pn4dVhxulxoyeAwVdVxXA8mPJb9+Q0IvIy9k40s7DsSu3+gw4r2MSAwya6/eFNMSjDM3Y86gorFBcaken+hrbGDCz5C4fw5PjLxo3pZVXap6DxtikQUWaMDawx2cOn6C1X8T+cuTM8C5Z9cA47YnJ+1A/mEj4v6yeQDAsh/tao1c/GxQdFIw2RPwrGzisu4MFxAW3yoFOdLlCrCkqHoEPPpEWHnv+QtGkj+oxf3DthYAYPG/fK8jNj5xwKi171LLhkH86c8uk6W3UniwdEnRQmBgeW8yNqvkHzBcNMtxgW6+NXhiYceXxuM3px4xxl9MKNFrYyxxlK8I4OJHR68yKwCrr8TixCyrQL1WSBpGtevG+hTyTAjjb8IjZ9g9YyC6jUr6mcIDSPIEXlGJ2Wnke2K2IGYMZr0Jrlyd2cTRYtV5n1vepNyxfIVyW139vJ/97uIF5djFT5WzY9fk8whUD7Bi6eKccyWRUxC4d18mYLpPV9nOTbkvjlGVgqmxomdV87zssRVdfZD+m59/QHn09juUGl9lzt+HAPzPyGnlpx/9r/L+yJ/m/fyLLX+m/OLpv+dybPXf3WqJ0IBNnz5zrhy5HUMz/mkNYrkvCioFQ6dH/PHJm62GeYjqhQ6dOq14V93G3ENkkMMY4Wwh4sPK9z3awYhf6N/h8dd3/TkTgn/4+U+l8QrYyoF6nSHy3uYm1qXI4n3MmzT+2hue8TelAGCUWNNr+9pVERg0UgRwwSECTAAs1nkHK/2Tv+vKy+JnA8IFGUMCdu1PB8p57UH+dp6jvYqF0wwXBCKgkr9dOzGGwmrkh+WGi14q+YGDvz+myAzZyW8KD0AHTsiyH+3qMrJGwGpAvN/35czpkvHQFCO1nvDT8YXVa5TPLWtiz8nC8e+/eZdOqrFgDT5mIb+pBAC49I2tw6oItJMIpMe//eWTGS0/yPzy24eYCKRCT/bhb5EzePahTez/EIp88dV9P0ybNCQURH7DC30sJQAkAtldf3gA6YBEHrL6uQBxwO/BS8j0WgR5yG+aHEA6EfAsX9ru8HiCdO/MQLfaqdj+q4G8yJ8qBGTNifymFYBkEVDKkBg0Y+yfrrgHy3gUx5saw2Ymv6kFQBcBnEBn1QKpRSDTWv9/fPQhUYzIb18BAJAx9a1pkVoEsO6fDoW6/gTDyW/6ENZthbM5Wyx08vQrRpcNmwHNtXVp3X8jgaKhfHA2eE3K4qIkoKOvywrkt4wA6CKgPm0uRwNRuZEu/h+fmjL0GLLVHyTj5XcOseVISVHWxh5bhgCpGP3W813uxvpuu+4GlC9+d+mCQjAVuq1GfksKABOBrTv2uGqqN8u8TMijFJjABcwzFbl5BwlAek9ggNUKGDRj0GygQh5TQE/2DVj1A7itfPaxTIjkYHjkzCtGTRsui7t/8cI8wqdLDIo+holQ7uYZiRKAlkr22VIAgNnk4Pd7e2IT13facY+9c2Nj8wQAiUF8r5B6/lLw3V+9QdWDGhyeit5rPf/cY4fP4rTLRRn99s4ed2ODLcuHMxEP/QEEw+P9druQ31YCAFzesm2ocv3aFqfPN2Cnz3XwD8cyCkC6JUKCMJe/xYjtukgASgwJrj7/0mZtqdAW3oA+xy8VWAn4yd920YqAeKuPJb7NVo/3pRCA2ZBg6449akiw0eF220KxMxXXIA9Q6IQgeA4jO/6JhCM3cO9stOoSXz6wdTXN5JtvB6cO/+bVRV/9EjYmxbxBn1U/C0psQfa1i5fM+9nSRdVK1z2fV3wVFcq54FjaoSD6INB9f/U15W/uulvxuSuUd079kb1uut9LBfoOUn/X5lb/ZRT2hN47YutaE4csV3TJ3pf8sfHxV+KhcJtVPwMsNqx9rhoAhAwQAh1YMkyXK0hXtptpLLhEE4Fg9bG8F5Dhw7plEYDLW7bhgrY3vryzM35jcnciFrPctCFY9n/8+Wss7s+W/NNHf+cC5gQS5sX6/TJ9aKdsV/nKs739lRvWtzgrK/ut2E+Adf8H/+1fuXQD5tvhJwFA+hbZyC+lAABspWBHXxerG7BgkhCewFf2/aCkTT0gIPAmJAeuPUp5u+yY4accQJ5gYUEotDMRjvitePyI2zE1CLmBL7SsyZgXwKhwxPGoK8gkHFmTgPYp80U42CujxScByCYEfc/1xKdCW6yYH5gX36tigKQhle/Oi/P3qsTvoVNBApAWTa/tqw1/EnjGLkJAuEV89bFHVlefBICEgIhPIAEgISDiE0gAikLd9u5Oh9dj2WShBAgolNwjATBACNocbveWRDTaQWfDFECn3l67deqRAJgcrLz4+o0tiXC4k8KDsrj5/RrxA3Q6SADKioYXtnckpqe/riQSHXacSmQya/+qlefwkQDYOzyAJ9Dh9Hq3xMPhVjojXIDhm0jqDVBSjwTAUiFCNBjsUGLxxxLRaBudkYKAeP4NjfTk4pMAWN8zcPq8HfFQ+EF4COqDcgbzY3q49YfJ0pMAyCAIrQ5PRUciMg1BkNU7GEoi/DDdFSQA0qKx77m22I1J5AwgCHj22+wjBrRYHoQfpiU7EgBCjpBBEwJ4B3dqgmCVpOKwRviPNSs/TC49CQCBU+ig5Q8gDDVJomB0GDGURPZx7f9BcuVJAAjlFQh/UvhQy8FrAKF16x2gjLy98P8CDADMneNqQ66ddAAAAABJRU5ErkJggg==" - longDescription: "Creating a service instance provisions a database. Binding applications provisions unique credentials for each application to access the database." - providerDisplayName: "Pivotal Software" - documentationUrl: "https://github.com/cloudfoundry/cf-mysql-release/blob/master/README.md" - supportUrl: "https://support.pivotal.io" - dashboard_client: - id: p-mysql - secret: (( property_overrides.broker.dashboard_secret )) - plans: (( property_overrides.broker.plans )) - acceptance_tests: - smoke_tests_only: (( property_overrides.acceptance_tests.smoke_tests_only || nil )) - standalone_tests_only: (( property_overrides.acceptance_tests.standalone_tests_only || nil )) - timeout_scale: (( property_overrides.acceptance_tests.timeout_scale || nil )) - - <<: (( merge )) - -# The keys below should not be included in the final stub -config_from_cf: (( merge )) -iaas_settings: (( merge )) -instance_count_overrides: (( merge || nil )) -property_overrides: (( merge )) -release_versions: (( merge || nil )) -empty_hash: {} -default_mysql_cluster_ips: -- (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) -- (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) -- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) -default_proxy_ips: -- (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) -- (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) -# spiff does not allow concating multiple fields of different types -# without adding new temporary properties -database_startup_timeout_str: (( "" .properties.cf_mysql.mysql.database_startup_timeout )) -canary_end_in_sec: (( database_startup_timeout_str "000" )) diff --git a/manifest-generation/examples/aws/iaas-settings.yml b/manifest-generation/examples/aws/iaas-settings.yml index 519f89e7..aa4ad7d0 100644 --- a/manifest-generation/examples/aws/iaas-settings.yml +++ b/manifest-generation/examples/aws/iaas-settings.yml @@ -6,6 +6,7 @@ properties: availability_zone3: REPLACE_WITH_AZ_3 iaas_settings: + <<: (( merge || nil )) stemcell: &stemcell name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent version: latest diff --git a/scripts/generate-deployment-manifest b/scripts/generate-deployment-manifest index 00a79a66..e86b612f 100755 --- a/scripts/generate-deployment-manifest +++ b/scripts/generate-deployment-manifest @@ -11,25 +11,28 @@ function usage(){ Mandatory arguments: -c CF Manifest -p Property overrides stub file - -i Infrastructure type stub file + -i Infrastructure settings stub file Optional arguments -n Instance count overrides stub file (single node, 3 node) -v Release versions stub file - -b Backups manifest stub file + -j Manifest stub file for additional jobs " exit 1 } -while getopts "c:p:i:n:v:b:" opt; do +property_overrides=() +iaas_settings=() + +while getopts "c:p:i:n:v:j:" opt; do case $opt in c) cf_manifest=$OPTARG ;; p) - property_overrides=$OPTARG + property_overrides+=($OPTARG) ;; i) - infrastructure_type=$OPTARG + iaas_settings+=($OPTARG) ;; n) instance_counts=$OPTARG @@ -37,8 +40,8 @@ while getopts "c:p:i:n:v:b:" opt; do v) release_versions=$OPTARG ;; - b) - backup_stub=$OPTARG + j) + other_jobs_stub=$OPTARG ;; *) echo "Unknown arguments" @@ -47,12 +50,16 @@ while getopts "c:p:i:n:v:b:" opt; do esac done -if [ -z "${infrastructure_type}" ]; then - >&2 echo "infrastructure_type stub file is missing" - argument_error=true -elif [ ! -f "${infrastructure_type}" ]; then - >&2 echo "infrastructure_type stub ${infrastructure_type} is not a regular file" +if [ ${#iaas_settings[@]} == 0 ]; then + >&2 echo "iaas_settings stub file is missing" argument_error=true +else + for file in "${iaas_settings[@]}"; do + if [ ! -f "${file}" ]; then + >&2 echo "iaas_settings stub ${file} is not a regular file" + argument_error=true + fi + done fi if [ -z "${cf_manifest}" ]; then @@ -63,12 +70,16 @@ elif [ ! -f "${cf_manifest}" ]; then argument_error=true fi -if [ -z ${property_overrides} ]; then +if [ ${#property_overrides[@]} == 0 ]; then >&2 echo "property-overrides stub file is missing" argument_error=true -elif [ ! -f "${property_overrides}" ]; then - >&2 echo "Property Overrides stub '${property_overrides}' is not a regular file" - argument_error=true +else + for file in "${property_overrides[@]}"; do + if [ ! -f "${file}" ]; then + >&2 echo "Property Overrides stub '${file}' is not a regular file" + argument_error=true + fi + done fi if [[ ! -z "${instance_counts}" && ! -f "${instance_counts}" ]]; then @@ -81,8 +92,8 @@ if [[ ! -z "${release_versions}" && ! -f "${release_versions}" ]]; then argument_error=true fi -if [[ ! -z "${backup_stub}" && ! -f "${backup_stub}" ]]; then - >&2 echo "Backup Stub file '${backup_stub}' is not a regular file" +if [[ ! -z "${other_jobs_stub}" && ! -f "${other_jobs_stub}" ]]; then + >&2 echo "Additional Job Stub file '${other_jobs_stub}' is not a regular file" argument_error=true fi @@ -101,11 +112,11 @@ spiff merge \ spiff merge \ ${MANIFEST_DIR}/cf-mysql-template.yml \ - ${backup_stub} \ + ${other_jobs_stub} \ ${tmpdir}/config-from-cf.yml \ - ${infrastructure_type} \ + "${iaas_settings[@]}" \ ${instance_counts} \ - ${property_overrides} \ + "${property_overrides[@]}" \ ${release_versions} \ > ${tmpdir}/mysql-config.yml From b568ce4a97cfacbe4df0c146a8d11b4fc11d193f Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Tue, 12 Jan 2016 12:13:52 -0800 Subject: [PATCH 041/110] Move "Copy sample stubs" section before "Generate manifest" in README - Improves clarity by making steps more chronological [#110775864] --- README.md | 62 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 32 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 4ce0678f..73f10ac9 100644 --- a/README.md +++ b/README.md @@ -295,29 +295,24 @@ The brokers each register a route with the router, which load balances requests #### Deploy on AWS or vSphere -For other environments, we provide a script called `generate-deployment-manifest`. - -``` -Usage: -The script requires the following arguments- - -c CF Manifest - -p Property overrides stub file (Use this file to provide credentials and broker plans) - -i Infrastructure type stub file (AWS or vSphere) -The following arguments are optional- - -n Instance count overrides stub file (single node, 3 node) - -v Release versions stub file (Use this file to specify the cf-mysql release version, defaults to latest) -``` +##### Copy sample stubs and fill in values -For example: +We have provided example stubs to serve as a starting point. +Copy these stubs to your config repository, and fill in the `REPLACE_WITH` text with values for your environment. +For example, the following files can be used for an AWS deployment: ``` -$ ./scripts/generate-deployment-manifest \ - -c /cf-deployment.yml \ - -p /cf-mysql/property-overrides.yml \ - -i /cf-mysql/iaas-settings.yml \ - > cf-mysql.yml +$ cp cf-mysql-release/manifest-generation/examples/aws/iaas-settings.yml \ + cf-mysql-release/manifest-generation/examples/property-overrides.yml \ + /cf-mysql/ ``` +Additional example stubs can be found under [cf-mysql-release/manifest-generation/examples/](manifest-generation/examples). +These include: +- Deploying with a minimal number of VMs ([examples/minimal/](manifest-generation/examples/minimal/)) +- Deploying without a running CF deployment ([examples/standalone/](manifest-generation/examples/standalone/)) +- Replacing the arbitrator with a full MySQL node ([examples/no-arbitrator/](manifest-generation/examples/no-arbitrator/)) + ##### Provide a CF manifest The script must obtain some configuration options from an existing CF manifest (e.g. CF Admin credentials). @@ -332,23 +327,30 @@ This admin user must have the `cloud_controller.admin` UAA permission. Note: This change to the CF manifest is temporary while we investigate better methods for sharing properties across deployments. -##### Copy sample stubs and fill in values +##### Generate AWS or vSphere manifest -We have provided example stubs to serve as a starting point. -Copy these stubs to your config repository, and fill in the `REPLACE_WITH` text with values for your environment. +Run the `./scripts/generate-deployment-manifest` with the stubs you created in the preceeding steps. -For example, the following files can be used for an AWS deployment: ``` -$ cp cf-mysql-release/manifest-generation/examples/aws/iaas-settings.yml \ - cf-mysql-release/manifest-generation/examples/property-overrides.yml \ - /cf-mysql/ +Usage: +The script requires the following arguments- + -c CF Manifest + -p Property overrides stub file (Use this file to provide credentials and broker plans) + -i Infrastructure type stub file (AWS or vSphere) +The following arguments are optional- + -n Instance count overrides stub file (single node, 3 node) + -v Release versions stub file (Use this file to specify the cf-mysql release version, defaults to latest) ``` -Additional example stubs can be found under [cf-mysql-release/manifest-generation/examples/](manifest-generation/examples). -These include: -- Deploying with a minimal number of VMs ([examples/minimal/](manifest-generation/examples/minimal/)) -- Deploying without a running CF deployment ([examples/standalone/](manifest-generation/examples/standalone/)) -- Replacing the arbitrator with a full MySQL node ([examples/no-arbitrator/](manifest-generation/examples/no-arbitrator/)) +For example: + +``` +$ ./scripts/generate-deployment-manifest \ + -c /cf-deployment.yml \ + -p /cf-mysql/property-overrides.yml \ + -i /cf-mysql/iaas-settings.yml \ + > cf-mysql.yml +``` ##### Deploying From 21bcde37685d1394a7798fc01e3e7dc4eafbba71 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Tue, 12 Jan 2016 12:20:57 -0800 Subject: [PATCH 042/110] Update proxy documentation to reference new manifest properties [#110775864] --- docs/proxy.md | 21 ++++++++----------- .../examples/property-overrides.yml | 2 +- 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/docs/proxy.md b/docs/proxy.md index 69d7b345..e80a43ca 100644 --- a/docs/proxy.md +++ b/docs/proxy.md @@ -55,25 +55,22 @@ Bound applications are provided with a hostname or IP address to reach a databas ### Configuring load balancer -Configure the load balancer to route traffic for TCP port 3306 to the IPs of all proxy instances on TCP port 3306. Next, configure the load balancer's healthcheck to use the proxy health port. This is TCP port 1936 by default to maintain backwards compatibility with previous releases, but this port can be configured by changing the following manifest property: +Configure the load balancer to route traffic for TCP port 3306 to the IPs of all proxy instances on TCP port 3306. +Next, configure the load balancer's healthcheck to use the proxy health port. +This is TCP port 1936 by default to maintain backwards compatibility with previous releases, but this port can be configured by changing the following manifest property in the `property-overrides` stub: ``` -jobs: -- name: proxy_z1 - properties: - proxy: - health_port: +property_overrides: + proxy: + health_port: ``` ### Configuring cf-mysql-release to give applications the address of the load balancer -To ensure that bound applications will use the load balancer to reach bound databases, the manifest property `properties.mysql_node.host` must be updated for the cf-mysql-broker job: +To ensure that bound applications will use the load balancer to reach bound databases, the set `property_overrides.host` in the `property-overrides` stub: ``` -jobs: -- name: cf-mysql-broker_z1 - properties: - mysql_node: - host: +property_overrides: + host: ``` ### AWS Route 53 diff --git a/manifest-generation/examples/property-overrides.yml b/manifest-generation/examples/property-overrides.yml index 21a9e963..38222dfa 100644 --- a/manifest-generation/examples/property-overrides.yml +++ b/manifest-generation/examples/property-overrides.yml @@ -2,7 +2,7 @@ property_overrides: <<: (( merge || nil )) syslog_aggregator: ~ standalone: false - host: YOUR_LOAD_BALANCER_ADDRESS # Optional, set to your Load Balancer address if configured + host: REPLACE_WITH_LB_HOSTNAME # Optional, set to your Load Balancer address if configured; delete this line otherwise mysql: admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process bootstrap_endpoint: From 02b38fe1724a905376207c9d28fa2a883901579c Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Tue, 12 Jan 2016 12:36:12 -0800 Subject: [PATCH 043/110] Add instructions for filling in a cf-stub in README - Add properties directly to the CF manifest felt a bit scary - We'll re-add those instructions once this UAA story is finished #111089660 [#110775864] [ci skip] --- README.md | 14 +++++--------- manifest-generation/examples/cf-stub.yml | 19 +++++++++++++++++++ 2 files changed, 24 insertions(+), 9 deletions(-) create mode 100644 manifest-generation/examples/cf-stub.yml diff --git a/README.md b/README.md index 73f10ac9..65d3b004 100644 --- a/README.md +++ b/README.md @@ -304,6 +304,7 @@ For example, the following files can be used for an AWS deployment: ``` $ cp cf-mysql-release/manifest-generation/examples/aws/iaas-settings.yml \ cf-mysql-release/manifest-generation/examples/property-overrides.yml \ + cf-mysql-release/manifest-generation/examples/cf-stub.yml \ /cf-mysql/ ``` @@ -313,16 +314,11 @@ These include: - Deploying without a running CF deployment ([examples/standalone/](manifest-generation/examples/standalone/)) - Replacing the arbitrator with a full MySQL node ([examples/no-arbitrator/](manifest-generation/examples/no-arbitrator/)) -##### Provide a CF manifest +##### Fill in the CF stub -The script must obtain some configuration options from an existing CF manifest (e.g. CF Admin credentials). -Add the following properties to your CF manifest prior to running this script: -``` -properties: - admin_username: REPLACE_WITH_ADMIN_USERNAME - admin_password: REPLACE_WITH_ADMIN_PASSWORD - skip_ssl_validation: # optional, defaults to false -``` +The script must obtain some configuration options from the CF deployment (e.g. CF Admin credentials). +Edit the copied `cf-stub.yml` from the previous section to include values from your CF manifest. +Existing admin users can be found under `properties.uaa.scim.users` in the CF manifest. This admin user must have the `cloud_controller.admin` UAA permission. Note: This change to the CF manifest is temporary while we investigate better methods for sharing properties across deployments. diff --git a/manifest-generation/examples/cf-stub.yml b/manifest-generation/examples/cf-stub.yml new file mode 100644 index 00000000..8c5dc11c --- /dev/null +++ b/manifest-generation/examples/cf-stub.yml @@ -0,0 +1,19 @@ +# Standalone deployments have no CF, but these fields are required +# for all deployments. +name: REPLACE_WITH_CF_DEPLOYMENT_NAME +director_uuid: REPLACE_WITH_DIRECTOR_UUID +properties: + admin_username: REPLACE_WITH_ADMIN_USERNAME + admin_password: REPLACE_WITH_ADMIN_PASSWORD + skip_ssl_validation: false # Optional, change to `true` in envs with self-signed SSL certs + nats: + machines: + - REPLACE_WITH_FIRST_NATS_IP + - REPLACE_WITH_SECOND_NATS_IP + user: REPLACE_WITH_NATS_USER + password: REPLACE_WITH_NATS_PASSWORD + port: REPLACE_WITH_NATS_PORT + system_domain: REPLACE_WITH_SYSTEM_DOMAIN + app_domains: + - REPLACE_WITH_APP_DOMAIN + domain: REPLACE_WITH_DOMAIN From 9771c1af93b0924583830590b3cfa5491a2ee351 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Tue, 12 Jan 2016 15:36:50 -0800 Subject: [PATCH 044/110] Add `REPLACE_WITH_ELB_NAME_NOT_DNS_HOSTNAME` to aws stub - Adds instructions for user to configure ELB via the stu [#109744228] [ci skip] --- docs/proxy.md | 9 +++++++++ manifest-generation/examples/aws/iaas-settings.yml | 5 +++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/proxy.md b/docs/proxy.md index e80a43ca..801a74f7 100644 --- a/docs/proxy.md +++ b/docs/proxy.md @@ -73,6 +73,15 @@ property_overrides: host: ``` +If deploying on AWS, also add the ELB name (not the address) in the `iaas-settings` stub: + +``` +properties: + template_only: + aws: + mysql_elb_names: [] +``` + ### AWS Route 53 To set up a Round Robin DNS across multiple proxy IPs using AWS Route 53, diff --git a/manifest-generation/examples/aws/iaas-settings.yml b/manifest-generation/examples/aws/iaas-settings.yml index aa4ad7d0..a580f504 100644 --- a/manifest-generation/examples/aws/iaas-settings.yml +++ b/manifest-generation/examples/aws/iaas-settings.yml @@ -1,6 +1,7 @@ properties: template_only: aws: + mysql_elb_names: [REPLACE_WITH_ELB_NAME_NOT_DNS_HOSTNAME] # optional, remove this line to skip binding an ELB availability_zone: REPLACE_WITH_AZ_1 # e.g. us-east-1a availability_zone2: REPLACE_WITH_AZ_2 availability_zone3: REPLACE_WITH_AZ_3 @@ -41,13 +42,13 @@ iaas_settings: cloud_properties: instance_type: m3.medium availability_zone: (( properties.template_only.aws.availability_zone )) - elbs: (( merge || nil )) + elbs: (( properties.template_only.aws.mysql_elb_names || nil )) - name: proxy_z2 cloud_properties: instance_type: m3.medium availability_zone: (( properties.template_only.aws.availability_zone2 )) - elbs: (( merge || nil )) + elbs: (( properties.template_only.aws.mysql_elb_names || nil )) - name: cf-mysql-broker_z1 cloud_properties: From ce8a414d95e5116311d9db7d37c9ca9fdca052f8 Mon Sep 17 00:00:00 2001 From: Marco Nicosia Date: Tue, 12 Jan 2016 16:01:06 -0800 Subject: [PATCH 045/110] Typo fix cf-development.yml is actually the cf-stub.yml stub. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 65d3b004..89fa5b70 100644 --- a/README.md +++ b/README.md @@ -342,7 +342,7 @@ For example: ``` $ ./scripts/generate-deployment-manifest \ - -c /cf-deployment.yml \ + -c /cf-stub.yml \ -p /cf-mysql/property-overrides.yml \ -i /cf-mysql/iaas-settings.yml \ > cf-mysql.yml From 62a7d650e2c88b8f4d2c11c5efcab6d90882b424 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Tue, 12 Jan 2016 16:42:10 -0800 Subject: [PATCH 046/110] Reorder networks in iaas stubs - It was easy to miss the networks at the bottom of the stubs [#110775864] [ci skip] --- .../examples/aws/iaas-settings.yml | 94 +++++++++---------- .../examples/vsphere/iaas-settings.yml | 94 +++++++++---------- 2 files changed, 94 insertions(+), 94 deletions(-) diff --git a/manifest-generation/examples/aws/iaas-settings.yml b/manifest-generation/examples/aws/iaas-settings.yml index a580f504..2c601166 100644 --- a/manifest-generation/examples/aws/iaas-settings.yml +++ b/manifest-generation/examples/aws/iaas-settings.yml @@ -6,6 +6,53 @@ properties: availability_zone2: REPLACE_WITH_AZ_2 availability_zone3: REPLACE_WITH_AZ_3 +# NOTE: +# Replace 10.0.{0,1,2} part of the octet +# with your network configuration + subnet_configs: + - name: mysql1 + type: manual + subnets: + - range: 10.10.32.0/24 + reserved: + - 10.10.32.2 - 10.10.32.9 + - 10.10.32.130 - 10.10.32.254 + static: + - 10.10.32.10 - 10.10.32.14 + gateway: 10.10.32.1 + dns: + - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 + cloud_properties: + subnet: REPLACE_WITH_YOUR_SUBNET_ID_1 + - name: mysql2 + type: manual + subnets: + - range: 10.10.33.0/24 + reserved: + - 10.10.33.2 - 10.10.33.9 + - 10.10.33.130 - 10.10.33.254 + static: + - 10.10.33.10 - 10.10.33.14 + gateway: 10.10.33.1 + dns: + - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 + cloud_properties: + subnet: REPLACE_WITH_YOUR_SUBNET_ID_2 + - name: mysql3 + type: manual + subnets: + - range: 10.10.34.0/24 + reserved: + - 10.10.34.2 - 10.10.34.9 + - 10.10.34.130 - 10.10.34.254 + static: + - 10.10.34.10 - 10.10.34.14 + gateway: 10.10.34.1 + dns: + - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 + cloud_properties: + subnet: REPLACE_WITH_YOUR_SUBNET_ID_3 + iaas_settings: <<: (( merge || nil )) stemcell: &stemcell @@ -70,50 +117,3 @@ iaas_settings: disk_size: 100000 cloud_properties: type: gp2 - -# NOTE: -# Replace 10.0.{0,1,2} part of the octet -# with your network configuration - subnet_configs: - - name: mysql1 - type: manual - subnets: - - range: 10.10.32.0/24 - reserved: - - 10.10.32.2 - 10.10.32.9 - - 10.10.32.130 - 10.10.32.254 - static: - - 10.10.32.10 - 10.10.32.14 - gateway: 10.10.32.1 - dns: - - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 - cloud_properties: - subnet: REPLACE_WITH_YOUR_SUBNET_ID_1 - - name: mysql2 - type: manual - subnets: - - range: 10.10.33.0/24 - reserved: - - 10.10.33.2 - 10.10.33.9 - - 10.10.33.130 - 10.10.33.254 - static: - - 10.10.33.10 - 10.10.33.14 - gateway: 10.10.33.1 - dns: - - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 - cloud_properties: - subnet: REPLACE_WITH_YOUR_SUBNET_ID_2 - - name: mysql3 - type: manual - subnets: - - range: 10.10.34.0/24 - reserved: - - 10.10.34.2 - 10.10.34.9 - - 10.10.34.130 - 10.10.34.254 - static: - - 10.10.34.10 - 10.10.34.14 - gateway: 10.10.34.1 - dns: - - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 - cloud_properties: - subnet: REPLACE_WITH_YOUR_SUBNET_ID_3 diff --git a/manifest-generation/examples/vsphere/iaas-settings.yml b/manifest-generation/examples/vsphere/iaas-settings.yml index 300ce439..47521fb9 100644 --- a/manifest-generation/examples/vsphere/iaas-settings.yml +++ b/manifest-generation/examples/vsphere/iaas-settings.yml @@ -16,6 +16,53 @@ properties: - REPLACE_WITH_YOUR_THIRD_CLUSTER: resource_pool: REPLACE_WITH_YOUR_THIRD_RESOURCE_POOL +# NOTE: +# Replace 10.0.{0,1,2} part of the octet +# with your network configuration + subnet_configs: + - name: mysql1 + subnets: + - range: 10.0.0.0/24 + gateway: 10.0.0.1 + reserved: + - 10.0.0.2-10.0.0.109 + - 10.0.0.130-10.0.0.254 + static: + - 10.0.0.120-10.0.0.124 + dns: + - 10.0.0.3 + - 8.8.8.8 + cloud_properties: + name: REPLACE_WITH_YOUR_SUBNET_NAME #This should match your vSphere network name, e.g. "VM Network". + - name: mysql2 + subnets: + - range: 10.0.1.0/24 + gateway: 10.0.1.1 + reserved: + - 10.0.1.2-10.0.1.109 + - 10.0.1.130-10.0.1.254 + static: + - 10.0.1.120-10.0.1.124 + dns: + - 10.0.1.3 + - 8.8.8.8 + cloud_properties: + name: REPLACE_WITH_YOUR_SUBNET_NAME_2 + - name: mysql3 + subnets: + - range: 10.0.2.0/24 + gateway: 10.0.2.1 + reserved: + - 10.0.2.2-10.0.2.109 + - 10.0.2.130-10.0.2.254 + static: + - 10.0.2.120-10.0.2.124 + dns: + - 10.0.2.3 + - 8.8.8.8 + cloud_properties: + name: REPLACE_WITH_YOUR_SUBNET_NAME_3 + iaas_settings: stemcell: &stemcell name: bosh-vsphere-esxi-ubuntu-trusty-go_agent @@ -112,50 +159,3 @@ iaas_settings: disk_pools: - name: mysql-persistent-disk disk_size: 100000 - -# NOTE: -# Replace 10.0.{0,1,2} part of the octet -# with your network configuration - subnet_configs: - - name: mysql1 - subnets: - - range: 10.0.0.0/24 - gateway: 10.0.0.1 - reserved: - - 10.0.0.2-10.0.0.109 - - 10.0.0.130-10.0.0.254 - static: - - 10.0.0.120-10.0.0.124 - dns: - - 10.0.0.3 - - 8.8.8.8 - cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME #This should match your vSphere network name, e.g. "VM Network". - - name: mysql2 - subnets: - - range: 10.0.1.0/24 - gateway: 10.0.1.1 - reserved: - - 10.0.1.2-10.0.1.109 - - 10.0.1.130-10.0.1.254 - static: - - 10.0.1.120-10.0.1.124 - dns: - - 10.0.1.3 - - 8.8.8.8 - cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME_2 - - name: mysql3 - subnets: - - range: 10.0.2.0/24 - gateway: 10.0.2.1 - reserved: - - 10.0.2.2-10.0.2.109 - - 10.0.2.130-10.0.2.254 - static: - - 10.0.2.120-10.0.2.124 - dns: - - 10.0.2.3 - - 8.8.8.8 - cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME_3 From 54105da69f54631f43c3b143587cc2d41e863721 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Tue, 12 Jan 2016 17:11:13 -0800 Subject: [PATCH 047/110] Move manifest header about network config [ci skip] --- manifest-generation/examples/aws/iaas-settings.yml | 5 +++-- manifest-generation/examples/vsphere/iaas-settings.yml | 4 +++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/manifest-generation/examples/aws/iaas-settings.yml b/manifest-generation/examples/aws/iaas-settings.yml index 2c601166..6fdcd8bf 100644 --- a/manifest-generation/examples/aws/iaas-settings.yml +++ b/manifest-generation/examples/aws/iaas-settings.yml @@ -6,6 +6,9 @@ properties: availability_zone2: REPLACE_WITH_AZ_2 availability_zone3: REPLACE_WITH_AZ_3 +iaas_settings: + <<: (( merge || nil )) + # NOTE: # Replace 10.0.{0,1,2} part of the octet # with your network configuration @@ -53,8 +56,6 @@ properties: cloud_properties: subnet: REPLACE_WITH_YOUR_SUBNET_ID_3 -iaas_settings: - <<: (( merge || nil )) stemcell: &stemcell name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent version: latest diff --git a/manifest-generation/examples/vsphere/iaas-settings.yml b/manifest-generation/examples/vsphere/iaas-settings.yml index 47521fb9..09d39223 100644 --- a/manifest-generation/examples/vsphere/iaas-settings.yml +++ b/manifest-generation/examples/vsphere/iaas-settings.yml @@ -16,6 +16,9 @@ properties: - REPLACE_WITH_YOUR_THIRD_CLUSTER: resource_pool: REPLACE_WITH_YOUR_THIRD_RESOURCE_POOL +iaas_settings: + <<: (( merge || nil )) + # NOTE: # Replace 10.0.{0,1,2} part of the octet # with your network configuration @@ -63,7 +66,6 @@ properties: cloud_properties: name: REPLACE_WITH_YOUR_SUBNET_NAME_3 -iaas_settings: stemcell: &stemcell name: bosh-vsphere-esxi-ubuntu-trusty-go_agent version: latest From 5b5fd3ac7b45cbcd1a4ea0a32c20f0a18018d135 Mon Sep 17 00:00:00 2001 From: Lyle Franklin Date: Wed, 13 Jan 2016 09:00:21 -0800 Subject: [PATCH 048/110] Move REPLACE_WITH_YOUR_SUBNET_ID to top of stubs - Easy to miss if it's farther down the page [#110775864] [ci skip] --- .../examples/aws/iaas-settings.yml | 10 ++- .../examples/vsphere/iaas-settings.yml | 83 ++++++++++--------- 2 files changed, 51 insertions(+), 42 deletions(-) diff --git a/manifest-generation/examples/aws/iaas-settings.yml b/manifest-generation/examples/aws/iaas-settings.yml index 6fdcd8bf..7c1051fc 100644 --- a/manifest-generation/examples/aws/iaas-settings.yml +++ b/manifest-generation/examples/aws/iaas-settings.yml @@ -5,6 +5,10 @@ properties: availability_zone: REPLACE_WITH_AZ_1 # e.g. us-east-1a availability_zone2: REPLACE_WITH_AZ_2 availability_zone3: REPLACE_WITH_AZ_3 + subnet_ids: + mysql1: REPLACE_WITH_YOUR_SUBNET_ID_1 + mysql2: REPLACE_WITH_YOUR_SUBNET_ID_2 + mysql3: REPLACE_WITH_YOUR_SUBNET_ID_3 iaas_settings: <<: (( merge || nil )) @@ -26,7 +30,7 @@ iaas_settings: dns: - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 cloud_properties: - subnet: REPLACE_WITH_YOUR_SUBNET_ID_1 + subnet: (( properties.template_only.aws.subnet_ids.mysql1 )) - name: mysql2 type: manual subnets: @@ -40,7 +44,7 @@ iaas_settings: dns: - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 cloud_properties: - subnet: REPLACE_WITH_YOUR_SUBNET_ID_2 + subnet: (( properties.template_only.aws.subnet_ids.mysql2 )) - name: mysql3 type: manual subnets: @@ -54,7 +58,7 @@ iaas_settings: dns: - 10.10.0.2 # amazon has only 1 DNS server for the whole VPC at x.x.0.2 cloud_properties: - subnet: REPLACE_WITH_YOUR_SUBNET_ID_3 + subnet: (( properties.template_only.aws.subnet_ids.mysql3 )) stemcell: &stemcell name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent diff --git a/manifest-generation/examples/vsphere/iaas-settings.yml b/manifest-generation/examples/vsphere/iaas-settings.yml index 09d39223..62d3815a 100644 --- a/manifest-generation/examples/vsphere/iaas-settings.yml +++ b/manifest-generation/examples/vsphere/iaas-settings.yml @@ -1,20 +1,25 @@ properties: - datacenters: - - name: z1 - datacenter_name: REPLACE_WITH_YOUR_FIRST_DATACENTER - clusters: - - REPLACE_WITH_YOUR_FIRST_CLUSTER: - resource_pool: REPLACE_WITH_YOUR_FIRST_RESOURCE_POOL - - name: z2 - datacenter_name: REPLACE_WITH_YOUR_SECOND_DATACENTER - clusters: - - REPLACE_WITH_YOUR_SECOND_CLUSTER: - resource_pool: REPLACE_WITH_YOUR_SECOND_RESOURCE_POOL - - name: z3 - datacenter_name: REPLACE_WITH_YOUR_THIRD_DATACENTER - clusters: - - REPLACE_WITH_YOUR_THIRD_CLUSTER: - resource_pool: REPLACE_WITH_YOUR_THIRD_RESOURCE_POOL + vsphere: + datacenters: + - name: z1 + datacenter_name: REPLACE_WITH_YOUR_FIRST_DATACENTER + clusters: + - REPLACE_WITH_YOUR_FIRST_CLUSTER: + resource_pool: REPLACE_WITH_YOUR_FIRST_RESOURCE_POOL + - name: z2 + datacenter_name: REPLACE_WITH_YOUR_SECOND_DATACENTER + clusters: + - REPLACE_WITH_YOUR_SECOND_CLUSTER: + resource_pool: REPLACE_WITH_YOUR_SECOND_RESOURCE_POOL + - name: z3 + datacenter_name: REPLACE_WITH_YOUR_THIRD_DATACENTER + clusters: + - REPLACE_WITH_YOUR_THIRD_CLUSTER: + resource_pool: REPLACE_WITH_YOUR_THIRD_RESOURCE_POOL + subnet_names: + mysql1: REPLACE_WITH_YOUR_SUBNET_NAME_1 #This should match your vSphere network name, e.g. "VM Network". + mysql2: REPLACE_WITH_YOUR_SUBNET_NAME_2 + mysql3: REPLACE_WITH_YOUR_SUBNET_NAME_3 iaas_settings: <<: (( merge || nil )) @@ -36,7 +41,7 @@ iaas_settings: - 10.0.0.3 - 8.8.8.8 cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME #This should match your vSphere network name, e.g. "VM Network". + name: (( properties.vsphere.subnet_names.mysql1 )) - name: mysql2 subnets: - range: 10.0.1.0/24 @@ -50,7 +55,7 @@ iaas_settings: - 10.0.1.3 - 8.8.8.8 cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME_2 + name: (( properties.vsphere.subnet_names.mysql2 )) - name: mysql3 subnets: - range: 10.0.2.0/24 @@ -64,7 +69,7 @@ iaas_settings: - 10.0.2.3 - 8.8.8.8 cloud_properties: - name: REPLACE_WITH_YOUR_SUBNET_NAME_3 + name: (( properties.vsphere.subnet_names.mysql3 )) stemcell: &stemcell name: bosh-vsphere-esxi-ubuntu-trusty-go_agent @@ -74,8 +79,8 @@ iaas_settings: disk: 20480 cpu: 4 datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) + - name: (( .properties.vsphere.datacenters.z1.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z1.clusters )) resource_pool_cloud_properties: - name: mysql_z1 cloud_properties: @@ -83,8 +88,8 @@ iaas_settings: disk: 10480 cpu: 2 datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) + - name: (( .properties.vsphere.datacenters.z1.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z1.clusters )) - name: mysql_z2 cloud_properties: @@ -92,8 +97,8 @@ iaas_settings: disk: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.disk )) cpu: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.cpu )) datacenters: - - name: (( .properties.datacenters.z2.datacenter_name )) - clusters: (( .properties.datacenters.z2.clusters )) + - name: (( .properties.vsphere.datacenters.z2.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z2.clusters )) - name: mysql_z3 cloud_properties: @@ -101,8 +106,8 @@ iaas_settings: disk: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.disk )) cpu: (( iaas_settings.resource_pool_cloud_properties.mysql_z1.cloud_properties.cpu )) datacenters: - - name: (( .properties.datacenters.z3.datacenter_name )) - clusters: (( .properties.datacenters.z3.clusters )) + - name: (( .properties.vsphere.datacenters.z3.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z3.clusters )) - name: arbitrator_z3 cloud_properties: @@ -110,8 +115,8 @@ iaas_settings: disk: 4096 cpu: 1 datacenters: - - name: (( .properties.datacenters.z3.datacenter_name )) - clusters: (( .properties.datacenters.z3.clusters )) + - name: (( .properties.vsphere.datacenters.z3.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z3.clusters )) - name: proxy_z1 cloud_properties: @@ -119,8 +124,8 @@ iaas_settings: disk: 4096 cpu: 2 datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) + - name: (( .properties.vsphere.datacenters.z1.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z1.clusters )) - name: proxy_z2 cloud_properties: @@ -128,8 +133,8 @@ iaas_settings: disk: (( iaas_settings.resource_pool_cloud_properties.proxy_z1.cloud_properties.disk )) cpu: (( iaas_settings.resource_pool_cloud_properties.proxy_z1.cloud_properties.cpu )) datacenters: - - name: (( .properties.datacenters.z2.datacenter_name )) - clusters: (( .properties.datacenters.z2.clusters )) + - name: (( .properties.vsphere.datacenters.z2.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z2.clusters )) - name: cf-mysql-broker_z1 cloud_properties: @@ -137,8 +142,8 @@ iaas_settings: disk: 4096 cpu: 1 datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) + - name: (( .properties.vsphere.datacenters.z1.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z1.clusters )) - name: cf-mysql-broker_z2 cloud_properties: @@ -146,8 +151,8 @@ iaas_settings: disk: (( iaas_settings.resource_pool_cloud_properties.cf-mysql-broker_z1.cloud_properties.disk )) cpu: (( iaas_settings.resource_pool_cloud_properties.cf-mysql-broker_z1.cloud_properties.cpu )) datacenters: - - name: (( .properties.datacenters.z2.datacenter_name )) - clusters: (( .properties.datacenters.z2.clusters )) + - name: (( .properties.vsphere.datacenters.z2.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z2.clusters )) - name: errands_z1 cloud_properties: @@ -155,8 +160,8 @@ iaas_settings: disk: 4096 cpu: 1 datacenters: - - name: (( .properties.datacenters.z1.datacenter_name )) - clusters: (( .properties.datacenters.z1.clusters )) + - name: (( .properties.vsphere.datacenters.z1.datacenter_name )) + clusters: (( .properties.vsphere.datacenters.z1.clusters )) disk_pools: - name: mysql-persistent-disk From f4bf05d8a6e63f404d79d3658383fc2aff09ee52 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Wed, 13 Jan 2016 14:25:21 -0800 Subject: [PATCH 049/110] Bump src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer Bump : Aditya Anchuri: Group by users when selecting violators instead of db Morgan Fine: Updated database.go to handle db and user Changed reformer and violator queries in quota enforcer to return DB and user Signed-off-by: Ben Calegari --- src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer index 6a74438a..bfcab576 160000 --- a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer +++ b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer @@ -1 +1 @@ -Subproject commit 6a74438a05c104a7dc113c523c8526488c874576 +Subproject commit bfcab576221c39304949275f91044814dccedb5c From e28ab9da58e2d04948f55a578944bd8d8bafe5ce Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Wed, 13 Jan 2016 14:45:58 -0800 Subject: [PATCH 050/110] Bump src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer Bump : Aditya Anchuri: Grant/revoke queries are not using exec with args anymore Signed-off-by: Aditya Anchuri --- src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer index bfcab576..df200556 160000 --- a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer +++ b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer @@ -1 +1 @@ -Subproject commit bfcab576221c39304949275f91044814dccedb5c +Subproject commit df20055636916a26a1f50328063014fcffb56111 From bb48dfa13e7b331b5cbe64fe91073afc92f22564 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Thu, 14 Jan 2016 11:07:13 -0800 Subject: [PATCH 051/110] Bump src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests Bump : Ben Calegari: Disabled CF connection quota pending completion of proxy sync epic Signed-off-by: Ben Calegari --- src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests b/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests index 86e4cfa3..e0eece0b 160000 --- a/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests +++ b/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests @@ -1 +1 @@ -Subproject commit 86e4cfa30671904938b0951a8ad30203310ec72f +Subproject commit e0eece0bb70ca3b2d46faeb5db6bf79a0ad8a8b6 From 9c9824d1cc694acf61c5edda111ea7a210f22715 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Thu, 14 Jan 2016 16:44:53 -0800 Subject: [PATCH 052/110] Changed default_cluster_ips property to also include arbitrator IP - Now default cluster IPs potentially include all 4 IPs (3 mysql nodes plus arbitrator, unless any are nil) [#110772724] Signed-off-by: Ben Calegari --- manifest-generation/cf-mysql-template.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 2bf3bc4d..5dcc3a7b 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -299,7 +299,8 @@ empty_hash: {} default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) -- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +- (( jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) +- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) default_proxy_ips: - (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) - (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) From 47250fc60ea248894be939158ce02892e0ee4b49 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Thu, 14 Jan 2016 18:30:02 -0800 Subject: [PATCH 053/110] Added documentation for arbitrator node, and upgrade path to it [#110772724] Signed-off-by: Aditya Anchuri --- docs/arbitrator.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 docs/arbitrator.md diff --git a/docs/arbitrator.md b/docs/arbitrator.md new file mode 100644 index 00000000..f23a91a9 --- /dev/null +++ b/docs/arbitrator.md @@ -0,0 +1,33 @@ +# Arbitrator node + +As part of CF MySQL v26, we provide an [arbitrator node](http://galeracluster.com/documentation-webpages/arbitrator.html) as a replacement for one of the MySQL nodes. The arbitrator is a Galera node which does not participate in database transactions or data replication, but just votes in order to maintain quorum. This enables the user to save on resource cost (since the arbitrator is pretty lightweight compared to a normal database node), while still avoiding split-brain conditions which can happen when one of the database nodes is unreachable. + +In other words, we are going from a 3-node configuration to a 2-node-plus-arbitrator configuration as part of v26. In a typical bosh deployment of CF MySQL, the node is called `arbitrator_z3` and replaces the earlier `mysql_z3` node. + +### Deploying 2-node-plus-arbitrator configuration as fresh deploy + +For a fresh deployment of CF MySQL v26, simply follow the steps in the [README](https://github.com/cloudfoundry/cf-mysql-release/blob/develop/README.md#create-manifest-and-deploy). + +### Upgrading to 2-node-plus-arbitrator from already existing 3-node deployment + +If you already have a 3-node deployment of CF MySQL (e.g. v25), follow the below steps to upgrade to the new configuration as a rolling deploy: + +1. Generate a 3 node + arbitrator manifest: `./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > .yml` +1. `bosh deployment .yml` +1. `bosh deploy` +1. Upon successful deployment, generate the 2 node + arbitrator manifest: +`./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml > .yml` +1. `bosh deployment .yml` +1. `bosh deploy` + +### Rolling from 2-node-plus-arbitrator back to 3-node deployment + +If you wish to go back to a 3-node deployment from a 2-node-plus-arbitrator deployment, follow the steps below to perform the downgrade: + +1. Generate a 3-node + arbitrator manifest: `./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > .yml` +1. `bosh deployment .yml` +1. `bosh deploy` +1. Upon successful deployment, generate the 3 node manifest: +`./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/no-arbitrator/instance-count-overrides.yml > .yml` +1. `bosh deployment .yml` +1. `bosh deploy` From d688bd3bbaf8c641d35cf1f9bfc521ad29f8910f Mon Sep 17 00:00:00 2001 From: Marco Nicosia Date: Thu, 14 Jan 2016 23:17:21 -0800 Subject: [PATCH 054/110] Add two notes re: configuration of Load Balancers --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 89fa5b70..3904d100 100644 --- a/README.md +++ b/README.md @@ -252,6 +252,9 @@ Once a load balancer is configured, the brokers will hand out the address of the Currently, load balancing requests across both proxies can increase the possibility of deadlocks. See the [routing](https://github.com/cloudfoundry/cf-mysql-release/blob/master/docs/proxy.md#consistent-routing) documentation for more information. To avoid this problem, configure the load balancer to route requests to the second proxy only in the event of a failure. +- **Note:** When using an Elastic Load Balancer (ELB) on Amazon, make sure to create the ELB in the same VPC as your cf-mysql deployment +- **Note:** For all load balancers, take special care to configure health checks to use the health_port of the proxies (default 1936). Do not configure the load balancer to use port 3306. + ### Deployment Components From bfbaf90c05c34e49c95ff8328e8d780d760c1649 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Fri, 15 Jan 2016 13:27:37 -0800 Subject: [PATCH 055/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Fixed mariadb_helper test, checks call counts ( Travis Unknown ) Refactor mariadb_helper Seed function to smaller methods ( Travis Unknown ) Morgan Fine: Add interface to Seeder and don't export struct ( Travis Unknown ) Extract CreateDBIfNeeded to Seeder object ( Travis Unknown ) Ben Calegari: Backfill tests on Seeder object ( Travis Unknown ) Extract Seed functionality into Seeder object ( Travis Unknown ) Signed-off-by: Aditya Anchuri --- packages/mariadb_ctrl/spec | 2 ++ src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/packages/mariadb_ctrl/spec b/packages/mariadb_ctrl/spec index 7246f631..f62f90e3 100644 --- a/packages/mariadb_ctrl/spec +++ b/packages/mariadb_ctrl/spec @@ -12,6 +12,8 @@ files: - github.com/cloudfoundry/mariadb_ctrl/integration_test/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/mariadb_helper/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/mariadb_helper/fakes/*.go # gosub + - github.com/cloudfoundry/mariadb_ctrl/mariadb_helper/seeder/*.go # gosub + - github.com/cloudfoundry/mariadb_ctrl/mariadb_helper/seeder/fakes/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/os_helper/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/os_helper/fakes/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/start_manager/*.go # gosub diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index fe7eaa82..cdcb83f9 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit fe7eaa822024504272a7847d8af7f1a981af3069 +Subproject commit cdcb83f95cfd5b158f217a639afc00a9427e9f26 From 6ac256ebfb7e84f01139bdafb9f5a3de31f05902 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 18 Jan 2016 10:15:26 -0800 Subject: [PATCH 056/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Set up read-only user creation in seeder and mariadb_helper ( Travis Unknown ) --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index cdcb83f9..8b1eba96 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit cdcb83f95cfd5b158f217a639afc00a9427e9f26 +Subproject commit 8b1eba968cd8ee97a80f5f26869fb9ad53c070a8 From e5ecff347aa936f21791c0e6c3a7ba29976dbc8d Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 18 Jan 2016 12:34:30 -0800 Subject: [PATCH 057/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Created monitored call to mysqld_safe ( Travis Unknown ) --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 8b1eba96..07bdf890 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 8b1eba968cd8ee97a80f5f26869fb9ad53c070a8 +Subproject commit 07bdf8908ee0ddc7467c14bea1787256bbd13964 From 412354144c27119966b329b47f0465817b69b549 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 18 Jan 2016 14:11:39 -0800 Subject: [PATCH 058/110] Add default read-only user name [#110369648] --- jobs/mysql/templates/mariadb_ctl_config.yml.erb | 1 + 1 file changed, 1 insertion(+) diff --git a/jobs/mysql/templates/mariadb_ctl_config.yml.erb b/jobs/mysql/templates/mariadb_ctl_config.yml.erb index ff5a7cf0..6c6bdb2b 100644 --- a/jobs/mysql/templates/mariadb_ctl_config.yml.erb +++ b/jobs/mysql/templates/mariadb_ctl_config.yml.erb @@ -17,6 +17,7 @@ Db: User: <%= seed["username"]%> Password: <%= seed["password"] %> <% end %> + ROUser: roadmin Upgrader: PackageVersionFile: /var/vcap/packages/mariadb/VERSION LastUpgradedVersionFile: /var/vcap/store/mysql/mysql_upgrade_info From 76a2bb7ffbfc9dbdadac8e120f0921fe8a031749 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 19 Jan 2016 11:44:54 -0800 Subject: [PATCH 059/110] Add creation of read only user to mariadb_ctrl - Quota enforcer does not modify roadmin - Username is hardcoded to roadmin and not configurable via job spec [#110369648] Bump cloudfoundry/mariadb_ctrl: Ben Calegari: Reformat fakes ( Travis Unknown ) Add .test files to .gitignore ( Travis Unknown ) Add go fmt to bin/test ( Travis Unknown ) Restructure mariadb_helper_test and regenerate fakes [#110369648] ( Travis Unknown ) test: Revert "Added functionality to wait on joining mysql cluster" ( Travis Unknown ) Revert "Created monitored call to mysqld_safe" ( Travis Unknown ) Revert "Added monitorJoin field to startManager to be used for prestart" ( Travis Unknown ) Aaron Hurley: Create read only user from start_manager ( Travis Unknown ) mariadb_helper can update Read Only User password ( Travis Unknown ) mariadb_helper Seed function creates Read Only user ( Travis Unknown ) add function to create read only user ( Travis Unknown ) Aditya Anchuri: Added functionality to wait on joining mysql cluster ( Travis Unknown ) Added monitorJoin field to startManager to be used for prestart ( Travis Unknown ) Bump pivotal-cf-experimental/cf-mysql-quota-enforcer: Ben Calegari: Do not enforce a quota on read only user ( Travis Unknown ) Move reformer_test to reformer_repo_test to match violator_repo ( Travis Unknown ) Remove debug log lines ( Travis Unknown ) Signed-off-by: Morgan Fine --- jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb | 1 + jobs/mysql/spec | 2 ++ jobs/mysql/templates/mariadb_ctl_config.yml.erb | 2 ++ manifest-generation/bosh-lite-stubs/property-overrides.yml | 1 + manifest-generation/cf-mysql-template.yml | 2 ++ manifest-generation/examples/property-overrides.yml | 1 + src/github.com/cloudfoundry/mariadb_ctrl | 2 +- src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer | 2 +- 8 files changed, 11 insertions(+), 2 deletions(-) diff --git a/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb b/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb index f9d19ba5..be556a45 100644 --- a/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb +++ b/jobs/cf-mysql-broker/templates/quota_enforcer_config.yml.erb @@ -4,3 +4,4 @@ Port: <%= p('cf_mysql.mysql.port') %> User: <%= p('cf_mysql.mysql.admin_username') %> Password: <%= p('cf_mysql.mysql.admin_password') %> DBName: mysql_broker +ReadOnlyUser: roadmin diff --git a/jobs/mysql/spec b/jobs/mysql/spec index 95b911f1..0a7872d4 100644 --- a/jobs/mysql/spec +++ b/jobs/mysql/spec @@ -78,6 +78,8 @@ properties: cf_mysql.mysql.seeded_databases: description: 'Set of databases to seed' default: {} + cf_mysql.mysql.roadmin_password: + description: 'Password for the MySQL server read-only admin user' cf_mysql.mysql.database_startup_timeout: description: 'How long the startup script waits for the database to come online (in seconds)' cf_mysql.mysql.bootstrap_endpoint.username: diff --git a/jobs/mysql/templates/mariadb_ctl_config.yml.erb b/jobs/mysql/templates/mariadb_ctl_config.yml.erb index 6c6bdb2b..750edfa6 100644 --- a/jobs/mysql/templates/mariadb_ctl_config.yml.erb +++ b/jobs/mysql/templates/mariadb_ctl_config.yml.erb @@ -11,6 +11,8 @@ Db: UpgradePath: /var/vcap/packages/mariadb/bin/mysql_upgrade User: <%= p('cf_mysql.mysql.admin_username')%> Password: <%= p('cf_mysql.mysql.admin_password')%> + ReadOnlyUser: roadmin + ReadOnlyPassword: <%= p('cf_mysql.mysql.roadmin_password')%> PreseededDatabases: <% p("cf_mysql.mysql.seeded_databases").each do |seed| %> - DBName: <%= seed["name"] %> diff --git a/manifest-generation/bosh-lite-stubs/property-overrides.yml b/manifest-generation/bosh-lite-stubs/property-overrides.yml index 1c85af59..25515a5f 100644 --- a/manifest-generation/bosh-lite-stubs/property-overrides.yml +++ b/manifest-generation/bosh-lite-stubs/property-overrides.yml @@ -1,6 +1,7 @@ property_overrides: mysql: admin_password: password + roadmin_password: password bootstrap_endpoint: username: username password: password diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 14661233..7320b538 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -65,6 +65,7 @@ properties: healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) bootstrap_endpoint: (( property_overrides.mysql.bootstrap_endpoint )) cluster_ips: (( property_overrides.mysql.cluster_ips || default_mysql_cluster_ips )) + roadmin_password: (( property_overrides.mysql.roadmin_password )) seeded_databases: (( property_overrides.mysql.seeded_databases || nil )) database_startup_timeout: (( property_overrides.mysql.database_startup_timeout || 600 )) max_heap_table_size: (( property_overrides.mysql.max_heap_table_size || nil )) @@ -351,6 +352,7 @@ properties: healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) bootstrap_endpoint: (( property_overrides.mysql.bootstrap_endpoint )) cluster_ips: (( property_overrides.mysql.cluster_ips || default_mysql_cluster_ips )) + roadmin_password: (( property_overrides.mysql.roadmin_password )) seeded_databases: (( property_overrides.mysql.seeded_databases || nil )) database_startup_timeout: (( property_overrides.mysql.database_startup_timeout || 600 )) max_heap_table_size: (( property_overrides.mysql.max_heap_table_size || nil )) diff --git a/manifest-generation/examples/property-overrides.yml b/manifest-generation/examples/property-overrides.yml index 38222dfa..4f44266a 100644 --- a/manifest-generation/examples/property-overrides.yml +++ b/manifest-generation/examples/property-overrides.yml @@ -5,6 +5,7 @@ property_overrides: host: REPLACE_WITH_LB_HOSTNAME # Optional, set to your Load Balancer address if configured; delete this line otherwise mysql: admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process + roadmin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP bootstrap_endpoint: username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 07bdf890..2952dc7d 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 07bdf8908ee0ddc7467c14bea1787256bbd13964 +Subproject commit 2952dc7d530ae8e69b16499391c4fa66cfcf5231 diff --git a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer index df200556..87302d35 160000 --- a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer +++ b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer @@ -1 +1 @@ -Subproject commit df20055636916a26a1f50328063014fcffb56111 +Subproject commit 87302d350992ec23a19083f8a2aa3b7052fa2519 From 299261835c5007242377819d10ffe13166d7c8f1 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 19 Jan 2016 11:59:41 -0800 Subject: [PATCH 060/110] Add missing roadmin key to sample stubs Signed-off-by: Morgan Fine --- .../bosh-lite-stubs/property-overrides-standalone.yml | 1 + manifest-generation/examples/standalone/property-overrides.yml | 1 + 2 files changed, 2 insertions(+) diff --git a/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml index a3f492dd..dfbb9c13 100644 --- a/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml +++ b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml @@ -1,5 +1,6 @@ property_overrides: standalone: true + roadmin_password: password mysql: admin_password: password bootstrap_endpoint: diff --git a/manifest-generation/examples/standalone/property-overrides.yml b/manifest-generation/examples/standalone/property-overrides.yml index 02873682..006ea717 100644 --- a/manifest-generation/examples/standalone/property-overrides.yml +++ b/manifest-generation/examples/standalone/property-overrides.yml @@ -4,6 +4,7 @@ property_overrides: host: YOUR_LOAD_BALANCER_ADDRESS # Optional, set to your Load Balancer address if configured mysql: admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process + roadmin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP bootstrap_endpoint: username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP From 6d33c14055c0006fa14db6e604afa533100f6047 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 19 Jan 2016 13:53:13 -0800 Subject: [PATCH 061/110] Write mysql err logs to disk when syslog is enabled - This way we always write mysql error logs to disk and syslog is still optional [#111344064] Signed-off-by: Morgan Fine --- jobs/mysql/templates/my.cnf.erb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/mysql/templates/my.cnf.erb b/jobs/mysql/templates/my.cnf.erb index 5ee075d0..8574ba4a 100644 --- a/jobs/mysql/templates/my.cnf.erb +++ b/jobs/mysql/templates/my.cnf.erb @@ -50,7 +50,7 @@ datadir = /var/vcap/store/mysql tmpdir = /var/vcap/data/mysql/tmp language = /var/vcap/packages/mariadb/share/english pid-file = /var/vcap/sys/run/mysql/mysql.pid -<% unless use_syslog%>log-error = /var/vcap/sys/log/mysql/mysql.err.log<%end%> +log-error = /var/vcap/sys/log/mysql/mysql.err.log init-file = /var/vcap/jobs/mysql/config/mariadb_init skip-external-locking = TRUE max_allowed_packet = 256M From 1aa2f3caccb5671ffca5f5c9b85b940c44aa4274 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 19 Jan 2016 15:15:34 -0800 Subject: [PATCH 062/110] Move roadmin_password into mysq instead of global - Whooops. Signed-off-by: Morgan Fine --- .../bosh-lite-stubs/property-overrides-standalone.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml index dfbb9c13..7d5013f8 100644 --- a/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml +++ b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml @@ -1,8 +1,8 @@ property_overrides: standalone: true - roadmin_password: password mysql: admin_password: password + roadmin_password: password bootstrap_endpoint: username: username password: password From 6a34d7ec41a22d2ad9ecdf5ff6cc74b11fd69fa6 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Thu, 21 Jan 2016 17:06:34 -0800 Subject: [PATCH 063/110] Removed duplicate property_overrides section, added section for backups [#111004656] --- manifest-generation/cf-mysql-template.yml | 100 ++++------------------ 1 file changed, 15 insertions(+), 85 deletions(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 7320b538..07e25efa 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -119,6 +119,21 @@ properties: smoke_tests_only: (( property_overrides.acceptance_tests.smoke_tests_only || nil )) standalone_tests_only: (( property_overrides.acceptance_tests.standalone_tests_only || nil )) timeout_scale: (( property_overrides.acceptance_tests.timeout_scale || nil )) + cf-mysql-backup: + symmetric_key: (( property_overrides.cf-mysql-backup.symmetric_key || nil )) + endpoint_credentials: + username: (( property_overrides.cf-mysql-backup.endpoint_credentials.username || nil )) + password: (( property_overrides.cf-mysql-backup.endpoint_credentials.password || nil )) + service-backup: + cron_schedule: (( property_overrides.service-backup.cron_schedule || nil )) + blobstore: + bucket_name: (( property_overrides.service-backup.blobstore.bucket_name || nil )) + bucket_path: (( property_overrides.service-backup.blobstore.bucket_path || nil )) + access_key_id: (( property_overrides.service-backup.blobstore.access_key_id || nil )) + secret_access_key: (( property_overrides.service-backup.blobstore.secret_access_key || nil )) + source_folder: (( property_overrides.service-backup.source_folder || nil )) + source_executable: (( property_overrides.service-backup.source_executable || nil )) + cleanup_executable: (( property_overrides.service-backup.cleanup_executable || nil )) # The keys below should not be included in the final stub config_from_cf: (( merge )) @@ -322,91 +337,6 @@ base_jobs: - name: mysql1 properties: {} -properties: - nats: - machines: (( config_from_cf.nats.machines )) - user: (( config_from_cf.nats.user )) - password: (( config_from_cf.nats.password )) - port: (( config_from_cf.nats.port )) - app_domains: (( config_from_cf.app_domains )) - domain: (( config_from_cf.domain )) - cf: - api_url: (( config_from_cf.api_url )) - app_domains: (( config_from_cf.app_domains )) - admin_username: (( config_from_cf.admin_username )) - admin_password: (( config_from_cf.admin_password )) - skip_ssl_validation: (( config_from_cf.skip_ssl_validation || nil )) - smoke_tests: - use_existing_org: (( config_from_cf.smoke_tests.use_existing_org || nil )) - org: (( config_from_cf.smoke_tests.org || nil )) - syslog_aggregator: (( property_overrides.syslog_aggregator || nil )) - cf_mysql: - standalone: (( property_overrides.standalone || false )) - external_host: (( "p-mysql." .properties.domain )) - host: (( property_overrides.host || jobs.proxy_z1.networks.mysql1.static_ips.[0] )) - mysql: - admin_username: (( property_overrides.mysql.admin_username || nil )) - admin_password: (( property_overrides.mysql.admin_password )) - port: (( property_overrides.mysql.port || nil )) - galera_port: (( property_overrides.mysql.galera_port || nil )) - healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) - bootstrap_endpoint: (( property_overrides.mysql.bootstrap_endpoint )) - cluster_ips: (( property_overrides.mysql.cluster_ips || default_mysql_cluster_ips )) - roadmin_password: (( property_overrides.mysql.roadmin_password )) - seeded_databases: (( property_overrides.mysql.seeded_databases || nil )) - database_startup_timeout: (( property_overrides.mysql.database_startup_timeout || 600 )) - max_heap_table_size: (( property_overrides.mysql.max_heap_table_size || nil )) - tmp_table_size: (( property_overrides.mysql.tmp_table_size || nil )) - wsrep_max_ws_rows: (( property_overrides.mysql.wsrep_max_ws_rows || nil )) - wsrep_max_ws_size: (( property_overrides.mysql.wsrep_max_ws_size || nil )) - skip_name_resolve: (( property_overrides.mysql.skip_name_resolve || nil )) - gcache_size: (( property_overrides.mysql.gcache_size || nil )) - ib_log_file_size: (( property_overrides.mysql.ib_log_file_size || nil )) - innodb_buffer_pool_size: (( property_overrides.mysql.innodb_buffer_pool_size || nil )) - max_connections: (( property_overrides.mysql.max_connections || nil )) - persistent_disk: (( disk_pools.mysql-persistent-disk.disk_size )) - proxy: - api_username: (( property_overrides.proxy.api_username )) - api_password: (( property_overrides.proxy.api_password )) - health_port: (( property_overrides.proxy.health_port || nil )) - api_force_https: (( property_overrides.proxy.api_force_https || nil )) - proxy_ips: (( property_overrides.proxy.proxy_ips || default_proxy_ips )) - arbitrator_ip: (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || nil )) - port: (( property_overrides.proxy.port || nil )) - healthcheck_timeout_millis: (( property_overrides.proxy.healthcheck_timeout_millis || nil )) - api_port: (( property_overrides.proxy.api_port || nil )) - broker: - max_user_connections_default: (( property_overrides.broker.max_user_connections_default || nil )) - ssl_enabled: (( property_overrides.broker.ssl_enabled || nil )) - auth_username: (( property_overrides.broker.auth_username )) - auth_password: (( property_overrides.broker.auth_password )) - cookie_secret: (( property_overrides.broker.cookie_secret )) - external_host: (( property_overrides.broker.external_domain || .properties.cf_mysql.external_host )) - protocol: (( property_overrides.broker.protocol || nil )) - port: (( property_overrides.broker.port || nil )) - services: - - name: p-mysql - plan_updateable: true - id: (( property_overrides.broker.service_id || "44b26033-1f54-4087-b7bc-da9652c2a539" )) - description: MySQL databases on demand - tags: - - mysql - metadata: - displayName: "MySQL for Pivotal Cloud Foundry" - imageUrl: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAJo5JREFUeNrsXQtsVNeZvvPwzBiD3zyNwxgWhEKqmEibR9skdhQtaao2Rqtks4+qthKp1e5KxKhVCg3BbrKOst3lIW2rrUQVR5W2adgKp92ikG1ik2RbkpWCs4WWQomHVyAY8NhgPDOex97v+F4zHs97zrlz7z3/Jw1j/Ji5c+/9vv9x/v8/DoVgK9Rt7/arT37tv7XqozXT7zorK1clotFgYnp6PMtLDquPoPZ1YKxvd4DOsn3goFNgOYK3asRuUx81SQRvM/hQhpIEYlz7f1AViGG6SiQABH6WHMS+U/u61SKHDxGAp/CxJgzDqjAE6aqSABAyE75NI/iD2rPfZh8xoAnDYdfCquEr218coqtOAiC7K9+hEb5N0tMw5PBUHE5EpgcodCABsDvha5MI36HF8YRbQIgw4PR5D8dD4QEKGUgA7BLHg+yPSWzli7sx3e4hxeV8w11bO3B5y7YAnRESAKtZ+i2KdZJ2pobT6x2Oh8N74SGQZ0ACYFbig/Rf18hPEHGzulzqP44BR0XFq1d39A3QGSEBMIOLD0vfSTG94WIQdHi9/a5FC/dSiEACYCgaXtjeEZ+aImtvnnzBQCIa3auGB0N0NkgARFp8WPqdiv3W6O1xM3s9gUQ40qsKQT+dDRIAXqSHa/+M5uqTm2+R8MBZ6dvrXe3fc+HJpylpSAJAxCchICEgASDikxCQEJAAZCF/DxHf/kJwZfuLPXQ2SACSid+pUHJPnpve6wk4fb7eK8/29pMAyE38No34bUQLCW9+t3vI3VDfe3nLtiESAPni/N3KTAEPQe6wQHF4PP2+tau7ZcwPOCQkf6dGforzCXPzAwurumULCxwSER/x/Ssyufuwbs5K35zvOauqsv5NfHJy7v+nQkoiFpOGEE6fd8hVU9MlS3mxQxLy9yg2zO67Fqpkdjox3FMlu3OG7M75pOcFXRxmRCGuPk8pielp9n+bIeiuq907+u2dPSQA1rf6BxSLt+UyS161gBEd5Hb6fGrcWmGqY2SioIkBhMIOnoOWJLS1N+CwMflRzLPTilYf5Haprroak7Jns5G9EFGIh9THjUklpopCIjJtydyAq66md3Trjj0kANYgfq0W63dY6CZTXNWLLE/4XIAAxCYmlBgEYeK6xXIDvgHfujVddlspcNiM/G2ay296q8+sfHW14q5ZlDMxZ1dABGLjE+zZCuGCw+NBbmCzneoGHDYif4/m8pvb0tfVKupNJCxRR2JgiLfWa5cEocMGxDe9yw/33l1fx54JuREdC86KgWmv6aKFA96WVZYPCRwWJ3+rRn7TZfnh4rvr6hR3Y/3MLDtCUTkDiEF0bMyUCUSH2zXsaVrRdekbW4dJAIwnf4dGflPF+1ib1918Al+vIHrlqulqDpAXUK951+i3nh8gATCO/Fji222mYwLh3fW10ib0jAJqDKY/G2UrCabKC9TVdFtxqdBhQfLD6neaifgVS5fYdunOzOHB9GeXmWdgmnuhvq5f9QS6SADEEN9UyT4ivnmEIHL+gmk8AqvVCzgsRP5BxQTJPsT4FUsXk6tPoUFmEahaMOxb09JuBRFwEPnzPFGqpfc2NxHxTQ4sHUY+vVj2VQOIQHzyZrvZtzJzmJz8ID0q+/xlO0Eul+JubGBWn2AdwBvAqkE5i4rUeyfgWblis5mXCR0mJz8sf9nW01C441mxnOJ8K+cHVG+gnAVFWCb0LF/ablYRcBD507v7ID5V7tkoLDh3oWzegJlFwEHknwvd3afqPZt5Ayr59bCARMCkAlBO8oPwnuYmsvrkDUglAg4TkR+kP6qUIeHHYn2V/GT15fEGIAJlyg2A/KZZHXCaiPyD5SC/Z8Uyxeu/jcgvEXCtcc1x7cuAVmfVgsGm1/aZolnEYSLyG7rOz9b1V91GffmSA81F4TNnDa8bMEuxkBk8AMPbeVHN51u7hshPYPcA7gWjcz/xyZutoZOnXyn35y+r36s19jxp5Hsiy++9baXicDrp7ifMeIPqveCurWEOceq+CEJzEdHo+kWPPeq/+dbgG9IJgNbS+x1D4/3mJqViSSPd8YSMnqHD44F1VtmZMCoEaV20+cvjNw8NHpEmB6AN8zhg2IdE0me1n1x+Qv55gU8Chi0VsnkCNdWbyzFUxFEG8hu61g/Se1Y2EfkJhbnnkWmWHDRqAlG5agScBpNf7+k3jPxk+QnFEbLC0HsnEYnURi58+orRy4NGZ8IMy/jr5Kf1fUKpoSPbg9EIEYjGWsMjZwxdGTCMHdrc/m8a8V6Y1uNBpp/ITyhVBLBCoN5PCAmwzZkBnsD6RV/9kuPmfw8N2SYHoO3YM2gY+Zub6M4lcAfKh42YQajNoGg3YgcihwHkR0wzYkTcT+Qn2EYEPJ5g5fq1LaIrBY3IARiS9NMbeggEkcA9ZsSeD0gKGlEpKDRI1op9nhH9IVjCDw09VN1HMACummpWLCS6f4BVCgouEnIIJL9fmWnvrRVOfsr2F4zmqoXKyqrs9e8TkYhyPHiVTlY6csZirFhIdJ2Ael8H3Y0NGy9v2RYQEjYLPPYDRH7jUV3hUTbUNcwSfENtPftetUf9fm1DSa99fvKGcm5ypof+t6OXVIEIqwJxTf3+dfX7N6Q6z/oSoWgRUIWmNnr1GkKBdst4AEZs1U3lvTO4b8nymcfiZSrpF6mkX1iW45iYVr2FsatMGNjz5Yvse7b3BNQwIHTqtPCyYXddrZAtyR0CyG+I6y9rOy8s+6amVYz0eDYzED4cunBWOXT+jK1DCYN6B4IVS5dwDwVEhADCs/7IxMpG/ida1iqP+9cy4lsFCDnw2LphIwsf9p08prw+csp2ngHuxYoVy9gSoUDUxsbHuYcCXD0A1fp3agIgzhVqbCjXKKeyEX/rhrvK5tqLCBX2nTyu7PvjMdsJAaYOY8NSkXDVVHddeba333QCYETBD2qyEffL4urvuvsBS1n8QgCPoPvDd1muwE4IB84KHTaKVYHKDeu5FQjxXDjfLZL8bLOOVc1SkB9u86FNm21LfgAezf72R5mHYyew6dICd5LCqkDo1Ce7uYUvnKx/m/rUKfLEYoCnLMt9P/7iw2zpTgbAy2m2SXijWWh2r4pEIhLpXLL3pTYzeQBCl/wQ88uS9INFXGkjQuQD5DjsBDaERmCeCqsN0avXdppCALTEX5uoD4u4H4k/WZCrOs+u4YDdgHtW5KThRDTa1vjyzs6yC4BI68+265Ik7ifYD6J3m4qHQjvLKgBaxZ/fqieQQBCdDxDZoZoIR/yNfc/1lEUAtGW/LcJcf9V9oo06CVYH7mGRIWx8KrSllDmCpXgAaPMVsuwnWjkJBCMhcrt5LAuGPwk8Y6gAiLb+5PoTKBQwxgso1gMQZv2R9ZfZ9UcnnXSfOXhNilBA1H1dihdQsACItP5MKVfK7fofunDGduWx2YB+gF3HPpLis3pWLBfm2RbrBRTjAQiz/kiWiCyjtAqeev/XUkziAfkff+egFHMDmIFT721RCcFivYBiBGCLqJODZAnhFjHQNWdXwMvZdOiAdCPHWEJQkJGDF1BwaFKg+9+pCNrO29u8UnF6vcR+DeF4TBm6dJ5N2EGtfLNNKgTRBdgzfETpOfqBNJZ/ntX1eJRYcFyAG5DwVd5/35nQe0fy3l+w0IEgQqr+ZE/85bKUj6sPiMBT6+5gvQJWbBTaP3JKeT1wSqr8Rsb7HQlB9Z6P3Zjk70l7PeBof96/X4D1F7alt2+NX3FWVRHb84Q+Duy+JctKHvQp0tKD7G9qSU1ZrX1Gd31yUgmdDojxMCorN1/d0ZfXVuOFeABfF2X9ify3AAs/Homw1YBsXoFuSeENQBAwCRiDQfFcDg8Bx4PlvN8HZwaC5jslGAK2aeUqaVYCZkmq3vOivIDE9DS4mpcA5OUBaIM+R4RYf0mHe2bC1jvuYjP0QKJdx48W7TJDFPQR4ez/i5fN+VkhQKIOewTMfH2NWXN9FHixI8ER0qAN+HFV8FiYM3hQumuNicJTJ04Kee2KpUta8hkgmq8HICTzjy2WiPyZCbxffYAc+9XYGcM0C7XIQDZPolyf6wn/WkZ82cGWBVUOiNhrMHb9BjjbzUsAOoUIgER9/qUQBo+ejfey8dqIqc1G6nzcfBD+kaZV0g07ycNSCxGARDjcyUUAtOQf98IfFvuT9c8bcOdBIt1ysjzA6KU5+QAzER4JSl28ZBlvVqwXICIXgMKghhe2d+RKBubjAQhJ/lHRDx/PQNmwcTZOR+YdMTr6CRCnixYGEB1bjuE4mhcsZNbdzoNMxXkBi8uWDHTnsP6w/B28DwyWnzL//MnIMuppdgtKTuLBayhKcEpIIhJy8KFqxhvmvsdgItEBDo/17Q4W6wF0iPjAFPtnBjbcFCEOZiYv1QjMcIL3zkLaVmXgcH9G8cnxGtyz/3rmk5Aedtw6Kxfs3POQtwConBDRI+D0erNy2JnF/ferT638P2gdsTyHNew5ekSaz4sSYSoPFseNeDjcumTvS/5iPAAx7j9Z/7y8ALQE290TQKETtgcjiOVGNBjsKEYAHuN9IGiCoH7//IC1/nt/+TNmIe2GmVbgAenKf/MJj4U0xcXiGbnsyOD+Q4rGeB8H5qKRB1A49LJZ1MxbeU0dooZ4n1z+LNZ6LChqm/G6dKsBbqPcf4xCIvIXh3PaTrrVRz3KI6oIYKkv3XKfGYElSHgxIH8xPQMyhgHTn17SM/jc4PR5064GZBKAB0W4/4T8rH0moiAngPyA3hegFwNhjd4sy3vHWTfgrQrFfPIY2T6zjABXeJcHx0PhBwsRAO4egKummq5sHni8ZR1rloHFz+Uqp5YBY70f1XjoANxQWz/bDSgibNDfF4VFekdgoa49iI/dgdnnlrAbMBtXBPQHgNNdOXMAavyPpb+jvN3/yg3r6crmAb0dWCdZMZ2A6ZDcGjwjFqpAeLx5Ex0otvU3FakdgbK2A2fD1PET3MMAFRvH+nYP5/IA+Ft/cv+LJgoeSACiCxCxdLFDNFN7A4xOxMHaI28B0pt1ipHdwwCHpwLczikA/ON/cv9LAtz6p9dtYA80/Ohjtsw+aksXsE1NtxHpTRAGJCLTD+aTA2jj7f6TByBGDAB4BL8fu8ae0QlYriU21oxUV6/cXjuTf6CGIVN6zW1ZBUCN/9t4v6OzagFdTdHEw8ANZe0cdx8twfAWzt28Mc/l19uFC7HkqbkDllNQv8a4chryIU4EYhPXub5mY99zbVe2vziUyQPgXvtP7r/x0AeFzlWKjXRirCYAGBTCWQBiNybB8VkBcAqP/6nvvyDIWCVX7IwC+3sAQoznHI47RXoAqG2m2v/CBcBqM/9KAcKUfX88RhfeOP60phUArf3XT9a//Oj+4F0pRACJSxm6Hk3GIb/W6zPPA/DzfifnQhKAYgBCgBiYC2BXcqApCBugyrY5qEk41JpOANoo/jcfSfSWYLsIATwbtALbWdy4CoBPyOTsWa4nrwLcyTV+cbko/ufkDSR3Aj61boPlimoQ52Nj0P0jJ6npp1ABqPQxLnEuC74znQD4eR84ga8Q6J2Aelmtmefu6/sWYDMTcvNL5xLnseH+dALAdQWAxn4XB73CL9ugTFhR/Fz/ndQqPKM3CAXZZ/YkKLwaUW8MotFg2bnEWQBa5wiA1gHIXbUIhQNVdugGfHrdHcqu4x/l1Qk4QzxY2VPzyJX8rG/eke530pE61QMBudn7aZWEhVYUpr4vPud92v6HBGO5BM6jM1D3ALiP6hGUvJAGK7VeeXQCFhs/z/bsm4Rg8EqsmscoJxwVQnJptckhQBv3g6YEIDchgKXEw4rjtXTSW2mMmQwegMb5IV0Aani+sovW/4WAxfobG9hOwee1CTy/Hb2ouuLXTJNoQ4LydrY56HK2QShZen4iwHnrsJpkD4BvDsDppCtmgGeQvFuw7urrHYA84vRcMbz+rHcGGp18lC4M4CsArckCwFmtKumKlQHZevB1MZjzfy2pl9GapyQNqfW3nB5AJffOQGE5AIeLPIBigbl7omLxVIEwS0x+noqDysEpxnkhTKUlwOLx5vkz0pXIYpWDUB5OObUuQM6v6qIrViRAfjTJyCICW/MYf04QA3AfHgB3ASAPoDQgo2/3Tjm94/H1EbL+eXFKTGWtn4J1E4sA20Dz+FHbeQOoZUCXo0yDT8wKJAG5VgGic4nAD9hBFxNznmhZy8qDrZqF15uZfnzyGHUElsAtzl2BtRAAvk1A5P4LIY/e/MMmAKti8EjTKtOLAY4b3YDYx4CsPR9u8W4KctNptV5ocPzoVTZQA1V3M5uDLldur6sve9Wd3hE4s2nJJWoDtkgIQLAo4EqfG5m7d2DyBqF6IQ/PAp7k6kIUEhW7MSiBBIAgykNQH5lcbngNK6sK23VmIhIha04CkGecQoNAzO81UBLOmjkA/oNBFFoGJBBkFhU6BQSC3AJAUxoIBDlRQx4AgSC5B0AVGgSCnBgnD4BAkNwDIBAIJAB8EJ+cpLNKIAiACG6RB0AgkAdAIBBkBPUCWAzYOxDbh6UiWzMO1fITsgnAMNc4he/sckISdt/9wJx9AOZgw8aiX5ftJZBlGrG+10A6YGAJwaAcAH9usb0BgzxfkfPEEoIyM9J7/0OPCuv3X1m1sOh2YaO2/27Wjk/mRiYB3ApSCCA5+UsFPBIMKBEJjEPDRqm6N4IJQwh5aMpQ6XBoY8FHeL6ob42f2oIlIL9OyHv/62fCXh/7ICLvkQkQAgxOtftAEiwBhk4HeL9si3Osb3eAqEbkLyV8EHGM+Pw//uLDWckPYCQakpyEwgHuC1kGpERgaQCh3tq02TI76/IeTorPDfHLZ+syWH8ZVjhEcUrPAQwpHPcHTMTixOISb36r7LKLEIBnLA6LDsufz+cH8WVZhRDAqaFkAeCsVlPEZAnID/Dc1y852ZcLGEr69Pu/lubeEFVirwvAMFcPYHqa2CwB+QFsWsIDWWsc0rn+quWXaUlQwBLgcLIAjFMOoHyA5UO220rkZ7P/Ry+VvG1ZMclOvDc2SZEJAjg1npoD2Mn7gGmXIL5ur2hgzz7dpTdiWQ2kR7xfSBIRgoMdhYn8fHMAQd6vzsIAEgDLkJ+RfvSiYevpyPDvvueBgr0e2Vx/gSE14zxbBhzr2z3M+9UpDLAW+QGjiIW1/Xwz/cnAaoNsrr8oLumcd6ckBbhtFDqTtVxMTE+DrXfcpWwtoXlHFHo23qPt73ctZwyeDfrWYeni/d6N9xaU7Et2/bs/eFfK+0XACsCswU8WgABXASAPIC0KzXYbCcTkeOQswCmDeIH8pSYcyQOYw3VlNgTQ8DHXuCUWIxGwEPnNjEMSby8ODglYAvw4nQAMcT/4EAkAkb80yOz6C+TQ0DwBqFi6JMD94G/QgFDEvUR+cv1NxqHheQJwecu2QHJswAMxySYEI7mX2r2G2nYif3FAxl/2nn8BHAqM9e2eXfZ3p1EGP7c8QGSaPRyeCqlcfFis10dOzcavKLAhESgMWI2QfdyYzh9R1j81BwAc5q5gExO2d/EPbeqYQ3Cs7yeXtnZ/+K7tB1Zwd/0/lNv1F8idwxkFwLWwintBUMzGeQC9gSddHXvq9596/9c0mbcA158EUxh3MnsAV7a/OMRfxa7b8uIgts/WxALPYNc9989Wu+ntq7JbNXL9y8sdNf4fyhYCACQCOYAy3v3tuVt3dQ9BB0ptH3/nIIlAFjxFIimSM/O4PU8AHJ4K/nmAcfvkAdC2W0gNP0Rgd9LvIwzoOXqEmJ4Gsoz3KiNnDucUgERkeoA8gPTAEl+uIZXpgARh8t9hhYBEYC5kGu9VRs7M47Yr9Ruh945cqrz/vmfUL/n18iYSirOyUnF6vZa+KH+aGFfalq9UlvgWFPy3+LvzN2812nx0dVRprlqkbKhroLtdxdcOv6WMhmiUnE7+6Bj3Dv2gGv935/QAMikFhQHaMIoP3is6RkX4kLo8SC4vuf4GcSUtp9MOBXX6vIfjoXAnhQHpXVUk8rD2Xyj08VebDh2Y7b3Ha1lh/n+yCKLdt7RzeG2OiJLrbwhX0ub2HOm+Wbe9u1Z9GuN9BJ7mJsVdV2uLi1TKQA9dRHQSFDsQFHmEXL37qTg/eV3q/fXMDrj+kXMXRLx0XXIJcFYBAOqf//ZgIhpt43kErupFitd/m20uVimDPVDo8vjgwdn/QwTy9Sr0DjnaG89+CAfOivAAhlTyt6f19jOz1fmGCNdGQG1z+WJX1XXdP1LcXHwUEqUuD+Yz7BLkZyEIkd92ADcEuf8ZuZxRANy1tQMijkRAdrOsKCWRl255MNvMO7zPvb/8GSXMbOz+C0JGLrsy/WDyzbeDVQ+3dSRisWVcVS4UUiqWNNrqwv3i7CclLQ8ijj99fWZrhqFL59MuD8LiU5WcvRE5c44tmXPGsOr+v1ywBwDEw+G93N2cWMx2XkCpy4MYj528CrCTJfduWXmEGUR++1t/AaO/gKwczrU78IDD5eL/Ya/Yz4UFYZ/Kc686NLwgCXhey8bry4PJjUOI8/Fz5AW6P3yXGGJ3ARDHiayhfFZ2h947Eqps+0KrEo+v5+oFRKNoPVYcHo+tLiIIi2q/bFN12XCQwYPK/sBMvA+yIwzwqkKLZ4QT4XiMPfDzQpf5CNYDxn5PX74ihPyq+/9qKR6A4qioeFXEkU1/NmrLi5ktkQcvIdWaJ/e+s8ahex4gRkgGgVzIyd2cAnB1Rx/CAO5BO4Yd2HVsOAp00i0PHrpwNu3vY5NNHWzLrLtJBKSx/ioHBA3+QO3/QMkCwLwAr7ffYnFP2ZGayNNj/XRI/T6WB5+gGYIU+5eGvDiblwC4Fi3cK+TDI/Npo8KgZCQn8nSA1Kk1/yB/OrKjzBjFQgR7W3+BK2J5cTavFD9qAha03889GQigLsAu/QGpQCLvt5cvKY+tWs2SfHjga7S9Vnu8yudVgv/wvoeUxb7KeX/LtupWH3gNgj0ROXdelAFE8u9H+fyiO99XRDIwEY128D5SlguYnFScVVW2vMgs8ffBu2w3XN3i52oiQmKQBobY3Pqr97zAgbl5e+x5L/JPDb53YsHDD3YqsRh3c439z+3qBQCo8su1PKgD6/4/+MP/EUPsbv3PfyrK+gfSDf4oKQcwS9RwpFfEEUMJ7To9WAfc+WyNQyxnMHhwdkMRgn2Be12g9S+Io45CX71+x7fGEgK8AOweVLl+ne0vPkKBVE8AiULaN0AeTJ04Kcr6Y+mvrpA/cBb6Ds5Kn5AVAZwQuxYHJQP5gGSi4+u/OHSAyC8JcI8LXPkqmJsFC4B3tX+PiMIgAGuidl0WnOPqa9OAEBJsOjRATT6SAPe2wHV/cHJPoX9UcKfP9f/8RajqkYcq1Q/Txv8MJdSTFFHctTW2vhGwtIea/9cDFO/LBIz6ioeEVb++rLr/bwr3AER7ASxBYvOEIEBz+eSC4Pu6KOtftABcePLpoKhcgK6UgnqjCQTjXX/1XhY06HM29k838FOYAIj2Agw4YQSCoa6/QINWtPUvSQBEewGyhAIEcv3LZf0BR6nvXt/77EgiHPGL+GSYRuRbv1YRMZWIQDDC9Q+dOCXS+qPqr6WUF3CWegROn69X5AmkUIBArn9GlMw9B4+jELGJSDI8K5Yp7kbaRJNgHWC9P/LpJZFvkXGzD0M9AMDdUN8r0k3HibTr9CCC/YB7VTD5uVh/bgJwecu2IYfH0y/y04bPnKWlQYIl4n7cq4LRr1r/IdMIAOBbu7pb1LIgO7GR6ZmNEwgEM8f96j0quJwdHOvm9WLcBIAtCy6s6hb5ydFCaYBrRSAUHaoKbPPV0V3Ksl8qHLyPruF73xmMh8JtIs+AnbYZJ9gDArf1TgaXxF8y3LyP0FVT0xUPXT6qfimMoTjRTp8Prcl056UBxk3diknjRSdQMaPBqT5m3UWbjm0r+Xwj6See/LD6Xbxf1CHiSBd/v7dHVcSdIs8GVh28q/3SiQDiy8R0hN10M+SeUu9A9TkSMayVekYYPKoiONXzX6leCye7Do4KD/uZbOQPfxIwIkHdq1r/HksIACC6NoBZJPWmgwjYtVIQlhw3GCP79LQR8SUfLxDbvlVUsOvDHjb1HFjGXyW/AUvU3F1/YSHA7As31HdFr1w9KmJ8WKr62kEEGNFD6gNTkkMhS9c9zArV2FyxZmGbKg52CN8MJL8Q11+4B8BCgV0vPBO9cm236DNkRU8A7npsYkIbi35TuhoHXCtn1QLmLbiqqy0VOhhIfgBZ/z2WFACg4XvbDqgWrUP0+2BVAKsDpraM6AwbV0mvuvZ2H31WTF7BpYYKrppqVRAWmfpYkfATuKNPMrDBx2ahvBH9CXzr1nRNnTjVlohEhK7b6RfETCIASzFLemptzukRRSPB2esIEdDFwEyenYHkF+r6G+YBAEv2vtQWvXJ10Ag3t9yeAJGeP8wiBgaSH2jnVe6b9dwa8Ukm33w7sPArmxxqKNAm+r2QQIM1QbLJ4XQadnMgY4+Rz9MXLiqx4Dg2USHm8hJV9VwyUb02pn4dVhxulxoyeAwVdVxXA8mPJb9+Q0IvIy9k40s7DsSu3+gw4r2MSAwya6/eFNMSjDM3Y86gorFBcaken+hrbGDCz5C4fw5PjLxo3pZVXap6DxtikQUWaMDawx2cOn6C1X8T+cuTM8C5Z9cA47YnJ+1A/mEj4v6yeQDAsh/tao1c/GxQdFIw2RPwrGzisu4MFxAW3yoFOdLlCrCkqHoEPPpEWHnv+QtGkj+oxf3DthYAYPG/fK8jNj5xwKi171LLhkH86c8uk6W3UniwdEnRQmBgeW8yNqvkHzBcNMtxgW6+NXhiYceXxuM3px4xxl9MKNFrYyxxlK8I4OJHR68yKwCrr8TixCyrQL1WSBpGtevG+hTyTAjjb8IjZ9g9YyC6jUr6mcIDSPIEXlGJ2Wnke2K2IGYMZr0Jrlyd2cTRYtV5n1vepNyxfIVyW139vJ/97uIF5djFT5WzY9fk8whUD7Bi6eKccyWRUxC4d18mYLpPV9nOTbkvjlGVgqmxomdV87zssRVdfZD+m59/QHn09juUGl9lzt+HAPzPyGnlpx/9r/L+yJ/m/fyLLX+m/OLpv+dybPXf3WqJ0IBNnz5zrhy5HUMz/mkNYrkvCioFQ6dH/PHJm62GeYjqhQ6dOq14V93G3ENkkMMY4Wwh4sPK9z3awYhf6N/h8dd3/TkTgn/4+U+l8QrYyoF6nSHy3uYm1qXI4n3MmzT+2hue8TelAGCUWNNr+9pVERg0UgRwwSECTAAs1nkHK/2Tv+vKy+JnA8IFGUMCdu1PB8p57UH+dp6jvYqF0wwXBCKgkr9dOzGGwmrkh+WGi14q+YGDvz+myAzZyW8KD0AHTsiyH+3qMrJGwGpAvN/35czpkvHQFCO1nvDT8YXVa5TPLWtiz8nC8e+/eZdOqrFgDT5mIb+pBAC49I2tw6oItJMIpMe//eWTGS0/yPzy24eYCKRCT/bhb5EzePahTez/EIp88dV9P0ybNCQURH7DC30sJQAkAtldf3gA6YBEHrL6uQBxwO/BS8j0WgR5yG+aHEA6EfAsX9ru8HiCdO/MQLfaqdj+q4G8yJ8qBGTNifymFYBkEVDKkBg0Y+yfrrgHy3gUx5saw2Ymv6kFQBcBnEBn1QKpRSDTWv9/fPQhUYzIb18BAJAx9a1pkVoEsO6fDoW6/gTDyW/6ENZthbM5Wyx08vQrRpcNmwHNtXVp3X8jgaKhfHA2eE3K4qIkoKOvywrkt4wA6CKgPm0uRwNRuZEu/h+fmjL0GLLVHyTj5XcOseVISVHWxh5bhgCpGP3W813uxvpuu+4GlC9+d+mCQjAVuq1GfksKABOBrTv2uGqqN8u8TMijFJjABcwzFbl5BwlAek9ggNUKGDRj0GygQh5TQE/2DVj1A7itfPaxTIjkYHjkzCtGTRsui7t/8cI8wqdLDIo+holQ7uYZiRKAlkr22VIAgNnk4Pd7e2IT13facY+9c2Nj8wQAiUF8r5B6/lLw3V+9QdWDGhyeit5rPf/cY4fP4rTLRRn99s4ed2ODLcuHMxEP/QEEw+P9druQ31YCAFzesm2ocv3aFqfPN2Cnz3XwD8cyCkC6JUKCMJe/xYjtukgASgwJrj7/0mZtqdAW3oA+xy8VWAn4yd920YqAeKuPJb7NVo/3pRCA2ZBg6449akiw0eF220KxMxXXIA9Q6IQgeA4jO/6JhCM3cO9stOoSXz6wdTXN5JtvB6cO/+bVRV/9EjYmxbxBn1U/C0psQfa1i5fM+9nSRdVK1z2fV3wVFcq54FjaoSD6INB9f/U15W/uulvxuSuUd079kb1uut9LBfoOUn/X5lb/ZRT2hN47YutaE4csV3TJ3pf8sfHxV+KhcJtVPwMsNqx9rhoAhAwQAh1YMkyXK0hXtptpLLhEE4Fg9bG8F5Dhw7plEYDLW7bhgrY3vryzM35jcnciFrPctCFY9n/8+Wss7s+W/NNHf+cC5gQS5sX6/TJ9aKdsV/nKs739lRvWtzgrK/ut2E+Adf8H/+1fuXQD5tvhJwFA+hbZyC+lAABspWBHXxerG7BgkhCewFf2/aCkTT0gIPAmJAeuPUp5u+yY4accQJ5gYUEotDMRjvitePyI2zE1CLmBL7SsyZgXwKhwxPGoK8gkHFmTgPYp80U42CujxScByCYEfc/1xKdCW6yYH5gX36tigKQhle/Oi/P3qsTvoVNBApAWTa/tqw1/EnjGLkJAuEV89bFHVlefBICEgIhPIAEgISDiE0gAikLd9u5Oh9dj2WShBAgolNwjATBACNocbveWRDTaQWfDFECn3l67deqRAJgcrLz4+o0tiXC4k8KDsrj5/RrxA3Q6SADKioYXtnckpqe/riQSHXacSmQya/+qlefwkQDYOzyAJ9Dh9Hq3xMPhVjojXIDhm0jqDVBSjwTAUiFCNBjsUGLxxxLRaBudkYKAeP4NjfTk4pMAWN8zcPq8HfFQ+EF4COqDcgbzY3q49YfJ0pMAyCAIrQ5PRUciMg1BkNU7GEoi/DDdFSQA0qKx77m22I1J5AwgCHj22+wjBrRYHoQfpiU7EgBCjpBBEwJ4B3dqgmCVpOKwRviPNSs/TC49CQCBU+ig5Q8gDDVJomB0GDGURPZx7f9BcuVJAAjlFQh/UvhQy8FrAKF16x2gjLy98P8CDADMneNqQ66ddAAAAABJRU5ErkJggg==" - longDescription: "Creating a service instance provisions a database. Binding applications provisions unique credentials for each application to access the database." - providerDisplayName: "Pivotal Software" - documentationUrl: "https://github.com/cloudfoundry/cf-mysql-release/blob/master/README.md" - supportUrl: "https://support.pivotal.io" - dashboard_client: - id: p-mysql - secret: (( property_overrides.broker.dashboard_secret )) - plans: (( property_overrides.broker.plans )) - acceptance_tests: - smoke_tests_only: (( property_overrides.acceptance_tests.smoke_tests_only || nil )) - standalone_tests_only: (( property_overrides.acceptance_tests.standalone_tests_only || nil )) - timeout_scale: (( property_overrides.acceptance_tests.timeout_scale || nil )) - # The keys below should not be included in the final stub config_from_cf: (( merge )) iaas_settings: (( merge )) From 9377b768a3f5d3a66aaf5d9128fed1cd62b78949 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Mon, 25 Jan 2016 17:16:13 -0800 Subject: [PATCH 064/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Morgan Fine: Only Seed Databases on BootStrap node ( Travis Unknown ) Add test for bad state in StartNodeFromState ( Travis Unknown ) Test interface to Starter in StartManager tests ( Travis Unknown ) bin/test-unit generates fakes ( Travis Unknown ) Backfill tests from StartManager to Starter ( Travis Unknown ) Aditya Anchuri: Extract node startering logic into node_starter ( Travis Unknown ) Added functionality to wait on joining mysql cluster ( Travis Unknown ) Added monitorJoin field to startManager to be used for prestart ( Travis Unknown ) Ben Calegari: Reformat fakes ( Travis Unknown ) Add .test files to .gitignore ( Travis Unknown ) Add go fmt to bin/test ( Travis Unknown ) Restructure mariadb_helper_test and regenerate fakes [#110369648] ( Travis Unknown ) test: Revert "Added functionality to wait on joining mysql cluster" ( Travis Unknown ) Revert "Created monitored call to mysqld_safe" ( Travis Unknown ) Revert "Added monitorJoin field to startManager to be used for prestart" ( Travis Unknown ) Aaron Hurley: Create read only user from start_manager ( Travis Unknown ) mariadb_helper can update Read Only User password ( Travis Unknown ) mariadb_helper Seed function creates Read Only user ( Travis Unknown ) add function to create read only user ( Travis Unknown ) Signed-off-by: Morgan Fine --- packages/mariadb_ctrl/spec | 2 ++ src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/packages/mariadb_ctrl/spec b/packages/mariadb_ctrl/spec index f62f90e3..f38b1b9f 100644 --- a/packages/mariadb_ctrl/spec +++ b/packages/mariadb_ctrl/spec @@ -18,6 +18,8 @@ files: - github.com/cloudfoundry/mariadb_ctrl/os_helper/fakes/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/start_manager/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/start_manager/fakes/*.go # gosub + - github.com/cloudfoundry/mariadb_ctrl/start_manager/node_starter/*.go # gosub + - github.com/cloudfoundry/mariadb_ctrl/start_manager/node_starter/fakes/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/fakes/*.go # gosub - github.com/fraenkel/candiedyaml/*.go # gosub diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 2952dc7d..d685ccaa 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 2952dc7d530ae8e69b16499391c4fa66cfcf5231 +Subproject commit d685ccaa04089008606f80fab1f3c25a8c531a5f From 5984a96c2b2d057aac9c1847d730ba67b137467d Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Mon, 25 Jan 2016 17:17:11 -0800 Subject: [PATCH 065/110] Bump src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer Bump : Signed-off-by: Morgan Fine --- src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer index 87302d35..df200556 160000 --- a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer +++ b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer @@ -1 +1 @@ -Subproject commit 87302d350992ec23a19083f8a2aa3b7052fa2519 +Subproject commit df20055636916a26a1f50328063014fcffb56111 From 53abe8508c50e2eda9d35b19b70135b1841eef3a Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 26 Jan 2016 16:19:54 -0800 Subject: [PATCH 066/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Added GetMysqlCmd function to node starter ( Travis Unknown ) --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index d685ccaa..c47580af 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit d685ccaa04089008606f80fab1f3c25a8c531a5f +Subproject commit c47580af8204843b4611f510fc43442b98ed0e2e From 23f9e794d66d83925c84c77a9ef61f1524822d2e Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Fri, 29 Jan 2016 15:02:22 -0800 Subject: [PATCH 067/110] Update bootstrapping.md --- docs/bootstrapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/bootstrapping.md b/docs/bootstrapping.md index 866c488f..28f41038 100644 --- a/docs/bootstrapping.md +++ b/docs/bootstrapping.md @@ -48,7 +48,7 @@ If one or more nodes are not reachable (i.e. the VM exists but in an unknown sta 1. `bosh -n stop mysql_z1 && bosh -n stop mysql_z2 && bosh -n stop _z3` 1. `bosh edit deployment` 1. Set `update.canaries` to 0, `update.max_in_flight` to 3, and `update.serial` to false. -1. `bosh deploy` +1. `bosh deploy`. Note, if you get a 503 error (like `Sending stop request to monit: Request failed, response: Response{ StatusCode: 503, Status: '503 Service Unavailable' }`), it means that monit is still trying to stop the vms. Please wait a few minutes and try this step again. 1. `bosh -n start mysql_z1 ; bosh -n start mysql_z2 ; bosh -n start _z3` (This will throw several errors, but it ensures that all the jobs are present on the VM) 1. `bosh instances` to verify that all jobs report as failing. 1. Try running the errand again using `bosh -n run errand bootstrap` as above. From ffab547d299daa94f5469a502f39305ba3de3bbf Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Fri, 29 Jan 2016 17:51:51 -0800 Subject: [PATCH 068/110] Update arbitrator.md --- docs/arbitrator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/arbitrator.md b/docs/arbitrator.md index f23a91a9..c8f54c0d 100644 --- a/docs/arbitrator.md +++ b/docs/arbitrator.md @@ -10,7 +10,7 @@ For a fresh deployment of CF MySQL v26, simply follow the steps in the [README]( ### Upgrading to 2-node-plus-arbitrator from already existing 3-node deployment -If you already have a 3-node deployment of CF MySQL (e.g. v25), follow the below steps to upgrade to the new configuration as a rolling deploy: +If you already have a 3-node deployment of CF MySQL (e.g. v25), follow the below steps to upgrade to the new configuration as a rolling deploy (the commands below apply to testing on a bosh-lite deployment. For other environments, the same steps are to be followed, except that stubs are different, as explained in the README linked above.). 1. Generate a 3 node + arbitrator manifest: `./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > .yml` 1. `bosh deployment .yml` From 6e05b2badd8f38de6f97a5f26ddcbd47dcf22ffb Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 1 Feb 2016 09:31:57 -0800 Subject: [PATCH 069/110] Added roadmin_password property to cf-mysql-template Signed-off-by: Morgan Fine --- manifest-generation/cf-mysql-template.yml | 1 + packages/galera-healthcheck/spec | 12 ------------ src/golang.org/x/net | 1 - 3 files changed, 1 insertion(+), 13 deletions(-) delete mode 160000 src/golang.org/x/net diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 07e25efa..a28747de 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -60,6 +60,7 @@ properties: mysql: admin_username: (( property_overrides.mysql.admin_username || nil )) admin_password: (( property_overrides.mysql.admin_password )) + roadmin_password: (( property_overrides.mysql.roadmin_password )) port: (( property_overrides.mysql.port || nil )) galera_port: (( property_overrides.mysql.galera_port || nil )) healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) diff --git a/packages/galera-healthcheck/spec b/packages/galera-healthcheck/spec index d9e3595e..be3ba11c 100644 --- a/packages/galera-healthcheck/spec +++ b/packages/galera-healthcheck/spec @@ -25,17 +25,5 @@ files: - github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/pivotal-golang/lager/*.go # gosub - github.com/tedsuo/rata/*.go # gosub - - golang.org/x/net/html/*.go # gosub - - golang.org/x/net/html/atom/*.go # gosub - golang.org/x/net/html/charset/*.go # gosub - - golang.org/x/text/encoding/*.go # gosub - - golang.org/x/text/encoding/charmap/*.go # gosub - - golang.org/x/text/encoding/internal/*.go # gosub - - golang.org/x/text/encoding/internal/identifier/*.go # gosub - - golang.org/x/text/encoding/japanese/*.go # gosub - - golang.org/x/text/encoding/korean/*.go # gosub - - golang.org/x/text/encoding/simplifiedchinese/*.go # gosub - - golang.org/x/text/encoding/traditionalchinese/*.go # gosub - - golang.org/x/text/encoding/unicode/*.go # gosub - - golang.org/x/text/transform/*.go # gosub - gopkg.in/validator.v2/*.go # gosub diff --git a/src/golang.org/x/net b/src/golang.org/x/net deleted file mode 160000 index c764672d..00000000 --- a/src/golang.org/x/net +++ /dev/null @@ -1 +0,0 @@ -Subproject commit c764672d0ee39ffd83cfcb375804d3181302b62b From b750e3d40c6e1bb0f487cc32c78cf09bfda09df8 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Mon, 1 Feb 2016 10:00:30 -0800 Subject: [PATCH 070/110] Deleted text package Signed-off-by: Aditya Anchuri --- src/golang.org/x/text | 1 - 1 file changed, 1 deletion(-) delete mode 160000 src/golang.org/x/text diff --git a/src/golang.org/x/text b/src/golang.org/x/text deleted file mode 160000 index 44d57623..00000000 --- a/src/golang.org/x/text +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 44d57623517a1936066b18e8a63e1e03e1bf8faa From 50b3c68dec46058a879c128655e28c6edc1a9b82 Mon Sep 17 00:00:00 2001 From: test Date: Mon, 1 Feb 2016 11:34:01 -0800 Subject: [PATCH 071/110] Revert "Deleted text package" This reverts commit b750e3d40c6e1bb0f487cc32c78cf09bfda09df8. --- src/golang.org/x/text | 1 + 1 file changed, 1 insertion(+) create mode 160000 src/golang.org/x/text diff --git a/src/golang.org/x/text b/src/golang.org/x/text new file mode 160000 index 00000000..44d57623 --- /dev/null +++ b/src/golang.org/x/text @@ -0,0 +1 @@ +Subproject commit 44d57623517a1936066b18e8a63e1e03e1bf8faa From 76f246b9e4f6dbe00202747cbcc1dc143251f33c Mon Sep 17 00:00:00 2001 From: test Date: Mon, 1 Feb 2016 11:58:41 -0800 Subject: [PATCH 072/110] Revert "Added roadmin_password property to cf-mysql-template" This reverts commit 6e05b2badd8f38de6f97a5f26ddcbd47dcf22ffb. --- manifest-generation/cf-mysql-template.yml | 1 - packages/galera-healthcheck/spec | 12 ++++++++++++ src/golang.org/x/net | 1 + 3 files changed, 13 insertions(+), 1 deletion(-) create mode 160000 src/golang.org/x/net diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index a28747de..07e25efa 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -60,7 +60,6 @@ properties: mysql: admin_username: (( property_overrides.mysql.admin_username || nil )) admin_password: (( property_overrides.mysql.admin_password )) - roadmin_password: (( property_overrides.mysql.roadmin_password )) port: (( property_overrides.mysql.port || nil )) galera_port: (( property_overrides.mysql.galera_port || nil )) healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) diff --git a/packages/galera-healthcheck/spec b/packages/galera-healthcheck/spec index be3ba11c..d9e3595e 100644 --- a/packages/galera-healthcheck/spec +++ b/packages/galera-healthcheck/spec @@ -25,5 +25,17 @@ files: - github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/pivotal-golang/lager/*.go # gosub - github.com/tedsuo/rata/*.go # gosub + - golang.org/x/net/html/*.go # gosub + - golang.org/x/net/html/atom/*.go # gosub - golang.org/x/net/html/charset/*.go # gosub + - golang.org/x/text/encoding/*.go # gosub + - golang.org/x/text/encoding/charmap/*.go # gosub + - golang.org/x/text/encoding/internal/*.go # gosub + - golang.org/x/text/encoding/internal/identifier/*.go # gosub + - golang.org/x/text/encoding/japanese/*.go # gosub + - golang.org/x/text/encoding/korean/*.go # gosub + - golang.org/x/text/encoding/simplifiedchinese/*.go # gosub + - golang.org/x/text/encoding/traditionalchinese/*.go # gosub + - golang.org/x/text/encoding/unicode/*.go # gosub + - golang.org/x/text/transform/*.go # gosub - gopkg.in/validator.v2/*.go # gosub diff --git a/src/golang.org/x/net b/src/golang.org/x/net new file mode 160000 index 00000000..c764672d --- /dev/null +++ b/src/golang.org/x/net @@ -0,0 +1 @@ +Subproject commit c764672d0ee39ffd83cfcb375804d3181302b62b From 0ae014ae7e6c12f4c1d8e09c480f6dea40f8d18c Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 1 Feb 2016 15:47:34 -0800 Subject: [PATCH 073/110] Bump route-registrar to origin/master - The route-registrar was dropping a lot of routes. Bumping to this version should remove the problem. There was a library, Gibson, that the registrar no longer uses that was the root cause. [#112556657] Signed-off-by: Morgan Fine --- .../templates/registrar_settings.yml.erb | 17 ++++++++++------- jobs/proxy/templates/route-registrar.yml.erb | 10 +++++++--- packages/acceptance-tests/spec | 2 -- packages/bootstrap/spec | 2 +- packages/galera-healthcheck/spec | 2 +- packages/mariadb_ctrl/spec | 2 +- packages/quota-enforcer/spec | 2 +- packages/route-registrar/spec | 10 ++++++---- packages/switchboard/spec | 2 +- src/github.com/apcera/gnatsd | 1 + src/github.com/cloudfoundry-incubator/cf-lager | 2 +- .../cloudfoundry-incubator/route-registrar | 2 +- src/github.com/nats-io/gnatsd | 1 + src/github.com/nats-io/nats | 1 + src/github.com/onsi/ginkgo | 2 +- src/github.com/onsi/gomega | 2 +- .../pivotal-cf-experimental/service-config | 2 +- src/github.com/pivotal-golang/lager | 2 +- src/github.com/tedsuo/ifrit | 2 +- src/gopkg.in/yaml.v2 | 1 + 20 files changed, 39 insertions(+), 28 deletions(-) create mode 160000 src/github.com/apcera/gnatsd create mode 160000 src/github.com/nats-io/gnatsd create mode 160000 src/github.com/nats-io/nats create mode 160000 src/gopkg.in/yaml.v2 diff --git a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb index 9ef187d8..cefc4adc 100644 --- a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb +++ b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb @@ -4,10 +4,13 @@ message_bus_servers: user: <%= p('nats.user') %> password: <%= p('nats.password') %> <% end %> -external_host: <%= p('cf_mysql.external_host') %> -external_ip: <%= spec.networks.send(p('network_name')).ip %> -port: 80 -health_checker: - name: script - interval_in_seconds: 10 - healthcheck_script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh +host: <%= spec.networks.send(p('network_name')).ip %> +routes: +- name: "broker_"<%= index %> + port: 80 + uris: + - <%= p('cf_mysql.external_host') %> +registration_interval: 10 +health_check: + name: script + script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh diff --git a/jobs/proxy/templates/route-registrar.yml.erb b/jobs/proxy/templates/route-registrar.yml.erb index 4a30bf36..5df85ab0 100644 --- a/jobs/proxy/templates/route-registrar.yml.erb +++ b/jobs/proxy/templates/route-registrar.yml.erb @@ -6,6 +6,10 @@ message_bus_servers: <% end %> <% my_ip = spec.networks.send(p('network_name')).ip %> <% proxy_index = p('cf_mysql.proxy.proxy_ips').index(my_ip) %> -external_host: proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> -external_ip: <%= my_ip %> -port: <%= p('cf_mysql.proxy.api_port') %> +host: <%= my_ip %> +routes: +- name: "proxy_"<%= index %> + port: <%= p('cf_mysql.proxy.api_port') %> + uris: + - proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> + registration_interval: 10 diff --git a/packages/acceptance-tests/spec b/packages/acceptance-tests/spec index 2d9b6399..658b0655 100644 --- a/packages/acceptance-tests/spec +++ b/packages/acceptance-tests/spec @@ -25,7 +25,6 @@ files: - github.com/onsi/ginkgo/config/*.go # gosub - github.com/onsi/ginkgo/ginkgo/*.go # gosub - github.com/onsi/ginkgo/ginkgo/convert/*.go # gosub - - github.com/onsi/ginkgo/ginkgo/interrupthandler/*.go # gosub - github.com/onsi/ginkgo/ginkgo/nodot/*.go # gosub - github.com/onsi/ginkgo/ginkgo/testrunner/*.go # gosub - github.com/onsi/ginkgo/ginkgo/testsuite/*.go # gosub @@ -49,7 +48,6 @@ files: - github.com/onsi/gomega/gexec/*.go # gosub - github.com/onsi/gomega/internal/assertion/*.go # gosub - github.com/onsi/gomega/internal/asyncassertion/*.go # gosub - - github.com/onsi/gomega/internal/oraclematcher/*.go # gosub - github.com/onsi/gomega/internal/testingtsupport/*.go # gosub - github.com/onsi/gomega/matchers/*.go # gosub - github.com/onsi/gomega/matchers/support/goraph/bipartitegraph/*.go # gosub diff --git a/packages/bootstrap/spec b/packages/bootstrap/spec index 816d1f72..018b7ddb 100644 --- a/packages/bootstrap/spec +++ b/packages/bootstrap/spec @@ -13,8 +13,8 @@ files: - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/config/*.go # gosub - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/fakes/*.go # gosub - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/test_helpers/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/pivotal-golang/lager/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/galera-healthcheck/spec b/packages/galera-healthcheck/spec index d9e3595e..d9a56e8c 100644 --- a/packages/galera-healthcheck/spec +++ b/packages/galera-healthcheck/spec @@ -19,7 +19,6 @@ files: - github.com/cloudfoundry-incubator/galera-healthcheck/mysqld_cmd/fakes/*.go # gosub - github.com/cloudfoundry-incubator/galera-healthcheck/sequence_number/*.go # gosub - github.com/cloudfoundry-incubator/galera-healthcheck/sequence_number/fakes/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub @@ -39,3 +38,4 @@ files: - golang.org/x/text/encoding/unicode/*.go # gosub - golang.org/x/text/transform/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/mariadb_ctrl/spec b/packages/mariadb_ctrl/spec index f38b1b9f..11d01dcc 100644 --- a/packages/mariadb_ctrl/spec +++ b/packages/mariadb_ctrl/spec @@ -22,7 +22,6 @@ files: - github.com/cloudfoundry/mariadb_ctrl/start_manager/node_starter/fakes/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/fakes/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub @@ -30,3 +29,4 @@ files: - github.com/tedsuo/ifrit/*.go # gosub - github.com/tedsuo/ifrit/sigmon/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/quota-enforcer/spec b/packages/quota-enforcer/spec index 797524ff..59d084c4 100644 --- a/packages/quota-enforcer/spec +++ b/packages/quota-enforcer/spec @@ -4,7 +4,6 @@ dependencies: - golang files: - github.com/cloudfoundry-incubator/cf-lager/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer/*.go # gosub @@ -17,3 +16,4 @@ files: - github.com/pivotal-golang/lager/*.go # gosub - github.com/tedsuo/ifrit/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/route-registrar/spec b/packages/route-registrar/spec index c589eccd..9e0a5505 100644 --- a/packages/route-registrar/spec +++ b/packages/route-registrar/spec @@ -8,12 +8,14 @@ files: - github.com/cloudfoundry-incubator/route-registrar/config/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/healthchecker/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/healthchecker/fakes/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/messagebus/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/messagebus/fakes/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/registrar/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/test_helpers/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/cloudfoundry/gibson/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/cloudfoundry/yagnats/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/fraenkel/candiedyaml/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/imdario/mergo/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nats-io/nats/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nats-io/nats/encoders/builtin/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nu7hatch/gouuid/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/pivotal-golang/lager/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/tedsuo/ifrit/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/switchboard/spec b/packages/switchboard/spec index 1366325f..c26aad86 100644 --- a/packages/switchboard/spec +++ b/packages/switchboard/spec @@ -16,7 +16,6 @@ files: - github.com/cloudfoundry-incubator/switchboard/dummies/*.go # gosub - github.com/cloudfoundry-incubator/switchboard/health/*.go # gosub - github.com/cloudfoundry-incubator/switchboard/proxy/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/onsi/ginkgo/*.go # gosub - github.com/onsi/ginkgo/config/*.go # gosub @@ -38,3 +37,4 @@ files: - github.com/tedsuo/ifrit/*.go # gosub - github.com/tedsuo/ifrit/grouper/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/src/github.com/apcera/gnatsd b/src/github.com/apcera/gnatsd new file mode 160000 index 00000000..972b9c18 --- /dev/null +++ b/src/github.com/apcera/gnatsd @@ -0,0 +1 @@ +Subproject commit 972b9c18ac61bed63d8ea6fbba3a9eb03b18d1b8 diff --git a/src/github.com/cloudfoundry-incubator/cf-lager b/src/github.com/cloudfoundry-incubator/cf-lager index 33247902..71788331 160000 --- a/src/github.com/cloudfoundry-incubator/cf-lager +++ b/src/github.com/cloudfoundry-incubator/cf-lager @@ -1 +1 @@ -Subproject commit 3324790250247b4d3bd3abc3e74654a537d79a30 +Subproject commit 7178833173e7124dab16158273f7708ed73b81ff diff --git a/src/github.com/cloudfoundry-incubator/route-registrar b/src/github.com/cloudfoundry-incubator/route-registrar index e8834cf6..7bf60c1d 160000 --- a/src/github.com/cloudfoundry-incubator/route-registrar +++ b/src/github.com/cloudfoundry-incubator/route-registrar @@ -1 +1 @@ -Subproject commit e8834cf6fd22131d5e943cbf129c9fcd9088e8ac +Subproject commit 7bf60c1db84826d74d43cae1a61b7568b3959326 diff --git a/src/github.com/nats-io/gnatsd b/src/github.com/nats-io/gnatsd new file mode 160000 index 00000000..972b9c18 --- /dev/null +++ b/src/github.com/nats-io/gnatsd @@ -0,0 +1 @@ +Subproject commit 972b9c18ac61bed63d8ea6fbba3a9eb03b18d1b8 diff --git a/src/github.com/nats-io/nats b/src/github.com/nats-io/nats new file mode 160000 index 00000000..355b5b97 --- /dev/null +++ b/src/github.com/nats-io/nats @@ -0,0 +1 @@ +Subproject commit 355b5b97e0842dc94f1106729aa88e33e06317ca diff --git a/src/github.com/onsi/ginkgo b/src/github.com/onsi/ginkgo index 17ea4797..7d3d52b3 160000 --- a/src/github.com/onsi/ginkgo +++ b/src/github.com/onsi/ginkgo @@ -1 +1 @@ -Subproject commit 17ea479729ee427265ac1e913443018350946ddf +Subproject commit 7d3d52b3a3a79d745c2a03ee81b5bc2086642201 diff --git a/src/github.com/onsi/gomega b/src/github.com/onsi/gomega index 2152b45f..85936b29 160000 --- a/src/github.com/onsi/gomega +++ b/src/github.com/onsi/gomega @@ -1 +1 @@ -Subproject commit 2152b45fa28a361beba9aab0885972323a444e28 +Subproject commit 85936b29809b7df066a8d839dd105e158e510264 diff --git a/src/github.com/pivotal-cf-experimental/service-config b/src/github.com/pivotal-cf-experimental/service-config index d1936ca6..b1dc94de 160000 --- a/src/github.com/pivotal-cf-experimental/service-config +++ b/src/github.com/pivotal-cf-experimental/service-config @@ -1 +1 @@ -Subproject commit d1936ca6555039c884eb26fabd3396a28e9c488d +Subproject commit b1dc94de6ada8bfddf159ce2234958a8f02aec77 diff --git a/src/github.com/pivotal-golang/lager b/src/github.com/pivotal-golang/lager index 389345f0..09a71f6e 160000 --- a/src/github.com/pivotal-golang/lager +++ b/src/github.com/pivotal-golang/lager @@ -1 +1 @@ -Subproject commit 389345f09521afe9c164379d98cb70335aff8726 +Subproject commit 09a71f6eb2dcc1c2335e37ceb78a91c6f618d561 diff --git a/src/github.com/tedsuo/ifrit b/src/github.com/tedsuo/ifrit index 61109605..3a41de6e 160000 --- a/src/github.com/tedsuo/ifrit +++ b/src/github.com/tedsuo/ifrit @@ -1 +1 @@ -Subproject commit 611096055721e35a95e3d1e4e2bc666fae509fcb +Subproject commit 3a41de6efdb78719e66467a1b170f2b7f1e1014d diff --git a/src/gopkg.in/yaml.v2 b/src/gopkg.in/yaml.v2 new file mode 160000 index 00000000..49c95bdc --- /dev/null +++ b/src/gopkg.in/yaml.v2 @@ -0,0 +1 @@ +Subproject commit 49c95bdc21843256fb6c4e0d370a05f24a0bf213 From db35e1211a1ff2ac4d6c5768e21660607550714f Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 1 Feb 2016 16:04:22 -0800 Subject: [PATCH 074/110] Add missing .gitmodule references to new submodules [#112556657] Signed-off-by: Morgan Fine --- .gitmodules | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/.gitmodules b/.gitmodules index 626dbd17..7f2b4f34 100644 --- a/.gitmodules +++ b/.gitmodules @@ -114,3 +114,19 @@ path = src/github.com/bmizerany/pat url = https://github.com/bmizerany/pat branch = master +[submodule "src/github.com/apcera/gnatsd"] + path = src/github.com/apcera/gnatsd + url = https://github/apcera/gnatsd + branch = master +[submodule "src/github.com/nats-io/nats"] + path = src/github.com/nats-io/nats + url = https://github.com/nats-io/nats + branch = master +[submodule "src/github.com/nats-io/gnatsd"] + path = src/github.com/nats-io/gnatsd + url = https://github.com/nats-io/gnatsd + branch = master +[submodule "src/gopkg.in/yaml.v2"] + path = src/gopkg.in/yaml.v2 + url = https://gopkg.in/yaml.v2 + branch = master From 2510132f030623217e3e4865a000359b9bd6a205 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Mon, 1 Feb 2016 16:08:02 -0800 Subject: [PATCH 075/110] Oopsy in .gitmodules Signed-off-by: Morgan Fine --- .gitmodules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitmodules b/.gitmodules index 7f2b4f34..dd6cb0f5 100644 --- a/.gitmodules +++ b/.gitmodules @@ -116,7 +116,7 @@ branch = master [submodule "src/github.com/apcera/gnatsd"] path = src/github.com/apcera/gnatsd - url = https://github/apcera/gnatsd + url = https://github.com/apcera/gnatsd branch = master [submodule "src/github.com/nats-io/nats"] path = src/github.com/nats-io/nats From 82a78ee5ed6c15e3eccd18c5478c49e529db33d0 Mon Sep 17 00:00:00 2001 From: test Date: Mon, 1 Feb 2016 16:46:48 -0800 Subject: [PATCH 076/110] Revert "Bump route-registrar to origin/master" This reverts commit 0ae014ae7e6c12f4c1d8e09c480f6dea40f8d18c. --- .../templates/registrar_settings.yml.erb | 17 +++++++---------- jobs/proxy/templates/route-registrar.yml.erb | 10 +++------- packages/acceptance-tests/spec | 2 ++ packages/bootstrap/spec | 2 +- packages/galera-healthcheck/spec | 2 +- packages/mariadb_ctrl/spec | 2 +- packages/quota-enforcer/spec | 2 +- packages/route-registrar/spec | 10 ++++------ packages/switchboard/spec | 2 +- src/github.com/apcera/gnatsd | 1 - src/github.com/cloudfoundry-incubator/cf-lager | 2 +- .../cloudfoundry-incubator/route-registrar | 2 +- src/github.com/nats-io/gnatsd | 1 - src/github.com/nats-io/nats | 1 - src/github.com/onsi/ginkgo | 2 +- src/github.com/onsi/gomega | 2 +- .../pivotal-cf-experimental/service-config | 2 +- src/github.com/pivotal-golang/lager | 2 +- src/github.com/tedsuo/ifrit | 2 +- src/gopkg.in/yaml.v2 | 1 - 20 files changed, 28 insertions(+), 39 deletions(-) delete mode 160000 src/github.com/apcera/gnatsd delete mode 160000 src/github.com/nats-io/gnatsd delete mode 160000 src/github.com/nats-io/nats delete mode 160000 src/gopkg.in/yaml.v2 diff --git a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb index cefc4adc..9ef187d8 100644 --- a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb +++ b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb @@ -4,13 +4,10 @@ message_bus_servers: user: <%= p('nats.user') %> password: <%= p('nats.password') %> <% end %> -host: <%= spec.networks.send(p('network_name')).ip %> -routes: -- name: "broker_"<%= index %> - port: 80 - uris: - - <%= p('cf_mysql.external_host') %> -registration_interval: 10 -health_check: - name: script - script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh +external_host: <%= p('cf_mysql.external_host') %> +external_ip: <%= spec.networks.send(p('network_name')).ip %> +port: 80 +health_checker: + name: script + interval_in_seconds: 10 + healthcheck_script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh diff --git a/jobs/proxy/templates/route-registrar.yml.erb b/jobs/proxy/templates/route-registrar.yml.erb index 5df85ab0..4a30bf36 100644 --- a/jobs/proxy/templates/route-registrar.yml.erb +++ b/jobs/proxy/templates/route-registrar.yml.erb @@ -6,10 +6,6 @@ message_bus_servers: <% end %> <% my_ip = spec.networks.send(p('network_name')).ip %> <% proxy_index = p('cf_mysql.proxy.proxy_ips').index(my_ip) %> -host: <%= my_ip %> -routes: -- name: "proxy_"<%= index %> - port: <%= p('cf_mysql.proxy.api_port') %> - uris: - - proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> - registration_interval: 10 +external_host: proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> +external_ip: <%= my_ip %> +port: <%= p('cf_mysql.proxy.api_port') %> diff --git a/packages/acceptance-tests/spec b/packages/acceptance-tests/spec index 658b0655..2d9b6399 100644 --- a/packages/acceptance-tests/spec +++ b/packages/acceptance-tests/spec @@ -25,6 +25,7 @@ files: - github.com/onsi/ginkgo/config/*.go # gosub - github.com/onsi/ginkgo/ginkgo/*.go # gosub - github.com/onsi/ginkgo/ginkgo/convert/*.go # gosub + - github.com/onsi/ginkgo/ginkgo/interrupthandler/*.go # gosub - github.com/onsi/ginkgo/ginkgo/nodot/*.go # gosub - github.com/onsi/ginkgo/ginkgo/testrunner/*.go # gosub - github.com/onsi/ginkgo/ginkgo/testsuite/*.go # gosub @@ -48,6 +49,7 @@ files: - github.com/onsi/gomega/gexec/*.go # gosub - github.com/onsi/gomega/internal/assertion/*.go # gosub - github.com/onsi/gomega/internal/asyncassertion/*.go # gosub + - github.com/onsi/gomega/internal/oraclematcher/*.go # gosub - github.com/onsi/gomega/internal/testingtsupport/*.go # gosub - github.com/onsi/gomega/matchers/*.go # gosub - github.com/onsi/gomega/matchers/support/goraph/bipartitegraph/*.go # gosub diff --git a/packages/bootstrap/spec b/packages/bootstrap/spec index 018b7ddb..816d1f72 100644 --- a/packages/bootstrap/spec +++ b/packages/bootstrap/spec @@ -13,8 +13,8 @@ files: - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/config/*.go # gosub - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/fakes/*.go # gosub - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/test_helpers/*.go # gosub + - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/pivotal-golang/lager/*.go # gosub - gopkg.in/validator.v2/*.go # gosub - - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/galera-healthcheck/spec b/packages/galera-healthcheck/spec index d9a56e8c..d9e3595e 100644 --- a/packages/galera-healthcheck/spec +++ b/packages/galera-healthcheck/spec @@ -19,6 +19,7 @@ files: - github.com/cloudfoundry-incubator/galera-healthcheck/mysqld_cmd/fakes/*.go # gosub - github.com/cloudfoundry-incubator/galera-healthcheck/sequence_number/*.go # gosub - github.com/cloudfoundry-incubator/galera-healthcheck/sequence_number/fakes/*.go # gosub + - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub @@ -38,4 +39,3 @@ files: - golang.org/x/text/encoding/unicode/*.go # gosub - golang.org/x/text/transform/*.go # gosub - gopkg.in/validator.v2/*.go # gosub - - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/mariadb_ctrl/spec b/packages/mariadb_ctrl/spec index 11d01dcc..f38b1b9f 100644 --- a/packages/mariadb_ctrl/spec +++ b/packages/mariadb_ctrl/spec @@ -22,6 +22,7 @@ files: - github.com/cloudfoundry/mariadb_ctrl/start_manager/node_starter/fakes/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/fakes/*.go # gosub + - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub @@ -29,4 +30,3 @@ files: - github.com/tedsuo/ifrit/*.go # gosub - github.com/tedsuo/ifrit/sigmon/*.go # gosub - gopkg.in/validator.v2/*.go # gosub - - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/quota-enforcer/spec b/packages/quota-enforcer/spec index 59d084c4..797524ff 100644 --- a/packages/quota-enforcer/spec +++ b/packages/quota-enforcer/spec @@ -4,6 +4,7 @@ dependencies: - golang files: - github.com/cloudfoundry-incubator/cf-lager/*.go # gosub + - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer/*.go # gosub @@ -16,4 +17,3 @@ files: - github.com/pivotal-golang/lager/*.go # gosub - github.com/tedsuo/ifrit/*.go # gosub - gopkg.in/validator.v2/*.go # gosub - - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/route-registrar/spec b/packages/route-registrar/spec index 9e0a5505..c589eccd 100644 --- a/packages/route-registrar/spec +++ b/packages/route-registrar/spec @@ -8,14 +8,12 @@ files: - github.com/cloudfoundry-incubator/route-registrar/config/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/healthchecker/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/healthchecker/fakes/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/messagebus/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/messagebus/fakes/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/registrar/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/test_helpers/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/cloudfoundry/gibson/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/cloudfoundry/yagnats/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/fraenkel/candiedyaml/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/imdario/mergo/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nats-io/nats/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nats-io/nats/encoders/builtin/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nu7hatch/gouuid/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/pivotal-golang/lager/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/tedsuo/ifrit/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/switchboard/spec b/packages/switchboard/spec index c26aad86..1366325f 100644 --- a/packages/switchboard/spec +++ b/packages/switchboard/spec @@ -16,6 +16,7 @@ files: - github.com/cloudfoundry-incubator/switchboard/dummies/*.go # gosub - github.com/cloudfoundry-incubator/switchboard/health/*.go # gosub - github.com/cloudfoundry-incubator/switchboard/proxy/*.go # gosub + - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/onsi/ginkgo/*.go # gosub - github.com/onsi/ginkgo/config/*.go # gosub @@ -37,4 +38,3 @@ files: - github.com/tedsuo/ifrit/*.go # gosub - github.com/tedsuo/ifrit/grouper/*.go # gosub - gopkg.in/validator.v2/*.go # gosub - - gopkg.in/yaml.v2/*.go # gosub diff --git a/src/github.com/apcera/gnatsd b/src/github.com/apcera/gnatsd deleted file mode 160000 index 972b9c18..00000000 --- a/src/github.com/apcera/gnatsd +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 972b9c18ac61bed63d8ea6fbba3a9eb03b18d1b8 diff --git a/src/github.com/cloudfoundry-incubator/cf-lager b/src/github.com/cloudfoundry-incubator/cf-lager index 71788331..33247902 160000 --- a/src/github.com/cloudfoundry-incubator/cf-lager +++ b/src/github.com/cloudfoundry-incubator/cf-lager @@ -1 +1 @@ -Subproject commit 7178833173e7124dab16158273f7708ed73b81ff +Subproject commit 3324790250247b4d3bd3abc3e74654a537d79a30 diff --git a/src/github.com/cloudfoundry-incubator/route-registrar b/src/github.com/cloudfoundry-incubator/route-registrar index 7bf60c1d..e8834cf6 160000 --- a/src/github.com/cloudfoundry-incubator/route-registrar +++ b/src/github.com/cloudfoundry-incubator/route-registrar @@ -1 +1 @@ -Subproject commit 7bf60c1db84826d74d43cae1a61b7568b3959326 +Subproject commit e8834cf6fd22131d5e943cbf129c9fcd9088e8ac diff --git a/src/github.com/nats-io/gnatsd b/src/github.com/nats-io/gnatsd deleted file mode 160000 index 972b9c18..00000000 --- a/src/github.com/nats-io/gnatsd +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 972b9c18ac61bed63d8ea6fbba3a9eb03b18d1b8 diff --git a/src/github.com/nats-io/nats b/src/github.com/nats-io/nats deleted file mode 160000 index 355b5b97..00000000 --- a/src/github.com/nats-io/nats +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 355b5b97e0842dc94f1106729aa88e33e06317ca diff --git a/src/github.com/onsi/ginkgo b/src/github.com/onsi/ginkgo index 7d3d52b3..17ea4797 160000 --- a/src/github.com/onsi/ginkgo +++ b/src/github.com/onsi/ginkgo @@ -1 +1 @@ -Subproject commit 7d3d52b3a3a79d745c2a03ee81b5bc2086642201 +Subproject commit 17ea479729ee427265ac1e913443018350946ddf diff --git a/src/github.com/onsi/gomega b/src/github.com/onsi/gomega index 85936b29..2152b45f 160000 --- a/src/github.com/onsi/gomega +++ b/src/github.com/onsi/gomega @@ -1 +1 @@ -Subproject commit 85936b29809b7df066a8d839dd105e158e510264 +Subproject commit 2152b45fa28a361beba9aab0885972323a444e28 diff --git a/src/github.com/pivotal-cf-experimental/service-config b/src/github.com/pivotal-cf-experimental/service-config index b1dc94de..d1936ca6 160000 --- a/src/github.com/pivotal-cf-experimental/service-config +++ b/src/github.com/pivotal-cf-experimental/service-config @@ -1 +1 @@ -Subproject commit b1dc94de6ada8bfddf159ce2234958a8f02aec77 +Subproject commit d1936ca6555039c884eb26fabd3396a28e9c488d diff --git a/src/github.com/pivotal-golang/lager b/src/github.com/pivotal-golang/lager index 09a71f6e..389345f0 160000 --- a/src/github.com/pivotal-golang/lager +++ b/src/github.com/pivotal-golang/lager @@ -1 +1 @@ -Subproject commit 09a71f6eb2dcc1c2335e37ceb78a91c6f618d561 +Subproject commit 389345f09521afe9c164379d98cb70335aff8726 diff --git a/src/github.com/tedsuo/ifrit b/src/github.com/tedsuo/ifrit index 3a41de6e..61109605 160000 --- a/src/github.com/tedsuo/ifrit +++ b/src/github.com/tedsuo/ifrit @@ -1 +1 @@ -Subproject commit 3a41de6efdb78719e66467a1b170f2b7f1e1014d +Subproject commit 611096055721e35a95e3d1e4e2bc666fae509fcb diff --git a/src/gopkg.in/yaml.v2 b/src/gopkg.in/yaml.v2 deleted file mode 160000 index 49c95bdc..00000000 --- a/src/gopkg.in/yaml.v2 +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 49c95bdc21843256fb6c4e0d370a05f24a0bf213 From d859f658fd9df08a0db7f47531084e5b1dc321f5 Mon Sep 17 00:00:00 2001 From: test Date: Mon, 1 Feb 2016 16:50:27 -0800 Subject: [PATCH 077/110] Revert "Oopsy in .gitmodules" This reverts commit 2510132f030623217e3e4865a000359b9bd6a205. --- .gitmodules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitmodules b/.gitmodules index dd6cb0f5..7f2b4f34 100644 --- a/.gitmodules +++ b/.gitmodules @@ -116,7 +116,7 @@ branch = master [submodule "src/github.com/apcera/gnatsd"] path = src/github.com/apcera/gnatsd - url = https://github.com/apcera/gnatsd + url = https://github/apcera/gnatsd branch = master [submodule "src/github.com/nats-io/nats"] path = src/github.com/nats-io/nats From 42babbd2989d1350b56df6f32137f930e2b9b1a9 Mon Sep 17 00:00:00 2001 From: test Date: Mon, 1 Feb 2016 16:50:53 -0800 Subject: [PATCH 078/110] Revert "Add missing .gitmodule references to new submodules [#112556657]" This reverts commit db35e1211a1ff2ac4d6c5768e21660607550714f. --- .gitmodules | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/.gitmodules b/.gitmodules index 7f2b4f34..626dbd17 100644 --- a/.gitmodules +++ b/.gitmodules @@ -114,19 +114,3 @@ path = src/github.com/bmizerany/pat url = https://github.com/bmizerany/pat branch = master -[submodule "src/github.com/apcera/gnatsd"] - path = src/github.com/apcera/gnatsd - url = https://github/apcera/gnatsd - branch = master -[submodule "src/github.com/nats-io/nats"] - path = src/github.com/nats-io/nats - url = https://github.com/nats-io/nats - branch = master -[submodule "src/github.com/nats-io/gnatsd"] - path = src/github.com/nats-io/gnatsd - url = https://github.com/nats-io/gnatsd - branch = master -[submodule "src/gopkg.in/yaml.v2"] - path = src/gopkg.in/yaml.v2 - url = https://gopkg.in/yaml.v2 - branch = master From 3b29211947bd766d83d8552b6bc9170f1cf7ac0e Mon Sep 17 00:00:00 2001 From: Marco Nicosia Date: Mon, 1 Feb 2016 19:24:00 -0800 Subject: [PATCH 079/110] Configuration documentation changes - Move additional configuration options out of README/md to its own file. - Add in documentation of the 'roadmin' user password configuration. --- README.md | 57 +------------------------------------------ docs/configuration.md | 57 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 56 deletions(-) create mode 100644 docs/configuration.md diff --git a/README.md b/README.md index 3904d100..1c990804 100644 --- a/README.md +++ b/README.md @@ -20,8 +20,6 @@ [Deregistering the Service Broker](#deregistering-broker) -[Additional Configuration Options](#additional-configuration-options) - [CI](http://www.github.com/cloudfoundry-incubator/cf-mysql-ci) @@ -253,7 +251,7 @@ Currently, load balancing requests across both proxies can increase the possibil To avoid this problem, configure the load balancer to route requests to the second proxy only in the event of a failure. - **Note:** When using an Elastic Load Balancer (ELB) on Amazon, make sure to create the ELB in the same VPC as your cf-mysql deployment -- **Note:** For all load balancers, take special care to configure health checks to use the health_port of the proxies (default 1936). Do not configure the load balancer to use port 3306. +- **Note:** For all load balancers, take special care to configure health checks to use the health_port of the proxies (default 1936). Do not configure the load balancer to use port 3306. ### Deployment Components @@ -491,56 +489,3 @@ $ cf delete-service-broker p-mysql ## Deployment Resources The service is configured to have a small footprint out of the box. These resources are sufficient for development, but may be insufficient for production workloads. If the service appears to be performing poorly, redeploying with increased resources may improve performance. See [deployment resources](docs/deployment-resources.md) for further details. - - -## Additional Configuration Options - -### Updating Service Plans - -Updating the service instances is supported; see [Service plans and instances](docs/service-plans-instances.md) for details. - -### Pre-seeding Databases - -Normally databases are created via the `cf create-service` command, and -a MySQL user is created and given access to that database when an app is bound to that service instance. -However, it is sometimes useful to have databases and users already available when the service is deployed, -without having to run `cf create-service` or bind an app. -To specify any preseeded databases, add the following to the deployment manifest: - -``` -jobs: -- name: mysql_z1 - properties: - seeded_databases: - - name: db1 - username: user1 - password: pw1 - - name: db2 - username: user2 - password: pw2 -``` - -Note: If all you need is a database deployment, it is possible to deploy this -release with zero broker instances and completely remove any dependencies on Cloud Foundry. -See the [proxy](jobs/proxy/spec) and [acceptance-tests](jobs/acceptance-tests/spec) spec files for standalone configuration options. - -### Configuring how long the startup script waits for the database to come online - -On larger databases, the default database startup timeout may be too low. -This would result in the job reporting as failing, while MySQL continues to bootstrap in the background (see [Known Issues > Long SST Transfers](docs/Known-Issues.md#long-sst-transfers)). -To increase the duration that the startup script waits for MySQL to start, add the following to your deployment stub: - -```yaml -jobs: -- name: mysql_z1 - properties: - database_startup_timeout: 360 -``` - -Note: This is independent of the overall BOSH timeout which is also configurable in the manifest. The BOSH timeout should always be higher than the database startup timeout: - -```yaml -update: - canary_watch_time: 30000-600000 - update_watch_time: 30000-600000 -``` diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 00000000..51a313c4 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,57 @@ +# Configuration Options + +## Updating Service Plans + +Updating the service instances is supported; see [Service plans and instances](docs/service-plans-instances.md) for details. + +## Read-Only Administrator User + +The manifest optionally allows the user to specify a password for the `roadmin` user. By supplying this password, the service will automatically create a user that has access to read all databases, but permission to write to none of them. + +This parameter is defined in the [spec file](jobs/mysql/spec). + +## Pre-seeding Databases + +Normally databases are created via the `cf create-service` command, and +a MySQL user is created and given access to that database when an app is bound to that service instance. +However, it is sometimes useful to have databases and users already available when the service is deployed, +without having to run `cf create-service` or bind an app. +To specify any preseeded databases, add the following to the deployment manifest: + +``` +jobs: +- name: mysql_z1 + properties: + seeded_databases: + - name: db1 + username: user1 + password: pw1 + - name: db2 + username: user2 + password: pw2 +``` + +Note: If all you need is a database deployment, it is possible to deploy this +release with zero broker instances and completely remove any dependencies on Cloud Foundry. +See the [proxy](jobs/proxy/spec) and [acceptance-tests](jobs/acceptance-tests/spec) spec files for standalone configuration options. + +## Configuring how long the startup script waits for the database to come online + +On larger databases, the default database startup timeout may be too low. +This would result in the job reporting as failing, while MySQL continues to bootstrap in the background (see [Known Issues > Long SST Transfers](docs/Known-Issues.md#long-sst-transfers)). +To increase the duration that the startup script waits for MySQL to start, add the following to your deployment stub: + +```yaml +jobs: +- name: mysql_z1 + properties: + database_startup_timeout: 360 +``` + +Note: This is independent of the overall BOSH timeout which is also configurable in the manifest. The BOSH timeout should always be higher than the database startup timeout: + +```yaml +update: + canary_watch_time: 30000-600000 + update_watch_time: 30000-600000 +``` From d5eee28e30799fbb7c749ca3a79767abf1eeda12 Mon Sep 17 00:00:00 2001 From: Marco Nicosia Date: Mon, 1 Feb 2016 19:25:51 -0800 Subject: [PATCH 080/110] Fix bad path to spec file --- docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configuration.md b/docs/configuration.md index 51a313c4..c750058d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -8,7 +8,7 @@ Updating the service instances is supported; see [Service plans and instances](d The manifest optionally allows the user to specify a password for the `roadmin` user. By supplying this password, the service will automatically create a user that has access to read all databases, but permission to write to none of them. -This parameter is defined in the [spec file](jobs/mysql/spec). +This parameter is defined in the [spec file](../jobs/mysql/spec). ## Pre-seeding Databases From 15faab280419d91172aecc65bc08f4792ab27126 Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Tue, 2 Feb 2016 11:25:45 -0800 Subject: [PATCH 081/110] Bumped route-registrar to latest [#112556657] Signed-off-by: Aditya Anchuri --- .../templates/registrar_settings.yml.erb | 17 ++++++++++------- jobs/proxy/templates/route-registrar.yml.erb | 10 +++++++--- packages/bootstrap/spec | 2 +- packages/galera-healthcheck/spec | 2 +- packages/mariadb_ctrl/spec | 2 +- packages/quota-enforcer/spec | 2 +- packages/route-registrar/spec | 10 ++++++---- packages/switchboard/spec | 2 +- src/github.com/apcera/gnatsd | 1 + .../cloudfoundry-incubator/cf-mysql-bootstrap | 2 +- .../cloudfoundry-incubator/galera-healthcheck | 2 +- .../cloudfoundry-incubator/route-registrar | 2 +- .../cloudfoundry-incubator/switchboard | 2 +- src/github.com/go-yaml/yaml | 1 + src/github.com/nats-io/gnatsd | 1 + src/github.com/nats-io/nats | 1 + .../pivotal-cf-experimental/service-config | 2 +- src/gopkg.in/yaml.v2 | 1 + 18 files changed, 38 insertions(+), 24 deletions(-) create mode 160000 src/github.com/apcera/gnatsd create mode 160000 src/github.com/go-yaml/yaml create mode 160000 src/github.com/nats-io/gnatsd create mode 160000 src/github.com/nats-io/nats create mode 160000 src/gopkg.in/yaml.v2 diff --git a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb index 9ef187d8..cefc4adc 100644 --- a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb +++ b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb @@ -4,10 +4,13 @@ message_bus_servers: user: <%= p('nats.user') %> password: <%= p('nats.password') %> <% end %> -external_host: <%= p('cf_mysql.external_host') %> -external_ip: <%= spec.networks.send(p('network_name')).ip %> -port: 80 -health_checker: - name: script - interval_in_seconds: 10 - healthcheck_script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh +host: <%= spec.networks.send(p('network_name')).ip %> +routes: +- name: "broker_"<%= index %> + port: 80 + uris: + - <%= p('cf_mysql.external_host') %> +registration_interval: 10 +health_check: + name: script + script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh diff --git a/jobs/proxy/templates/route-registrar.yml.erb b/jobs/proxy/templates/route-registrar.yml.erb index 4a30bf36..5df85ab0 100644 --- a/jobs/proxy/templates/route-registrar.yml.erb +++ b/jobs/proxy/templates/route-registrar.yml.erb @@ -6,6 +6,10 @@ message_bus_servers: <% end %> <% my_ip = spec.networks.send(p('network_name')).ip %> <% proxy_index = p('cf_mysql.proxy.proxy_ips').index(my_ip) %> -external_host: proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> -external_ip: <%= my_ip %> -port: <%= p('cf_mysql.proxy.api_port') %> +host: <%= my_ip %> +routes: +- name: "proxy_"<%= index %> + port: <%= p('cf_mysql.proxy.api_port') %> + uris: + - proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> + registration_interval: 10 diff --git a/packages/bootstrap/spec b/packages/bootstrap/spec index 816d1f72..018b7ddb 100644 --- a/packages/bootstrap/spec +++ b/packages/bootstrap/spec @@ -13,8 +13,8 @@ files: - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/config/*.go # gosub - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/fakes/*.go # gosub - github.com/cloudfoundry-incubator/cf-mysql-bootstrap/test_helpers/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/pivotal-golang/lager/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/galera-healthcheck/spec b/packages/galera-healthcheck/spec index d9e3595e..d9a56e8c 100644 --- a/packages/galera-healthcheck/spec +++ b/packages/galera-healthcheck/spec @@ -19,7 +19,6 @@ files: - github.com/cloudfoundry-incubator/galera-healthcheck/mysqld_cmd/fakes/*.go # gosub - github.com/cloudfoundry-incubator/galera-healthcheck/sequence_number/*.go # gosub - github.com/cloudfoundry-incubator/galera-healthcheck/sequence_number/fakes/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub @@ -39,3 +38,4 @@ files: - golang.org/x/text/encoding/unicode/*.go # gosub - golang.org/x/text/transform/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/mariadb_ctrl/spec b/packages/mariadb_ctrl/spec index f38b1b9f..11d01dcc 100644 --- a/packages/mariadb_ctrl/spec +++ b/packages/mariadb_ctrl/spec @@ -22,7 +22,6 @@ files: - github.com/cloudfoundry/mariadb_ctrl/start_manager/node_starter/fakes/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/*.go # gosub - github.com/cloudfoundry/mariadb_ctrl/upgrader/fakes/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/service-config/*.go # gosub @@ -30,3 +29,4 @@ files: - github.com/tedsuo/ifrit/*.go # gosub - github.com/tedsuo/ifrit/sigmon/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/quota-enforcer/spec b/packages/quota-enforcer/spec index 797524ff..59d084c4 100644 --- a/packages/quota-enforcer/spec +++ b/packages/quota-enforcer/spec @@ -4,7 +4,6 @@ dependencies: - golang files: - github.com/cloudfoundry-incubator/cf-lager/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/go-sql-driver/mysql/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer/*.go # gosub @@ -17,3 +16,4 @@ files: - github.com/pivotal-golang/lager/*.go # gosub - github.com/tedsuo/ifrit/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/route-registrar/spec b/packages/route-registrar/spec index c589eccd..9e0a5505 100644 --- a/packages/route-registrar/spec +++ b/packages/route-registrar/spec @@ -8,12 +8,14 @@ files: - github.com/cloudfoundry-incubator/route-registrar/config/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/healthchecker/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/healthchecker/fakes/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/messagebus/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/messagebus/fakes/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/registrar/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/test_helpers/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/cloudfoundry/gibson/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/cloudfoundry/yagnats/*.go # gosub - - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/fraenkel/candiedyaml/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/imdario/mergo/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nats-io/nats/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nats-io/nats/encoders/builtin/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/nu7hatch/gouuid/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/pivotal-cf-experimental/service-config/*.go # gosub - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/pivotal-golang/lager/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/github.com/tedsuo/ifrit/*.go # gosub + - github.com/cloudfoundry-incubator/route-registrar/Godeps/_workspace/src/gopkg.in/yaml.v2/*.go # gosub diff --git a/packages/switchboard/spec b/packages/switchboard/spec index 1366325f..c26aad86 100644 --- a/packages/switchboard/spec +++ b/packages/switchboard/spec @@ -16,7 +16,6 @@ files: - github.com/cloudfoundry-incubator/switchboard/dummies/*.go # gosub - github.com/cloudfoundry-incubator/switchboard/health/*.go # gosub - github.com/cloudfoundry-incubator/switchboard/proxy/*.go # gosub - - github.com/fraenkel/candiedyaml/*.go # gosub - github.com/imdario/mergo/*.go # gosub - github.com/onsi/ginkgo/*.go # gosub - github.com/onsi/ginkgo/config/*.go # gosub @@ -38,3 +37,4 @@ files: - github.com/tedsuo/ifrit/*.go # gosub - github.com/tedsuo/ifrit/grouper/*.go # gosub - gopkg.in/validator.v2/*.go # gosub + - gopkg.in/yaml.v2/*.go # gosub diff --git a/src/github.com/apcera/gnatsd b/src/github.com/apcera/gnatsd new file mode 160000 index 00000000..ed3c20f1 --- /dev/null +++ b/src/github.com/apcera/gnatsd @@ -0,0 +1 @@ +Subproject commit ed3c20f13ec905cc19c139d2fe5cb090dd30e979 diff --git a/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap b/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap index e5fadb76..0bdabe51 160000 --- a/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap +++ b/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap @@ -1 +1 @@ -Subproject commit e5fadb76d3ce0d9ffdb1430bff9d82825ad8a0b7 +Subproject commit 0bdabe51e476bbc16ae185ed846655bfcb99b514 diff --git a/src/github.com/cloudfoundry-incubator/galera-healthcheck b/src/github.com/cloudfoundry-incubator/galera-healthcheck index 88d78c23..36468bae 160000 --- a/src/github.com/cloudfoundry-incubator/galera-healthcheck +++ b/src/github.com/cloudfoundry-incubator/galera-healthcheck @@ -1 +1 @@ -Subproject commit 88d78c23763a9e9edcd100e7f749dacc0ff58fa2 +Subproject commit 36468bae0c77517fb05e1dc5ecdbe9aab3852ed8 diff --git a/src/github.com/cloudfoundry-incubator/route-registrar b/src/github.com/cloudfoundry-incubator/route-registrar index e8834cf6..7bf60c1d 160000 --- a/src/github.com/cloudfoundry-incubator/route-registrar +++ b/src/github.com/cloudfoundry-incubator/route-registrar @@ -1 +1 @@ -Subproject commit e8834cf6fd22131d5e943cbf129c9fcd9088e8ac +Subproject commit 7bf60c1db84826d74d43cae1a61b7568b3959326 diff --git a/src/github.com/cloudfoundry-incubator/switchboard b/src/github.com/cloudfoundry-incubator/switchboard index de8edb53..ea2daace 160000 --- a/src/github.com/cloudfoundry-incubator/switchboard +++ b/src/github.com/cloudfoundry-incubator/switchboard @@ -1 +1 @@ -Subproject commit de8edb53739129601339dfaf0736c018779969be +Subproject commit ea2daaceb8217c88a45d7a18a90e677a53754a2a diff --git a/src/github.com/go-yaml/yaml b/src/github.com/go-yaml/yaml new file mode 160000 index 00000000..bec87e43 --- /dev/null +++ b/src/github.com/go-yaml/yaml @@ -0,0 +1 @@ +Subproject commit bec87e4332aede01fb63a4ab299d8af28480cd96 diff --git a/src/github.com/nats-io/gnatsd b/src/github.com/nats-io/gnatsd new file mode 160000 index 00000000..ed3c20f1 --- /dev/null +++ b/src/github.com/nats-io/gnatsd @@ -0,0 +1 @@ +Subproject commit ed3c20f13ec905cc19c139d2fe5cb090dd30e979 diff --git a/src/github.com/nats-io/nats b/src/github.com/nats-io/nats new file mode 160000 index 00000000..355b5b97 --- /dev/null +++ b/src/github.com/nats-io/nats @@ -0,0 +1 @@ +Subproject commit 355b5b97e0842dc94f1106729aa88e33e06317ca diff --git a/src/github.com/pivotal-cf-experimental/service-config b/src/github.com/pivotal-cf-experimental/service-config index d1936ca6..b1dc94de 160000 --- a/src/github.com/pivotal-cf-experimental/service-config +++ b/src/github.com/pivotal-cf-experimental/service-config @@ -1 +1 @@ -Subproject commit d1936ca6555039c884eb26fabd3396a28e9c488d +Subproject commit b1dc94de6ada8bfddf159ce2234958a8f02aec77 diff --git a/src/gopkg.in/yaml.v2 b/src/gopkg.in/yaml.v2 new file mode 160000 index 00000000..f7716cbe --- /dev/null +++ b/src/gopkg.in/yaml.v2 @@ -0,0 +1 @@ +Subproject commit f7716cbe52baa25d2e9b0d0da546fcf909fc16b4 From 96e34398d4e6d1061e51a1eea6cb53d48dc26d62 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Tue, 2 Feb 2016 11:31:07 -0800 Subject: [PATCH 082/110] Add back missing submodule refs [#111778592] Signed-off-by: Aaron Hurley --- .gitmodules | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/.gitmodules b/.gitmodules index 626dbd17..291339c5 100644 --- a/.gitmodules +++ b/.gitmodules @@ -114,3 +114,19 @@ path = src/github.com/bmizerany/pat url = https://github.com/bmizerany/pat branch = master +[submodule "src/github.com/apcera/gnatsd"] + path = src/github.com/apcera/gnatsd + url = https://github.com/apcera/gnatsd + branch = master +[submodule "src/github.com/nats-io/nats"] + path = src/github.com/nats-io/nats + url = https://github.com/nats-io/nats + branch = master +[submodule "src/github.com/nats-io/gnatsd"] + path = src/github.com/nats-io/gnatsd + url = https://github.com/nats-io/gnatsd + branch = master +[submodule "src/gopkg.in/yaml.v2"] + path = src/gopkg.in/yaml.v2 + url = https://gopkg.in/yaml.v2 + branch = master From 1a20d9d904e11ab2878e35f03c973d0bb1f94364 Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Tue, 2 Feb 2016 12:06:15 -0800 Subject: [PATCH 083/110] Added go-yaml repo to .gitmodules [#112556657] Signed-off-by: Aditya Anchuri --- .gitmodules | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/.gitmodules b/.gitmodules index 291339c5..073669c8 100644 --- a/.gitmodules +++ b/.gitmodules @@ -119,14 +119,18 @@ url = https://github.com/apcera/gnatsd branch = master [submodule "src/github.com/nats-io/nats"] - path = src/github.com/nats-io/nats - url = https://github.com/nats-io/nats - branch = master + path = src/github.com/nats-io/nats + url = https://github.com/nats-io/nats + branch = master [submodule "src/github.com/nats-io/gnatsd"] - path = src/github.com/nats-io/gnatsd - url = https://github.com/nats-io/gnatsd - branch = master + path = src/github.com/nats-io/gnatsd + url = https://github.com/nats-io/gnatsd + branch = master [submodule "src/gopkg.in/yaml.v2"] - path = src/gopkg.in/yaml.v2 - url = https://gopkg.in/yaml.v2 - branch = master + path = src/gopkg.in/yaml.v2 + url = https://gopkg.in/yaml.v2 + branch = master +[submodule "src/github.com/go-yaml/yaml"] + path = src/github.com/go-yaml/yaml + url = https://github.com/go-yaml/yaml + branch = master From e8ec7bc85ed846c7d41ee5c1691565bf828d978a Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Tue, 2 Feb 2016 13:04:43 -0800 Subject: [PATCH 084/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Updated mariadb_ctrl to have yaml tags in config struct ( Travis Unknown ) Signed-off-by: Aditya Anchuri --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index c47580af..596150d9 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit c47580af8204843b4611f510fc43442b98ed0e2e +Subproject commit 596150d9d5a4ec7dda4a915179a6b9ceb954b7ba From 73e697579a4f4d1e17b9e54fddf8e28026cc93de Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Tue, 2 Feb 2016 14:02:57 -0800 Subject: [PATCH 085/110] Cluster IPs should have all 4 IPs (3 nodes plus arbitrator) - For some reason this never got put into the manifest template, even though it was explicitly mentioned in the rolling deploy story [#110772724] [#111741306] Signed-off-by: Aditya Anchuri --- manifest-generation/cf-mysql-template.yml | 33 +++++------------------ 1 file changed, 6 insertions(+), 27 deletions(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 07e25efa..535866e0 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -135,33 +135,6 @@ properties: source_executable: (( property_overrides.service-backup.source_executable || nil )) cleanup_executable: (( property_overrides.service-backup.cleanup_executable || nil )) -# The keys below should not be included in the final stub -config_from_cf: (( merge )) -iaas_settings: (( merge )) -property_overrides: (( merge )) -instance_count_overrides: (( merge || nil )) -release_versions: (( merge || nil )) -job_overrides: (( merge || nil )) - -default_mysql_cluster_ips: -- (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) -- (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) -- (( jobs.arbitrator_z3.networks.mysql3.static_ips.[0] || jobs.mysql_z3.networks.mysql3.static_ips.[0] || nil )) -default_proxy_ips: -- (( jobs.proxy_z1.networks.mysql1.static_ips.[0] || nil )) -- (( jobs.proxy_z2.networks.mysql2.static_ips.[0] || nil )) -# spiff does not allow concating multiple fields of different types -# without adding new temporary properties -database_startup_timeout_str: (( "" .properties.cf_mysql.mysql.database_startup_timeout )) -canary_end_in_sec: (( database_startup_timeout_str "000" )) - -additional_disk_pools: (( iaas_settings.additional_disk_pools || empty_list )) -additional_releases: (( job_overrides.additional_releases || empty_list )) -additional_resource_pools: (( iaas_settings.additional_resource_pools || empty_list)) -additional_jobs: (( job_overrides.additional_jobs || empty_list )) -empty_hash: {} -empty_list: [] - base_releases: - name: cf-mysql version: (( release_versions.cf-mysql.version || "latest" )) @@ -356,3 +329,9 @@ default_proxy_ips: # without adding new temporary properties database_startup_timeout_str: (( "" .properties.cf_mysql.mysql.database_startup_timeout )) canary_end_in_sec: (( database_startup_timeout_str "000" )) +additional_disk_pools: (( iaas_settings.additional_disk_pools || empty_list )) +additional_releases: (( job_overrides.additional_releases || empty_list )) +additional_resource_pools: (( iaas_settings.additional_resource_pools || empty_list)) +additional_jobs: (( job_overrides.additional_jobs || empty_list )) +empty_hash: {} +empty_list: [] From f83bd9b7501987243236627e9144dadf7a431c73 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Tue, 2 Feb 2016 14:06:02 -0800 Subject: [PATCH 086/110] Remove duplicate empty_hash field [ci skip] Signed-off-by: Aaron Hurley --- manifest-generation/cf-mysql-template.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index 535866e0..a7b5acba 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -316,7 +316,6 @@ iaas_settings: (( merge )) instance_count_overrides: (( merge || nil )) property_overrides: (( merge )) release_versions: (( merge || nil )) -empty_hash: {} default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) From fa893ec56e09d118d954f0ebba9dd8a1e54daab9 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Tue, 2 Feb 2016 15:08:49 -0800 Subject: [PATCH 087/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aaron Hurley: Added some more YAML tags in struct fields ( Travis Unknown ) Signed-off-by: Aaron Hurley --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 596150d9..6093457b 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 596150d9d5a4ec7dda4a915179a6b9ceb954b7ba +Subproject commit 6093457b2964669e18f86ed62df14408f1f14f8e From 0886894a681b718c5a78b42a07da4034491ec063 Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Tue, 2 Feb 2016 16:10:57 -0800 Subject: [PATCH 088/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Missed YAML tag for read only user in previous commit ( Travis Unknown ) Signed-off-by: Aditya Anchuri --- jobs/mysql/templates/mariadb_ctl_config.yml.erb | 1 - src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/jobs/mysql/templates/mariadb_ctl_config.yml.erb b/jobs/mysql/templates/mariadb_ctl_config.yml.erb index 750edfa6..137e0085 100644 --- a/jobs/mysql/templates/mariadb_ctl_config.yml.erb +++ b/jobs/mysql/templates/mariadb_ctl_config.yml.erb @@ -19,7 +19,6 @@ Db: User: <%= seed["username"]%> Password: <%= seed["password"] %> <% end %> - ROUser: roadmin Upgrader: PackageVersionFile: /var/vcap/packages/mariadb/VERSION LastUpgradedVersionFile: /var/vcap/store/mysql/mysql_upgrade_info diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 6093457b..6c51f9c6 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 6093457b2964669e18f86ed62df14408f1f14f8e +Subproject commit 6c51f9c64dbe6f23a553dc16d269e41ce46c0c87 From 86d24d9d5b615b8617c9088199a6ce29153b775c Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Tue, 2 Feb 2016 16:54:58 -0800 Subject: [PATCH 089/110] Quote was in the wrong place Signed-off-by: Aditya Anchuri --- jobs/cf-mysql-broker/templates/registrar_settings.yml.erb | 2 +- jobs/proxy/templates/route-registrar.yml.erb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb index cefc4adc..b37eada1 100644 --- a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb +++ b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb @@ -6,7 +6,7 @@ message_bus_servers: <% end %> host: <%= spec.networks.send(p('network_name')).ip %> routes: -- name: "broker_"<%= index %> +- name: "broker_<%= index %>" port: 80 uris: - <%= p('cf_mysql.external_host') %> diff --git a/jobs/proxy/templates/route-registrar.yml.erb b/jobs/proxy/templates/route-registrar.yml.erb index 5df85ab0..e9da0f31 100644 --- a/jobs/proxy/templates/route-registrar.yml.erb +++ b/jobs/proxy/templates/route-registrar.yml.erb @@ -8,7 +8,7 @@ message_bus_servers: <% proxy_index = p('cf_mysql.proxy.proxy_ips').index(my_ip) %> host: <%= my_ip %> routes: -- name: "proxy_"<%= index %> +- name: "proxy_<%= index %>" port: <%= p('cf_mysql.proxy.api_port') %> uris: - proxy-<%= proxy_index %>-<%= p('cf_mysql.external_host') %> From b690c87e9f8b9e96b8865385bf67acad0912006b Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Tue, 2 Feb 2016 18:00:35 -0800 Subject: [PATCH 090/110] Fixed yaml formatting issue in route registrar config [#112556657] Signed-off-by: Aaron Hurley --- .../templates/registrar_settings.yml.erb | 14 +++++++------- jobs/proxy/templates/route-registrar.yml.erb | 6 +++--- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb index b37eada1..5490f222 100644 --- a/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb +++ b/jobs/cf-mysql-broker/templates/registrar_settings.yml.erb @@ -1,8 +1,8 @@ message_bus_servers: <% p('nats.machines').each do |ip| %> - - host: <%= ip %>:<%= p('nats.port') %> - user: <%= p('nats.user') %> - password: <%= p('nats.password') %> +- host: <%= ip %>:<%= p('nats.port') %> + user: <%= p('nats.user') %> + password: <%= p('nats.password') %> <% end %> host: <%= spec.networks.send(p('network_name')).ip %> routes: @@ -10,7 +10,7 @@ routes: port: 80 uris: - <%= p('cf_mysql.external_host') %> -registration_interval: 10 -health_check: - name: script - script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh + registration_interval: 10 + health_check: + name: script + script_path: /var/vcap/jobs/cf-mysql-broker/bin/healthcheck.sh diff --git a/jobs/proxy/templates/route-registrar.yml.erb b/jobs/proxy/templates/route-registrar.yml.erb index e9da0f31..8aa3fdf1 100644 --- a/jobs/proxy/templates/route-registrar.yml.erb +++ b/jobs/proxy/templates/route-registrar.yml.erb @@ -1,8 +1,8 @@ message_bus_servers: <% p('nats.machines').each do |ip| %> - - host: <%= ip %>:<%= p('nats.port') %> - user: <%= p('nats.user') %> - password: <%= p('nats.password') %> +- host: <%= ip %>:<%= p('nats.port') %> + user: <%= p('nats.user') %> + password: <%= p('nats.password') %> <% end %> <% my_ip = spec.networks.send(p('network_name')).ip %> <% proxy_index = p('cf_mysql.proxy.proxy_ips').index(my_ip) %> From d4f0edc94d116fa4c8db3c0626ba2f20930a7fad Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Wed, 3 Feb 2016 08:34:31 -0800 Subject: [PATCH 091/110] Bump src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer Bump pivotal-cf-experimental/cf-mysql-quota-enforcer: Aaron Hurley: Use yaml construct tags in config ( Travis Unknown ) Ben Calegari: Do not enforce a quota on read only user ( Travis Unknown ) Move reformer_test to reformer_repo_test to match violator_repo ( Travis Unknown ) Remove debug log lines ( Travis Unknown ) Signed-off-by: Aaron Hurley --- src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer index df200556..a60f2f2d 160000 --- a/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer +++ b/src/github.com/pivotal-cf-experimental/cf-mysql-quota-enforcer @@ -1 +1 @@ -Subproject commit df20055636916a26a1f50328063014fcffb56111 +Subproject commit a60f2f2d7d5a3cd5f5432a478a2c56916a9eb1c9 From bac27ffc6cf62ea7925a56a9e40599f3cebc7c63 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Wed, 3 Feb 2016 10:22:56 -0800 Subject: [PATCH 092/110] Bump src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap Bump : test: Revert "Revert "Update node_manager to handle an arbitrator node"" Signed-off-by: Aaron Hurley --- src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap b/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap index 0bdabe51..0418851b 160000 --- a/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap +++ b/src/github.com/cloudfoundry-incubator/cf-mysql-bootstrap @@ -1 +1 @@ -Subproject commit 0bdabe51e476bbc16ae185ed846655bfcb99b514 +Subproject commit 0418851b7f34c6ed615030c6ef5b60c1d64957b9 From 698a08c2362139523f9481222bd16f823fb08da8 Mon Sep 17 00:00:00 2001 From: Marco Nicosia Date: Wed, 3 Feb 2016 14:00:56 -0800 Subject: [PATCH 093/110] Change formatting to make copy/paste easier --- docs/arbitrator.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/arbitrator.md b/docs/arbitrator.md index c8f54c0d..0569ef11 100644 --- a/docs/arbitrator.md +++ b/docs/arbitrator.md @@ -12,22 +12,24 @@ For a fresh deployment of CF MySQL v26, simply follow the steps in the [README]( If you already have a 3-node deployment of CF MySQL (e.g. v25), follow the below steps to upgrade to the new configuration as a rolling deploy (the commands below apply to testing on a bosh-lite deployment. For other environments, the same steps are to be followed, except that stubs are different, as explained in the README linked above.). -1. Generate a 3 node + arbitrator manifest: `./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > .yml` -1. `bosh deployment .yml` +1. Generate a 3 node + arbitrator manifest: + 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > [manifest export path].yml
+1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` 1. Upon successful deployment, generate the 2 node + arbitrator manifest: -`./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml > .yml` -1. `bosh deployment .yml` + 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml > [manifest export path].yml
+1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` ### Rolling from 2-node-plus-arbitrator back to 3-node deployment If you wish to go back to a 3-node deployment from a 2-node-plus-arbitrator deployment, follow the steps below to perform the downgrade: -1. Generate a 3-node + arbitrator manifest: `./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > .yml` -1. `bosh deployment .yml` +1. Generate a 3-node + arbitrator manifest: + 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > [manifest export path].yml
+1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` 1. Upon successful deployment, generate the 3 node manifest: -`./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/no-arbitrator/instance-count-overrides.yml > .yml` -1. `bosh deployment .yml` + 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/no-arbitrator/instance-count-overrides.yml > [manifest export path].yml
+1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` From 3cce3e73e72773358c6eeb7c98b7a06c4e7e62f6 Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Wed, 3 Feb 2016 17:02:56 -0800 Subject: [PATCH 094/110] Added manifest properties for enabling read only user [#112020565] Signed-off-by: Aditya Anchuri --- jobs/mysql/spec | 3 +++ jobs/mysql/templates/mariadb_ctl_config.yml.erb | 1 + src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/jobs/mysql/spec b/jobs/mysql/spec index 0a7872d4..1ba5874f 100644 --- a/jobs/mysql/spec +++ b/jobs/mysql/spec @@ -78,6 +78,9 @@ properties: cf_mysql.mysql.seeded_databases: description: 'Set of databases to seed' default: {} + cf_mysql.mysql.roadmin_enabled: + description: 'Whether read only user is enabled' + default: false cf_mysql.mysql.roadmin_password: description: 'Password for the MySQL server read-only admin user' cf_mysql.mysql.database_startup_timeout: diff --git a/jobs/mysql/templates/mariadb_ctl_config.yml.erb b/jobs/mysql/templates/mariadb_ctl_config.yml.erb index 137e0085..f4a21488 100644 --- a/jobs/mysql/templates/mariadb_ctl_config.yml.erb +++ b/jobs/mysql/templates/mariadb_ctl_config.yml.erb @@ -11,6 +11,7 @@ Db: UpgradePath: /var/vcap/packages/mariadb/bin/mysql_upgrade User: <%= p('cf_mysql.mysql.admin_username')%> Password: <%= p('cf_mysql.mysql.admin_password')%> + ReadOnlyUserEnabled: <%= p('cf_mysql.mysql.roadmin_enabled')%> ReadOnlyUser: roadmin ReadOnlyPassword: <%= p('cf_mysql.mysql.roadmin_password')%> PreseededDatabases: diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 6c51f9c6..c5128fed 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 6c51f9c64dbe6f23a553dc16d269e41ce46c0c87 +Subproject commit c5128fed270ebf3728163c104c7e4563e6156789 From f809be2bfb0cf9a92d90f75172f8b336df49b53b Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Thu, 4 Feb 2016 10:04:21 -0800 Subject: [PATCH 095/110] Added read-only user enabled field in manifest generation stubs [#112020565] Signed-off-by: Aaron Hurley --- .../bosh-lite-stubs/property-overrides-standalone.yml | 1 + manifest-generation/bosh-lite-stubs/property-overrides.yml | 1 + manifest-generation/cf-mysql-template.yml | 1 + manifest-generation/examples/property-overrides.yml | 1 + 4 files changed, 4 insertions(+) diff --git a/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml index 7d5013f8..522a24c1 100644 --- a/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml +++ b/manifest-generation/bosh-lite-stubs/property-overrides-standalone.yml @@ -2,6 +2,7 @@ property_overrides: standalone: true mysql: admin_password: password + roadmin_enabled: true roadmin_password: password bootstrap_endpoint: username: username diff --git a/manifest-generation/bosh-lite-stubs/property-overrides.yml b/manifest-generation/bosh-lite-stubs/property-overrides.yml index 25515a5f..2f090cbd 100644 --- a/manifest-generation/bosh-lite-stubs/property-overrides.yml +++ b/manifest-generation/bosh-lite-stubs/property-overrides.yml @@ -1,6 +1,7 @@ property_overrides: mysql: admin_password: password + roadmin_enabled: true roadmin_password: password bootstrap_endpoint: username: username diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index a7b5acba..ea302468 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -65,6 +65,7 @@ properties: healthcheck_port: (( property_overrides.mysql.healthcheck_port || nil )) bootstrap_endpoint: (( property_overrides.mysql.bootstrap_endpoint )) cluster_ips: (( property_overrides.mysql.cluster_ips || default_mysql_cluster_ips )) + roadmin_enabled: (( property_overrides.mysql.roadmin_enabled || false )) roadmin_password: (( property_overrides.mysql.roadmin_password )) seeded_databases: (( property_overrides.mysql.seeded_databases || nil )) database_startup_timeout: (( property_overrides.mysql.database_startup_timeout || 600 )) diff --git a/manifest-generation/examples/property-overrides.yml b/manifest-generation/examples/property-overrides.yml index 4f44266a..c19c6ecc 100644 --- a/manifest-generation/examples/property-overrides.yml +++ b/manifest-generation/examples/property-overrides.yml @@ -5,6 +5,7 @@ property_overrides: host: REPLACE_WITH_LB_HOSTNAME # Optional, set to your Load Balancer address if configured; delete this line otherwise mysql: admin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP #The admin password for the MySQL server process + roadmin_enabled: true roadmin_password: REPLACE_WITH_A_PASSWORD_YOU_MAKE_UP bootstrap_endpoint: username: REPLACE_WITH_A_USERNAME_YOU_MAKE_UP From 09ad20c4ff2a44d9d0270d2d0cf0ae8592b53f7a Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Thu, 4 Feb 2016 10:18:49 -0800 Subject: [PATCH 096/110] Turned on wsrep_load_data_splitting [#112648647] Signed-off-by: Aditya Anchuri --- jobs/mysql/templates/my.cnf.erb | 1 + 1 file changed, 1 insertion(+) diff --git a/jobs/mysql/templates/my.cnf.erb b/jobs/mysql/templates/my.cnf.erb index 8574ba4a..51e23a05 100644 --- a/jobs/mysql/templates/my.cnf.erb +++ b/jobs/mysql/templates/my.cnf.erb @@ -35,6 +35,7 @@ wsrep_sst_method=xtrabackup-v2 wsrep_sst_auth=<%= p('cf_mysql.mysql.admin_username')%>:<%= p('cf_mysql.mysql.admin_password')%> wsrep_max_ws_rows=<%= p('cf_mysql.mysql.wsrep_max_ws_rows') %> wsrep_max_ws_size=<%= p('cf_mysql.mysql.wsrep_max_ws_size') %> +wsrep_load_data_splitting='ON' #wsrep_on='OFF' #wsrep_desync='ON' #wsrep_OSU_method='RSU' From 30deb5a9eaff020c39ef1b6d3f831e17e6fb8610 Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Thu, 4 Feb 2016 10:38:36 -0800 Subject: [PATCH 097/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: ReadOnlyUserEnabled does not need a validate nonzero ( Travis Unknown ) Signed-off-by: Aditya Anchuri --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index c5128fed..54727799 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit c5128fed270ebf3728163c104c7e4563e6156789 +Subproject commit 54727799b9b6b2bc53e3a9a77384fe3a5f7d8771 From d861c30511b2d118ec2bf75c4e817115940aa8c3 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Thu, 4 Feb 2016 10:42:44 -0800 Subject: [PATCH 098/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aaron Hurley: Took out test for Readonlyuserenabled validation ( Travis Unknown ) Signed-off-by: Aaron Hurley --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 54727799..461df262 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 54727799b9b6b2bc53e3a9a77384fe3a5f7d8771 +Subproject commit 461df2629fa4812d58795e67b69b9d351546ad35 From 6e07b33d5cf180a9344cbcf1473136f1e337749d Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Thu, 4 Feb 2016 15:04:47 -0800 Subject: [PATCH 099/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Fixed drop user query ( Travis Unknown ) Signed-off-by: Aditya Anchuri --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 461df262..c8906d66 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 461df2629fa4812d58795e67b69b9d351546ad35 +Subproject commit c8906d6686aceb174bb505a813f348518a50186b From e0359820d4b43edc536e66273ac61f11ccdb9cab Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Fri, 5 Feb 2016 11:11:48 -0800 Subject: [PATCH 100/110] Updated mysql wsrep_node_name - Must have both job name and index to ensure uniqueness across jobs [#113135203] Signed-off-by: Morgan Fine --- jobs/mysql/templates/my.cnf.erb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/mysql/templates/my.cnf.erb b/jobs/mysql/templates/my.cnf.erb index 51e23a05..c3029dec 100644 --- a/jobs/mysql/templates/my.cnf.erb +++ b/jobs/mysql/templates/my.cnf.erb @@ -29,7 +29,7 @@ wsrep_provider=/var/vcap/packages/mariadb/lib/plugin/libgalera_smm.so wsrep_provider_options="gcache.size=<%= p('cf_mysql.mysql.gcache_size') %>M;pc.recovery=TRUE" wsrep_cluster_address="gcomm://<%= cluster_ips.join(",") %>" wsrep_node_address='<%= spec.networks.send(p('network_name')).ip %>' -wsrep_node_name='mysql/<%= index %>' +wsrep_node_name='<%= name %>/<%= index %>' wsrep_cluster_name='cf-mariadb-galera-cluster' wsrep_sst_method=xtrabackup-v2 wsrep_sst_auth=<%= p('cf_mysql.mysql.admin_username')%>:<%= p('cf_mysql.mysql.admin_password')%> From 57d5d62c8008df1b2ac605ccd9929628e5222692 Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Mon, 8 Feb 2016 16:40:17 -0800 Subject: [PATCH 101/110] How did job_overrides get deleted?? Signed-off-by: Aditya Anchuri --- manifest-generation/cf-mysql-template.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/manifest-generation/cf-mysql-template.yml b/manifest-generation/cf-mysql-template.yml index ea302468..040a3ff0 100644 --- a/manifest-generation/cf-mysql-template.yml +++ b/manifest-generation/cf-mysql-template.yml @@ -317,6 +317,7 @@ iaas_settings: (( merge )) instance_count_overrides: (( merge || nil )) property_overrides: (( merge )) release_versions: (( merge || nil )) +job_overrides: (( merge || nil )) default_mysql_cluster_ips: - (( jobs.mysql_z1.networks.mysql1.static_ips.[0] )) - (( jobs.mysql_z2.networks.mysql2.static_ips.[0] || nil )) From 24dd8a5cb56d90cbf4e156a7c26531c93666c107 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Fri, 12 Feb 2016 15:23:50 -0800 Subject: [PATCH 102/110] Bump src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests Bump cloudfoundry-incubator/cf-mysql-acceptance-tests: Morgan Fine: Merge pull request #3 from mdelillo/configurable-broker-protocol Emily Casey: Add config option for broker protocol [#113219017] --- src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests b/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests index e0eece0b..532eca1f 160000 --- a/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests +++ b/src/github.com/cloudfoundry-incubator/cf-mysql-acceptance-tests @@ -1 +1 @@ -Subproject commit e0eece0bb70ca3b2d46faeb5db6bf79a0ad8a8b6 +Subproject commit 532eca1f72cf8683e5def78431047ccb46c36225 From 0acab38af6e6e95106a11678b777cd75d889c60e Mon Sep 17 00:00:00 2001 From: Marco Nicosia Date: Fri, 12 Feb 2016 17:38:45 -0800 Subject: [PATCH 103/110] Formatting changes Make some formatting changes so that bullet items and comments are easily distinguished. [#111778592] --- docs/bootstrapping.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/bootstrapping.md b/docs/bootstrapping.md index 28f41038..693180d7 100644 --- a/docs/bootstrapping.md +++ b/docs/bootstrapping.md @@ -48,11 +48,13 @@ If one or more nodes are not reachable (i.e. the VM exists but in an unknown sta 1. `bosh -n stop mysql_z1 && bosh -n stop mysql_z2 && bosh -n stop _z3` 1. `bosh edit deployment` 1. Set `update.canaries` to 0, `update.max_in_flight` to 3, and `update.serial` to false. -1. `bosh deploy`. Note, if you get a 503 error (like `Sending stop request to monit: Request failed, response: Response{ StatusCode: 503, Status: '503 Service Unavailable' }`), it means that monit is still trying to stop the vms. Please wait a few minutes and try this step again. -1. `bosh -n start mysql_z1 ; bosh -n start mysql_z2 ; bosh -n start _z3` (This will throw several errors, but it ensures that all the jobs are present on the VM) +1. `bosh deploy` + - Note, if you get a 503 error (like `Sending stop request to monit: Request failed, response: Response{ StatusCode: 503, Status: '503 Service Unavailable' }`), it means that monit is still trying to stop the vms. Please wait a few minutes and try this step again. +1. `bosh -n start mysql_z1 ; bosh -n start mysql_z2 ; bosh -n start _z3` + - This will throw several errors, but it ensures that all the jobs are present on the VM. 1. `bosh instances` to verify that all jobs report as failing. 1. Try running the errand again using `bosh -n run errand bootstrap` as above. -1. Once the errand succeeds, the cluster is synced, although some jobs might still report as failing. + - Once the errand succeeds, the cluster is synced, although some jobs might still report as failing. 1. `bosh edit deployment` 1. Set `update.canaries` to 1, `update.max_in_flight` to 1, and `update.serial` to true. 1. Verify that deployment succeeds and all jobs are healthy. A healthy deployment should look like this: From 83111d4917c15b3df6102af161371d3cd984de48 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Tue, 16 Feb 2016 12:59:07 -0800 Subject: [PATCH 104/110] Get a blank value for read only password if not provided [#112020565] Signed-off-by: Aditya Anchuri --- jobs/mysql/templates/mariadb_ctl_config.yml.erb | 4 +++- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/jobs/mysql/templates/mariadb_ctl_config.yml.erb b/jobs/mysql/templates/mariadb_ctl_config.yml.erb index f4a21488..7f606163 100644 --- a/jobs/mysql/templates/mariadb_ctl_config.yml.erb +++ b/jobs/mysql/templates/mariadb_ctl_config.yml.erb @@ -13,7 +13,9 @@ Db: Password: <%= p('cf_mysql.mysql.admin_password')%> ReadOnlyUserEnabled: <%= p('cf_mysql.mysql.roadmin_enabled')%> ReadOnlyUser: roadmin - ReadOnlyPassword: <%= p('cf_mysql.mysql.roadmin_password')%> +<% if_p('cf_mysql.mysql.roadmin_password') do |rpwd| %> + ReadOnlyPassword: <%= rpwd %> +<% end %> PreseededDatabases: <% p("cf_mysql.mysql.seeded_databases").each do |seed| %> - DBName: <%= seed["name"] %> diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index c8906d66..94cdae9b 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit c8906d6686aceb174bb505a813f348518a50186b +Subproject commit 94cdae9b6ed5f6debf0c778ee4e65217489abb89 From 21e6e0ff5936d2f031b2ec7ab78d981e8229af24 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Wed, 17 Feb 2016 14:40:28 -0800 Subject: [PATCH 105/110] Properly enable/disable skip_name_resolve - Any presence of skip_name_resolve in my.cnf will enable this feature, regardless of value; removing the key disables it [#113964723] Signed-off-by: Morgan Fine --- jobs/mysql/templates/my.cnf.erb | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/jobs/mysql/templates/my.cnf.erb b/jobs/mysql/templates/my.cnf.erb index c3029dec..9239a022 100644 --- a/jobs/mysql/templates/my.cnf.erb +++ b/jobs/mysql/templates/my.cnf.erb @@ -55,10 +55,8 @@ log-error = /var/vcap/sys/log/mysql/mysql.err.log init-file = /var/vcap/jobs/mysql/config/mariadb_init skip-external-locking = TRUE max_allowed_packet = 256M -<% if p('cf_mysql.mysql.skip_name_resolve') %> -skip_name_resolve = 1 -<% else %> -skip_name_resolve = 0 +<% if_p('cf_mysql.mysql.skip_name_resolve') do |disable_dns| %> +<%= "skip_name_resolve" if disable_dns %> <% end %> innodb_file_per_table = ON From 5f6544b006c2c6180d1f13dc4b1a1aa5bb914a72 Mon Sep 17 00:00:00 2001 From: Ben Calegari Date: Wed, 17 Feb 2016 17:29:12 -0800 Subject: [PATCH 106/110] Added example plans to cf-mysql-broker spec [#108556658] [ci-skip] --- jobs/cf-mysql-broker/spec | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/jobs/cf-mysql-broker/spec b/jobs/cf-mysql-broker/spec index 20e39da0..75ecfaf0 100644 --- a/jobs/cf-mysql-broker/spec +++ b/jobs/cf-mysql-broker/spec @@ -83,6 +83,18 @@ properties: default: 40 cf_mysql.broker.services: description: 'Services and plans offered by the broker' + example: | + plans: + - name: 100mb + id: r4nd0m_gu1d + description: Shared MySQL Server + max_storage_mb: 10 + max_user_connections: 20 + - name: 1gb + id: an0th3r_r4nd0m_gu1d + description: Shared MySQL Server + max_storage_mb: 20 + max_user_connections: 40 cf_mysql.broker.ssl_enabled: description: 'Determines use of https in dashboard url and in callback uri for calls to UAA' default: true From e554ef83e18d0ea8dc7bea16a2b8cce22b0ab588 Mon Sep 17 00:00:00 2001 From: Aaron Hurley Date: Thu, 18 Feb 2016 15:26:46 -0800 Subject: [PATCH 107/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aditya Anchuri: Fixed config unit test to read from example YAML ( Travis Unknown ) Signed-off-by: Aditya Anchuri --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index 94cdae9b..c3fb81d2 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit 94cdae9b6ed5f6debf0c778ee4e65217489abb89 +Subproject commit c3fb81d2ba9b3d106a270f4e31f3b9ef503f976f From 68d8a5995120f2f8f136bf13e47f38c347fea7d6 Mon Sep 17 00:00:00 2001 From: Aditya Anchuri Date: Thu, 18 Feb 2016 15:45:27 -0800 Subject: [PATCH 108/110] Bump src/github.com/cloudfoundry/mariadb_ctrl Bump cloudfoundry/mariadb_ctrl: Aaron Hurley: Removed dangerous fmt references ( Travis Unknown ) Signed-off-by: Aaron Hurley --- src/github.com/cloudfoundry/mariadb_ctrl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/github.com/cloudfoundry/mariadb_ctrl b/src/github.com/cloudfoundry/mariadb_ctrl index c3fb81d2..562a5001 160000 --- a/src/github.com/cloudfoundry/mariadb_ctrl +++ b/src/github.com/cloudfoundry/mariadb_ctrl @@ -1 +1 @@ -Subproject commit c3fb81d2ba9b3d106a270f4e31f3b9ef503f976f +Subproject commit 562a500116413034ad649387afcde537ff6f86ee From e0ba77268829b63ba374508e31c6e732a9a2e188 Mon Sep 17 00:00:00 2001 From: Marco Nicosia Date: Thu, 18 Feb 2016 18:17:33 -0800 Subject: [PATCH 109/110] Update generate-deployment-manifest to refer to cf-stubs Yesterday, partially based on these instructions, I accidentally gave `generate-deployment-manifest` my actual cf-release manifest. I don't think that's what we want here. In cf-mysql-release we provide a `cf-stub.yml` which an operator should be providing for both (I think) bosh-lite and IaaS-based invocations of `generate-deployment-manifest`? [#114036369] --- docs/arbitrator.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/arbitrator.md b/docs/arbitrator.md index 0569ef11..e9a9c042 100644 --- a/docs/arbitrator.md +++ b/docs/arbitrator.md @@ -13,11 +13,11 @@ For a fresh deployment of CF MySQL v26, simply follow the steps in the [README]( If you already have a 3-node deployment of CF MySQL (e.g. v25), follow the below steps to upgrade to the new configuration as a rolling deploy (the commands below apply to testing on a bosh-lite deployment. For other environments, the same steps are to be followed, except that stubs are different, as explained in the README linked above.). 1. Generate a 3 node + arbitrator manifest: - 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > [manifest export path].yml
+ 
./scripts/generate-deployment-manifest -c /path/to/your-cf-stub.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > [manifest export path].yml
1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` 1. Upon successful deployment, generate the 2 node + arbitrator manifest: - 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml > [manifest export path].yml
+ 
./scripts/generate-deployment-manifest -c /path/to/your-cf-stub.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/remove-mysql-node/instance-count-overrides.yml > [manifest export path].yml
1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` @@ -26,10 +26,10 @@ If you already have a 3-node deployment of CF MySQL (e.g. v25), follow the below If you wish to go back to a 3-node deployment from a 2-node-plus-arbitrator deployment, follow the steps below to perform the downgrade: 1. Generate a 3-node + arbitrator manifest: - 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > [manifest export path].yml
+ 
./scripts/generate-deployment-manifest -c /path/to/your-cf-stub.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/upgrade-to-arbitrator/deploy-arbitrator/instance-count-overrides.yml > [manifest export path].yml
1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` 1. Upon successful deployment, generate the 3 node manifest: - 
./scripts/generate-deployment-manifest -c /tmp/bosh-lite-cf-manifest.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/no-arbitrator/instance-count-overrides.yml > [manifest export path].yml
+ 
./scripts/generate-deployment-manifest -c /path/to/your-cf-stub.yml -p manifest-generation/bosh-lite-stubs/property-overrides.yml -i manifest-generation/bosh-lite-stubs/iaas-settings.yml -n manifest-generation/examples/no-arbitrator/instance-count-overrides.yml > [manifest export path].yml
1. `bosh deployment [manifest export path].yml` 1. `bosh deploy` From 68d2c6b565e54e87fba0c93e183fdce6a8592048 Mon Sep 17 00:00:00 2001 From: Morgan Fine Date: Fri, 19 Feb 2016 08:58:51 -0800 Subject: [PATCH 110/110] Replace l33t guids with less l33t guids - Also matched storage mb with plan name [#108556658] --- jobs/cf-mysql-broker/spec | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/jobs/cf-mysql-broker/spec b/jobs/cf-mysql-broker/spec index 75ecfaf0..ede93ea4 100644 --- a/jobs/cf-mysql-broker/spec +++ b/jobs/cf-mysql-broker/spec @@ -86,14 +86,14 @@ properties: example: | plans: - name: 100mb - id: r4nd0m_gu1d + id: REPLACE-WITH-A-UNIQUE-GUID description: Shared MySQL Server - max_storage_mb: 10 + max_storage_mb: 100 max_user_connections: 20 - name: 1gb - id: an0th3r_r4nd0m_gu1d + id: REPLACE-WITH-A-UNIQUE-GUID description: Shared MySQL Server - max_storage_mb: 20 + max_storage_mb: 1000 max_user_connections: 40 cf_mysql.broker.ssl_enabled: description: 'Determines use of https in dashboard url and in callback uri for calls to UAA'