v36.9.0

Minor updates

The focus of this release is to catch up on a few items that didn't make v36.8.0.

• [PR #163] Disable running tests when packaging galera [#145631809]
• Bug fix: IPsec causes mariadb_ctrl to be left in an Execution Failed state when taking time to start [#152650758]
• Sally does not want to see a new warning from BOSH when running broker-registrar [#151862716]
• Sally would like the Interruptor turned off by default [#152651619]
  • Although we had some troubles last year, we are well past them. The default has long been set to disable automatic SST, which can be inconvenient especially when cf-mysql-release is used by cf-release and cf-deployment. This paranoid option is still available, we're just changing the default to false.

Changes to Manifest

• No changes

As always, please refer to cf-mysql-deployment for deployment instructions, manifest and sample overrides files.

v36.8.0

TLS, continued

We're working on making it possible for applications and service-key users to connect to cf-mysql using TLS. This effort is on-going, please consider TLS support experimental.

• Todd would like to be able to create a service-key such that he is able to connect to services using TLS so that he's able to connect to services from off platform. [#151419865]
• Sally would like to be confident that Binky can cf push new apps that can connect to services using encrypted communications so that sensitive information is not available to malicious actors [#151419862]
• Binky would like Java and Spring apps to connect using TLS when TLS is enabled [#151862576]
• Sally would like the platform to figure out which buildpack to use in the cipher_fin [#152248419]

Dependency Updates and Bug Fixes

• cf-mysql-release shall include MariaDB 10.1.26 so that Sally can benefit from bug fixes and CVE fixes. [#150463375]
• Bug fix: smoke tests do not run if there are no public plans [#150525177]
• Bug fix: pre-start fails if multiple jobs restart rsyslog [#152096419]

Changes to Manifest

• No changes

v36.7.0

Changes since v36

Configurable Table Locking

• Sally would like her specification of table-locking behavior to affect existing bindings. [#150565504]
• Sally would like to specify the behavior of bindings' ability to lock tables [#150565220]

Other Changes

• Upgraded nokogiri to 1.8.1, rubygems to 2.6.13 to address recent CVEs.
• Changed the name of the rsyslog configuration is now 01-mysql.conf to avoid any clash with other jobs which may be co-located on the instance which want to use 00_syslog_forwarder.conf. [#150757746]
• Sally would like to specify additional users that should be excluded from audit logs as a CSV [#148524225]
• Sally would like to run in single-node mode without loading or using any replication software. [#150519574]
• Nascent TLS support. [#151419854]

Documentation

• Use of load balancer via the cloud-config is not documented [#149533621]
• cloudfoundry/cf-mysql-release #171: rejoin-unsafe errand references bootstrapping docs? [#148456401]

Changes to Manifest

• cf.mysql.broker.allow_table_locks
• cf_mysql.mysql.enable_galera - boolean
• cf_mysql.mysql.server_audit_excluded_users_csv
  • Default of audit_excluded_users is still cluster-health-logger, quota-enforcer and galera-healthcheck
• cf_mysql.mysql.tls.server_certificate
• cf_mysql.mysql.tls.server_key

v36

cf-mysql-release v36

The theme for cf-mysql-release v36 is security and tuning!

In a world where MySQL is targeted by ransomware, it's important to keep your data stores are as secure as possible. You need specialized knowledge to deploy MySQL so that it is both secure and optimized for performance. It's especially difficult to go back and update existing servers.

v36 adds fifteen security and performance improvements and chooses sane defaults so that you don't have to be a DBA to run MySQL. Many of MySQL's standard settings are dated, and if left untuned, can spoil performance. Where it makes sense, we've updated the defaults or added a manifest property.

Upgrading to cf-mysql-release v36, and keeping current with stemcells, will help protect your databases from well-known attacks.

Deprecation Delivered

As first mentioned with the v34 release, we've removed spiff templates starting with v36. Please use cf-mysql-deployment v36 with the bosh2 cli to deploy. Let us know how it goes for you! Please file a GitHub issue if you're having troubles!

• Remove all spiff manifest-generation [#140167683]

Dependency Updates

• cf-mysql-release uses MariaDB 10.1.24 [#147901169]
• Upgrade any golang components to v1.8.3 [#145389045]
• Fix CVE-2017-5029 (nokogiri) in cf-mysql-broker [#147061901]

Security and Tuning

For more information, refer to the cluster configuration documentation and read the descriptions in the spec file.

• Set MySQL server to skip symbolic links [#144637795]

• As a developer, I don't want my MySQL client to accidentally send files from the client host [#144637793]

• Set MySQL server variables so that users cannot interact with the MySQL servers' file systems [#144637789]

• Make it optional to not allow the server mysql CLI to keep a history file [#144637787]

• Admin user should not be able to connect from arbitrary host via wildcard [#144637779]

• Audit automatically-created users to ensure that have the minimum privileges necessary [#144637773]

• As an Operator, I'd like smoke-tests to run as a non-privileged user [#145547181]

• cf-mysql-release has query_cache_type set to OFF [#145565343]

• Operator can tune innodb_log_buffer_size, default to 32MB [#145565355]

• Operator can configure innodb_large_prefix [#144634641]

• Operator can configure the size of the table_definition_cache [#145565357]

• Operator can change the table_open_cache [#145565341]

• Operator would like to deploy with innodb_strict_mode set to ON [#145565345]

• Operator can configure innodb_flush_method [#145565333]

• Operator can express buffer pool size to be a % which undersedes the setting that expresses it in megabytes [#145938549]

  Make sure to check this last out. Where previously you had to change the buffer cache property whenever deploying to different sized VMs, you can set a new property to have cf-mysql-release automatically compute how much RAM to use for buffer cache!

Bugfixes and Other Improvements

• Operator can specify a name for a cluster before initial deployment [#144962487]

  • Operator can follow documentation to change the cluster name of an existing deployment [#146552277]

    Use this to give your clusters a unique identifier. Now you can double-check that you're operating on the right deployment when running interactive MySQL.

• When deploying a new cluster, an Operator may need to decrease the timeout used by MySQL to detect if other nodes already exist [#145287747]

• Operator is told when a persistent disk could not be found, or is less than 10GB [#145971959]

• If no syslog configuration is provided we do not attempt to configure syslog [#146544497]

• [BUG] cluster_health log miss several columns sometimes [#140438237]

• [BUG] When shutting down during the wrong BOSH phase, MySQL may fail to leave the cluster gracefully. [#145228885]

• [BUG] Operator should not be provided with an empty set of files when running download-logs [#144945415]

  We significantly refactored mariadb_ctrl, the real brains behind cf-mysql-release.

  • [BUG] mariadb_ctrl PostStartSQLFiles are run without error checking. [#146056211]
  • [BUG] mariadb_ctrl upgrade does not wait for mysqld shutdown [#145530067]
  • [BUG] mariadb_ctrl should wait for the database to be synced before moving on to seeding databases [#146052481]
  • mariadb_ctrl should use the link bootstrap property to determine if it is the bootstrap node #145454113

• Release Integration can more easily consume BOSH links provided by cf-MySQL's jobs [#145350449], [#147457159]

cf-mysql-deployment improvements

• As an Operator, I'd like to find an example operations file to set cf_mysql.host [#139035695]
• Share the links in cf-mysql-deployment [#146118495]

---

Manifest Changes

New Job: smoke-tests-user

• This job should be colocated with MySQL if you would like to create the smoke-tests user.

New Job: cf-mysql-broker-user

• This job should be colocated with MySQL if you would like to create the cf-mysql-broker and quota-enforcer users if the quota enforcer has been enabled.

Proxy Job Spec Changes

• cf_mysql.mysql.cluster_ips, cf_mysql.proxy.proxy_ips, cf_mysql.proxy.arbitrator_ip have been removed and are now only accessible via BOSH links.

MySQL Job Spec Changes

• The mysql-database link has been renamed to internal-mysql-database and its type is now internal-database.
• The following properties have been added:
• cf_mysql.mysql.remote_admin_access
• cf_mysql.mysql.innodb_buffer_pool_size_percent
• cf_mysql.mysql.innodb_log_buffer_size
• cf_mysql.mysql.innodb_flush_method
• cf_mysql.mysql.innodb_large_prefix_enabled
• cf_mysql.mysql.innodb_strict_mode
• cf_mysql.mysql.table_definition_cache_size
• cf_mysql.mysql.table_open_cache
• cf_mysql.mysql.cluster_name
• cf_mysql.mysql.cluster_probe_timeout
• cf_mysql.mysql.cli_history
• cf_mysql.mysql.enable_local_file

Arbitrator Job Spec Changes

• cf_mysql.mysql.cluster_ips has been removed and are now only accessible via BOSH link.

Bootstrap Errand Spec Changes

• cf_mysql.mysql.cluster_ips has been removed and are now only accessible via BOSH link.
  cf-mysql-broker
• The following properties are available through the broker link:
• cf_mysql.broker.db_password
• cf_mysql.broker.disable_quota_enforcer
• cf_mysql.broker.quota_enforcer.password
• cf_mysql.mysql.admin_username, cf_mysql.mysql.admin_password have been removed and are now only accessible via BOSH link.

Rejoin-Unsafe Errand Spec Changes

• cf_mysql.mysql.cluster_ips has been removed and are now only accessible via BOSH link.

Smoke-Tests Errand Spec Changes

• cf_mysql.mysql.admin_username, cf_mysql.mysql.admin_password, cf_mysql.proxy.proxy_ips have been removed and are now only accessible via BOSH link.
• cf_mysql.smoke_tests.db_password will set the password for the user which has been created by the smoke-tests-user job.
  verify-cluster-schemas
• cf_mysql.mysql.cluster_ips, cf_mysql.proxy.arbitrator_ip have been removed and are now only accessible via BOSH link.

---

Feedback Time: Allow table locks?

In v34, we removed Apps' ability to lock tables, as a way of enforcing the fact that Galera doesn't replicate table locks. Perhaps this was too strict?

Galera is often advertised as a way to horizontally scale MySQL, but cf-mysql uses Galera solely to offer high uptime SLAs...

v35

cf-mysql-release v35

Notice: Based on your responses to the v34 poll, this will be the last release in which we include updated spiff templates. That means that starting with v36, you'll want to check out cf-mysql-deployment or continue building your manifest by other means.

Also, using cf-mysql-deployment just got easier with the official GA of the bosh v2 CLI!

Dependency Updates

• cf-mysql-release should use MariaDB 10.1.22 [#141072891]

• Use routing-release instead of our own fork of route-registrar [#135377279]

  For a long time, cf-mysql-release has used a fork of the route-registrar library. To stay current, we now require routing-release. Please note the README.md now covers this prerequisite.

  Note: When switching to the canonical distribution of route-registrar, we had to change the URL of the proxy. Where once you'd visit https://proxy-0-p-mysql.SYSTEM-DOMAIN/, now the instance index is pre-pended: https://0-proxy-p-mysql.SYSTEM-DOMAIN/.

  • As an operator I can navigate to a well-known URL to discover a list of URLs to the proxy dashboards [#138180969]

    If you use cf-mysql-deployment, and include the register-proxy-route.yml operations file, the deployment will automatically include a proxy aggregator: https://proxy-p-mysql.SYSTEM-DOMAIN/ which will include links to the proxy dashboards, regardless of the naming scheme. This feature is only available when using BOSH links.

  • As an Operator, I'd like the docs to cover how to upgrade from v34 to v35 which uses routing-release [#144161787]

  • cloudfoundry/cf-mysql-release #157: Unused nats properties [#143997101]

  Some nats properties are no longer used by our jobs directly, but routing-release still uses the same properties so deployment manifests do not need to change.

Providing and Consuming BOSH links

Using BOSH links simplifies manifests and manifest generation - every time a BOSH release uses a link, that's less copying and pasting via template.

• cloudfoundry/cf-mysql-release #154: Provide database links for mysql and proxy jobs [#143086033]
• cloudfoundry/cf-mysql-release #149: Optionally consume cc link for app_domains in smoke tests [#140458283]

Bug Fixes and Minor Improvements

• [BUG] Upgrading from older releases fails to start mariadb [#139334661]
• [BUG] Logs not draining to syslog [#143069405]
  • Fixes a regression in v34 that logs were not sent to syslog.
• [BUG] mysql_release fails to deploy to a BOSH director using local_dns [#143221877]
• [BUG] pre-start hangs indefinitely when cluster is not healthy [#141824537]
• [BUG] If a node is failing, mariadb_ctrl waits for mysql forever [#140517205]
• [BUG] roadmin user doesn't have read privs for admin operations? [#139594503]
• cloudfoundry/cf-mysql-release #155: Allow innodb_flush_log_at_trx_commit to be configurable [#143693837]
• cloudfoundry/cf-mysql-release #158: Nil the consul link in the example stub [#144138871]
• cloudfoundry/cf-mysql-release #159: add private key option to download-logs script [#144411627]
• As an Operator, if an errand is a no-op, it's better to exit 0 than error [#139054911]
• Stop attaching the tarball to the release notes of cf-mysql-release [#139611695]

Documentation

• Proxy startup and shutdown delay descriptions are backwards in spec [#139996733]
• NOTICE files on various repositories are out of date [#141360157]

Manifest Changes

• New: cf.mysql.innodb_flush_log_at_trx_commit, optional, defaults to 1.
• New: cf_mysql.proxy.api_aggregator_port: optional, defaults to 8082.
• New: cf_mysql.proxy.api_uri, required when deploying the proxy aggregator.
• Removed: cf_mysql.external_host
• Removed: cf_mysql.standalone

v34

cf-mysql-release v34

Hey! We're thinking of dropping spiff manifest templates in favor of using BOSH's native features.

• Is this OK? Click: 💚
• Is disaster? Click: 💔

To see more, check out our work in progress, cf-mysql-deployment!

Important Changes

• Retroactively remove lock permissions from existing service-broker-created users [#132881499]

  One major limitation of Galera is that table-level locks are not replicated. We've found that many applications which rely on table locking never notice this limitation when moving onto cf-mysql. In order to fail fast, starting in v32, new service bindings are explicitly disallowed from locking tables. Starting with v34, all existing service bindings will no longer have the ability to lock tables. Apps that attempt to lock tables will now see an error of the form:

    MariaDB [cf_eedd5768_9c6c_4388_ae0b_dc64f4022bf4]> LOCK TABLES fruit WRITE;
    ERROR 1044 (42000): Access denied for user 'uoY64cqdw6qyMtNl'@'%' to database 'cf_eedd5768_9c6c_4388_ae0b_dc64f4022bf4'
  

• Rename broker-deregistrar to deregister-and-purge-instances [#138006305]

  Too often, Operators have been burned trying out an innocuously-named errand. Renaming the errand, deregister-and-purge-instances makes it clear that this is an errand that should only be run just before bosh delete-deployment.

Dependency Upgrades

cf-mysql relies on a variety of OSS packages. Part of using cf-mysql is that you always get the latest bits!

• cf-mysql-release should use MariaDB 10.1.20 [#133906749]
• Golang 1.7 upgrade [#133294695]
• Bump cf CLI to v6.22.2 or greater [#133940783]
• Upgrade xtrabackup [#117744831]

Tuning and Configuration Improvements

• job spec for previous_admin_username should state that it is optional [#133758449]
• Allow monit startup timeout for mariadb_ctl job to be configurable [#135742055]
  • Remove unused database_startup_timeout property from mysql job spec [#133138729]
• As an App Developer, I'd like an exception if trying to store invalid data in the database [#137836651]
• As a Developer, I don't want DDLs and DMLs to be limited by the number of rows they affect [#137044863]
• [BUG] MySQL jobs fail to start if the mysql port is not 3306 [#136590189]
• [BUG] allow cf-mysql.mysql.binlog_enabled and cf-mysql.mysql.innodb_buffer_pool_instances to be overwritten from property-overrides stub [#133969391]

Switchboard Changes

Similar to CF's HTTP router, Switchboard can now account for the healthchecks of an upstream load balancer.

• Switchboard delays shutdown till external Load Balancer fails over [#137933317]
• Switchboard delays startup for a period of time so that a LB can notice and register that it is alive [#137933307]
• As a CF-MySQL operator, I expect the switchboard proxies to no longer claim a lock in consul when in consul-enabled mode [#135490687]

Bug Fixes

• [BUG] bootstrap errand should use the same property name for galera_healthcheck as other jobs [#136389887]
• [BUG] Interruptor scaretext directs Operator to do the wrong thing [#137205271]
• [BUG] cf-mysql-broker should always stop when asked to stop [#136631833]
• [BUG] mariadb_ctrl fails to start if mysql takes longer than 60 seconds to start [#137485517]
• [BUG] Link error for rejoin-unsafe errand if no arbitrator job [#138841321]
• [BUG] mariadb control should start if there's no prestart marker [#138540989]

Miscellaneous Other Improvements

• Reduce the chance of the mysql stalling due to disk filling up from binary logs. [#133888125]
• broker should log when it fails to run the initial mysql script [#133232467]

BOSH 2.0-ification

As mentioned in the cf-mysql-release v32 release notes, we're gradually improving in our ability to leverage BOSH 2.0 semantics. This work is now being centralized in cf-mysql-deployment.

• Ability to remove broker instances (BOSH 2) [#133894727]
• Ability to enable cert verification (BOSH 2) [#133894695]
• [BUG] Removing broker instances with overrides file should also remove broker related errands [#134320837]
• bosh 2 template should default to not requiring CF [#136082119]
• cloudfoundry/cf-mysql-release #141: simplify bosh template [#136469529]
• As an Operator, I'd like a sample bosh 2.0 override file to help understand how to assign static IPs to the proxy instances [#136539935]
• As an Operator, I'd like instructions that help me translate a spiff generated manifest to a BOSH 2.0 manifest [#136290983]

Documentation Updates

• cloudfoundry/cf-mysql-release #139: Update README.md [#135583739]
• [BUG] slow query log is undocumented [#138053415]
• [BUG] Security Groups documentation mistakenly deleted [#138685491]
• Update cf-stub [#137069195]
  You haven't needed to use our cf-stub.yml file in some time, so we've removed it. Just use a real cf-release deployment manifest.
• Explore if Galera 25.3.19 interferes with bootstrap errand or makes the interruptor irrelevant [#134379769]
  Galera now includes its own version of the Interruptor. It takes care of some additional corner cases (such as boostrapping from the wrong node), but doesn't conflict with our own Interruptor. We allow the to coexist. You can disable our Interruptor if it's too disruptive.

Community Contributions

As an OSS project, we welcome Pull Requests. Sometimes, those pull requests are not directly related to our product direction. So long as they are additive, and don't introduce conflicts, we're happy to include them. Including these features in the release does not imply that we routinely test this functionality - these features are considered, "community contributed." If there are issues with these features, we're happy to consider additional PRs from the community!

• cloudfoundry/cf-mysql-broker #19: Allow any 2.3+ ruby [#136116997]
• cloudfoundry/cf-mysql-release #103: Allow the main mysql host to be specified via config. [#118524849]
• cloudfoundry/cf-mysql-release #137: smoke_tests should specify domain when pushing cf app [#135204167]
• cloudfoundry/cf-mysql-release #108: Default cf_mysql.host to nil [#121247455]
• cloudfoundry/cf-mysql-release #142: enable event scheduler [#137246355]
• cloudfoundry/cf-mysql-release #144: adding properties override in cf-mysql-template.yml [#137528165]
• cloudfoundry/cf-mysql-release #145: Update comments for kill_and_wait() [#138063011]
• cloudfoundry/cf-mysql-release #146: Fix quoting in check_mount() [#138063035]

Manifest Changes

For those who manually update your manifests, please complete the survey above by clicking a heart!

• cf_mysql.mysql.healthcheck_port has been renamed to cf_mysql.mysql.galera_healthcheck_port
• cf_mysql.mysql.galera_healthcheck.endpoint_username, now defaults to "galera-healthcheck"
• cf_mysql.broker.auth_username now has a default of admin
• cf_mysql.mysql.wsrep_max_ws_rows now defaults to 0, rather than 128K
• cf_mysql.mysql.binlog_expire_days defaults to 7 days, not 60
• New: cf_mysql.mysql.advertise_host, optional
• New: cf_mysql.mysql.startup_timeout, replaces cf_mysql.mysql.database_startup_timeout
  Now defaults to 60...

v32

In v32, we continue our efforts to solve issues and polish the release. v32 also continues our efforts to protect users from Galera's limitations. We hope that these changes will make it explicit, upon moving to the service, that this is a clustered MySQL solution, and is not exactly the same a single-node deployments of MySQL/MariaDB.

The anchor change for this release is the addition of the default to disallow attempts to lock tables. This could be considered a breaking change for new applications, so please notify your users. It's important to know, however, that table locks were never guaranteed to succeed, we're just now making it explicit.

Deprecation Notice: This release does not remove the privilege to lock tables from existing bindings, so it won't break any applications that are currently deployed. We plan to retroactively remove this permission in v33. [#131408317]

Features

• service-broker created users should not be able to lock tables [#131223773]

• Preseeded users should not be able to lock tables [#131576399]

  Table-level locks are not distributed by Galera, so LOCK TABLES is not supported. Rather than mysteriously let an app appear to successfully get a lock, cf-mysql v32 will now explicitly reject those attempts.

  Here's how an app would see the interaction on previous versions of cf-mysql:

  MariaDB [cf_b236bd8f_38a4_4ee4_bf72_079fda8a6e47]> LOCK TABLES fruit WRITE ;
  Query OK, 0 rows affected (0.00 sec)
  MariaDB [cf_b236bd8f_38a4_4ee4_bf72_079fda8a6e47]> UNLOCK TABLES ;
  Query OK, 0 rows affected (0.00 sec)
  

  And here's what new applications bound to v32 and later will see:

  MariaDB [cf_eedd5768_9c6c_4388_ae0b_dc64f4022bf4]> LOCK TABLES fruit WRITE;
  ERROR 1044 (42000): Access denied for user 'uoY64cqdw6qyMtNl'@'%' to database 'cf_eedd5768_9c6c_4388_ae0b_dc64f4022bf4'
  MariaDB [cf_eedd5768_9c6c_4388_ae0b_dc64f4022bf4]> UNLOCK TABLES;
  Query OK, 0 rows affected (0.00 sec)
  

Switchboard Changes

• Switchboard should not expose the profiling port by default [#129555175]

• When switchboard is being stopped, the other node should be able to acquire the consul lock quickly. [#130918367]

  When embedded in projects such as Diego, which use consul to choose a proxy to write to, failover is now on the order of one second.

Bug Fixes

• Quota Enforcer should not block other queries from running [#131579997]

• Proxy doesn't have privilege to clear arp cache? [#130781143]

• Arbitrator should be optional [PR 131]

• verify-cluster-schemas errand is now using bosh links [PR 132]

  As we move to BOSH 2.0, a user noted that the verify-cluster-schemas errand didn't take advantage of bosh links properly. Merged, and thank you!

• Actually attach persistent disk in bosh2 manifests [a549b36]

New Documentation

• Add new documentation for bosh2 + gobosh deployment procedure: new-deployment-procedure

Logging Changes

• galera healthcheck logs should go to syslog [#131564453]

Interestings

• Allow override of interrupt_notify_cmd [#132017893]

v31

General Security Improvements

This release focuses on addressing and improving the release to address general
security concerns. Many of the changes improved the logging of components in the
release in which we were logging credentials.

• Upgrade MariaDB to 10.1.18 [#131653751]
• Add the wsrep_debug patch to add additional logging levels for MariaDB 10.1 [#130335561]
• The service broker should not use root credentials to access MySQL [#129985945]
• route-registrar should stop logging NATS password [#130791609]
• cf-mysql-broker should not log credentials [#129474883]

Other Improvements

• Don't prevent trigger creation when the binlog is enabled [#130568959]

  Allows service broker created users to use mysql triggers

• As an Operator, I'd like to specify a path to an executable to be run when my SST is interrupted. [#131763097]

  Allows the operator to configure the execution of a collocated job when the interruptor is triggered

• switchboard should only log useful statements at INFO [#131504989]

Community Involvement

• cloudfoundry/cf-mysql-release #104: Add openstack stub [#118640631]

  • Merged a PR that adds an IaaS override stub for openstack

• cloudfoundry/cf-mysql-release #127: Specify the cf-mysql-broker ip via manifest [#130726653]

• The ./update script should work when checked out to a tag [#130536105]

  Note The update script now lives in ./scripts/update to be consistent with other cloudfoundry releases

Bug Fixes

• galera-healthcheck should respect property to control which user is used to access MySQL [#128922163]

  Previously, the galera-healthcheck process would ignore the manifest property and connect as root

• plan sizes in manifest stubs for bosh-lite are confusing [#129698189]

  The manifest stubs now accurately reflect the actual size of the default plans in bosh-lite

• cf_mysql.mysql.galera_healthcheck.db_password does not exist in standalone example stub file [#131179845]

Manifest Changes

• Add optional cf_mysql.mysql.interrupt_notify_cmd
  • specifies a path to a file to run when the interruptor triggers
• Add cf_mysql.broker.db_password
  • the password for the service broker to connect to the database with
• Add optional property cf_mysql.broker.host
  • ip to be registered with the cf router for the broker; defaults to VM ip

Exploration

We have also taken time to find ways to improve the performance and our understanding
of different components in the cluster

• Explore ways to make the quota enforcer query less prone to blocking the service broker in a cluster with many tables [#131471503]
  • The quota enforcer runs a query that locks the mysql.db table when finding
    violators and reformers. This story investigated why it does this and how we can
    improve it.

v30

Cluster Stability Improvements

This release addresses a limitation in the quota enforcer in which it can interfere in service broker activities.
The quota enforcer runs a query is costly relative to the number of tables in the database.

We've added a property that allows the operator to disable the quota enforcer.

• Configure my deployment to run without a quota enforcer. [#131445093]
  Note If setting cf_mysql.broker.disable_quota_enforcer: true in a rolling deploy, service instances that the quota enforcer revoked INSERT privileges from will be stuck in this state. This can be solved by manually granting the user full permission to their service instances again with: GRANT INSERT, UPDATE, CREATE ON <DB_NAME>.* TO '<USERNAME>'@'%';. The USERNAME can be found in the service-binding via cf env.
• The super user username is configurable again [#130544249]

Manifest Changes

• Re-introduce the cf_mysql.mysql.admin_username
  • defaults to root
• Create non-required cf_mysql.mysql.previous_admin_username
  • If changing the admin_username property, set this property to the previous value of the admin_username. Otherwise, there will be two admins for the cluster.

v29

Security Update

While performing an upgrade, the team discovered a security issue, CVE-2016-6653. This affects cf-mysql releases v27 and v28.

In the case where either has been deployed, and the following three conditions are true:

• Configured to send logs to a syslog service
• Syslog transport is not encrypted
• Audit logging is enabled

... then, cf-mysql will mistakenly send those audit logs to to the syslog service without encryption.

Especially in the case where the query directive has been specified in the cf_mysql.mysql.server_audit_events property, this can transmit all application data in a way that is not protected from network observers.

Furthermore, in this configuration, BOSH will not be able to automatically upgrade, see below.

Bug Fixes

• Do not send the mysql audit logs to syslog [#131120795]
• Detaching the persistent disk failed when both syslog and audit logs were enabled [#131023259]

Upgrading from cf-mysql v27 or v28

• If upgrading from v27 or v28, if both cf_mysql.mysql.server_audit_events property and syslog_aggregator had been configured, you may encounter problems when bosh tries to detach the persistent disk from the MySQL VMs. This will look like the following:

  Started updating job mysql_z1 > mysql_z1/0 (55170f29-1796-48ef-ac48-abb325eec1a8) (canary). Failed: Action Failed get_task: Task 462ff34b-78ed-4d16-5ce9-fd707a45e9f1 result: Migrating persistent disk: Remounting persistent disk as readonly: Unmounting /var/vcap/store: Running command: 'umount /var/vcap/store', stdout: '', stderr: 'umount: /var/vcap/store: device is busy.

  (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1))

The problem can be resolved by:

1. Ssh onto the MySQL VMs, using your preferred method
2. Comment out lines 44-48 of /etc/rsyslog.d/00-syslog_forwarder.conf
3. kill the rsyslogd process
4. Run bosh deploy again; it should succeed this time

Features

In typical agile fashion, we had completed a few feature stories, so they're included as well.

• galera_healthcheck should log when it encounters a bad state or error discovering state
  [#128880727]
  • The galera_healthcheck job now logs more verbosely when it encounters problems.
• switchboard proxy should provide an HTTP healthcheck
  [#130696613]
  • This allows the cluster to work with Load Balancers that use only HTTP health checks.
  • The health check port should continue to work with load balancers that use TCP health checks.