config override documentation

README.md

@@ -1,6 +1,4 @@
-[![Build Status](https://travis-ci.org/cloudfoundry/ibm-websphere-liberty-buildpack.svg?branch=master)](https://travis-ci.org/cloudfoundry/ibm-websphere-liberty-buildpack)
-
-# IBM WebSphere Application Server Liberty Buildpack
+# IBM WebSphere Application Server Liberty Buildpack [![Build Status](https://travis-ci.org/cloudfoundry/ibm-websphere-liberty-buildpack.svg?branch=master)](https://travis-ci.org/cloudfoundry/ibm-websphere-liberty-buildpack)
 
 The `liberty-buildpack` is a [Cloud Foundry][] Buildpack for running applications on IBM's WebSphere Application Server [Liberty Profile][].
 
@@ -48,6 +46,7 @@ All Documentation is available in the Docs folder of the buildpack.
     * [Open JDK](docs/open-jdk.md)
 * [Server Behavior xml Options](docs/server-xml-options.md)
 * [Forking the buildpack](docs/forking.md)
+* [Overriding buildpack configuration](docs/configuration.md)
 * [Setting Environment Variables](docs/env.md)
 * [Installation (admin buildpack into CF)](docs/installation.md)
 * [Tuning](docs/tuning.md)
@@ -93,6 +92,6 @@ bundle install --gemfile Gemfile.rubymine-debug
 [tuning options]: docs/tuning.md
 [java main push]: docs/java-main.md
 
-[Liberty-License]: http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/downloads/wlp/8.5.5.3/lafiles/runtime//en.html
+[Liberty-License]: http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/downloads/wlp/8.5.5.5/lafiles/runtime//en.html
 [JVM-License]: http://www14.software.ibm.com/cgi-bin/weblap/lap.pl?la_formnum=&li_formnum=L-EWOD-99YA4J&title=IBM%C2%AE+SDK%2C+Java+Technology+Edition%2C+Version+7+Release+1&l=en
 [manifest documentation]: http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html

docs/configuration.md

@@ -0,0 +1,28 @@
+Configuration
+=============
+
+Buildpack configuration can be overridden with an environment variable matching the [configuration file](../config) you wish to override minus the `.yml` extension and with a prefix of `JBP_CONFIG`. The value of the variable should be valid inline yaml. For example:
+
+1. Configure the buildpack to use [Liberty profile](container-liberty.md) beta:
+
+    ```bash
+    cf set-env myApplication JBP_CONFIG_LIBERTY '[version: 2015.+]'
+    ```
+
+1. Configure the buildpack to use [OpenJDK](open-jdk.md) 8:
+
+   ```bash
+   cf set-env myApplication JBP_CONFIG_OPENJDK '[version: 1.8.+]'
+   cf set-env myApplication JVM openjdk
+   ```
+
+1. Disable [Spring Auto Reconfiguration](framework-spring-auto-reconfiguration.md):
+
+   ```bash
+   cf set-env myApplication JBP_CONFIG_SPRINGAUTORECONFIGURATION '[enabled: false]'
+   ```
+
+The environment variables can also be specified in the applications `manifest.yml` file. See the [Environment Variables][] documentation for more information.
+
+[Environment Variables]: http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#env-block
+

docs/ibm-jdk.md

@@ -1,5 +1,5 @@
 # IBM JDK
-The IBM JDK is the default used with Liberty profile. You do not need to change the *manifest.yml* file or configure anything else to use the IBM JDK. The behavior is the same as setting the `JVM` environment variable to `ibmjdk`. For example, using the *manifest.yml* file:
+The IBM JDK is the default used with Liberty profile. You do not need to change the `manifest.yml` file or configure anything else to use the IBM JDK. The behavior is the same as setting the `JVM` environment variable to `ibmjdk`. For example, using the `manifest.yml` file:
 
 ```bash
 ---
@@ -9,3 +9,16 @@ env:
 
 If you would prefer to use the Open JDK then please read [Open JDK](open-jdk.md).
 
+## Configuration
+
+The JRE can be configured by modifying the [`config/ibmjdk.yml`][] file in the buildpack fork or by passing [an environment variable](configuration.md) that overrides configuration in the yml file. 
+
+| Name | Description
+| ---- | -----------
+| `repository_root` | The URL of the IBM JRE repository index ([details][repositories]).
+| `version` | The version of Java runtime to use.  Candidate versions can be found [here][index.yml].
+
+[`config/ibmjdk.yml`]: ../config/ibmjdk.yml
+[index.yml]: http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/downloads/jre/index.yml
+[repositories]: util-repositories.md
+

docs/open-jdk.md

@@ -1,5 +1,5 @@
 # Open JDK
-The Open JDK provides Java runtimes from the [OpenJDK][] project. The IBM JDK is the default used with Liberty profile. If you would prefer to use the Open JDK then set the `JVM` environment variable to `openjdk`. For example, add the following to your *manifest.yml* file:
+The Open JDK provides Java runtimes from the [OpenJDK][] project. Versions of Java from the `1.6`, `1.7`, and `1.8` lines are available. The IBM JDK is the default used with Liberty profile. However, if you would prefer to use the Open JDK then set the `JVM` environment variable to `openjdk`. For example, add the following to your *manifest.yml* file:
 
 ```bash
 ---
@@ -9,5 +9,69 @@ env:
 
 If you are interested in using the IBM JDK please read [IBM JDK](ibm-jdk.md).
 
+## Configuration
+
+The JRE can be configured by modifying the [`config/openjdk.yml`][] file in the buildpack fork or by passing [an environment variable](configuration.md) that overrides configuration in the yml file. 
+
+| Name | Description
+| ---- | -----------
+| `memory_sizes` | Optional memory sizes, described below under "Memory Sizes".
+| `memory_heuristics` | Default memory size weightings, described below under "Memory Weightings.
+| `repository_root` | The URL of the OpenJDK repository index ([details][repositories]).
+| `version` | The version of Java runtime to use.  Candidate versions can be found in the listings for [centos6][], [lucid][], [mountainlion][], and [precise][]. Note: version 1.8.0 and higher require the `memory_sizes` and `memory_heuristics` mappings to specify `metaspace` rather than `permgen`.
+
+### Memory
+The total available memory is specified when an application is pushed as part of it's configuration. The Java buildpack uses this value to control the JRE's use of various regions of memory. The JRE memory settings can be influenced by configuring the `memory_sizes` and/or `memory_heuristics` mappings.
+
+Note: if the total available memory is scaled up or down, the Java buildpack does not re-calculate the JRE memory settings until the next time the appication is pushed.
+
+#### Memory Sizes
+The following optional properties may be specified in the `memory_sizes` mapping.
+
+| Name | Description
+| ---- | -----------
+| `heap` | The maximum heap size to use. It may be a single value such as `64m` or a range of acceptable values such as `128m..256m`. It is used to calculate the value of the Java command line options `-Xmx` and `-Xms`.
+| `metaspace` | The maximum Metaspace size to use. It is applicable to versions of OpenJDK from 1.8 onwards. It may be a single value such as `64m` or a range of acceptable values such as `128m..256m`. It is used to calculate the value of the Java command line options `-XX:MaxMetaspaceSize=` and `-XX:MetaspaceSize=`.
+| `native` | The amount of memory to reserve for native memory allocation. It should normally be omitted or specified as a range with no upper bound such as `100m..`. It does not correspond to a switch on the Java command line.
+| `permgen` | The maximum PermGen size to use. It is applicable to versions of OpenJDK earlier than 1.8. It may be a single value such as `64m` or a range of acceptable values such as `128m..256m`. It is used to calculate the value of the Java command line options `-XX:MaxPermSize=` and `-XX:PermSize=`.
+| `stack` | The stack size to use. It may be a single value such as `2m` or a range of acceptable values such as `2m..4m`. It is used to calculate the value of the Java command line option `-Xss`.
+
+Memory sizes together with _memory weightings_ (described in the next section) are used to calculate the amount of memory for each memory type. The calculation is described later.
+
+Memory sizes consist of a non-negative integer followed by a unit (`k` for kilobytes, `m` for megabytes, `g` for gigabytes; the case is not significant). Only the memory size `0` may be specified without a unit.
+
+The above memory size properties may be omitted, specified as a single value, or specified as a range. Ranges use the syntax `<lower bound>..<upper bound>`, although either bound may be omitted in which case the defaults of zero and the total available memory are used for the lower bound and upper bound, respectively. Examples of ranges are `100m..200m` (any value between 100 and 200 megabytes, inclusive) and `100m..` (any value greater than or equal to 100 megabytes).
+
+Each form of memory size is equivalent to a range. Omitting a memory size is equivalent to specifying the range `0..`. Specifying a single value is equivalent to specifying the range with that value as both the lower and upper bound, for example `128m` is equivalent to the range `128m..128m`.
+
+#### Memory Weightings
+Memory weightings are configured in the `memory_heuristics` mapping of [`config/openjdk.yml`][]. Each weighting is a non-negative number and represents a proportion of the total available memory (represented by the sum of all the weightings). For example, the following weightings:
+
+```yaml
+memory_heuristics:
+  heap: 15
+  permgen: 5
+  stack: 1
+  native: 2
+```
+
+represent a maximum heap size three times as large as the maximum PermGen size, and so on.
+
+Memory weightings are used together with memory ranges to calculate the amount of memory for each memory type, as follows.
+
+#### Memory Calculation
+The total available memory is allocated into heap, Metaspace or PermGen (depending on the version of OpenJDK), stack, and native memory types.
+
+The total available memory is allocated to each memory type in proportion to its weighting. If the resultant size of a memory type lies outside its range, the size is constrained to
+the range, the constrained size is excluded from the remaining memory, and no further calculation is required for the memory type. If the resultant size of a memory size lies within its range, the size is included in the remaining memory. The remaining memory is then allocated to the remaining memory types in a similar fashion. Allocation terminates when none of the sizes of the remaining memory types is constrained by the corresponding range.
+
+Termination is guaranteed since there is a finite number of memory types and in each iteration either none of the remaining memory sizes is constrained by the corresponding range and allocation terminates or at least one memory size is constrained by the corresponding range and is omitted from the next iteration.
+
+[`config/openjdk.yml`]: ../config/openjdk.yml
+[centos6]: http://download.pivotal.io.s3.amazonaws.com/openjdk/centos6/x86_64/index.yml
+[lucid]: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/index.yml
+[mountainlion]: http://download.pivotal.io.s3.amazonaws.com/openjdk/mountainlion/x86_64/index.yml
 [OpenJDK]: http://openjdk.java.net
+[precise]: http://download.pivotal.io.s3.amazonaws.com/openjdk/precise/x86_64/index.yml
+[repositories]: util-repositories.md
 

lib/liberty_buildpack/buildpack.rb

@@ -127,7 +127,7 @@ def initialize(app_dir)
       @buildpack_version = BuildpackVersion.new
       Buildpack.log_debug_data @logger
       Buildpack.require_component_files
-      components = Buildpack.components @logger
+      components = LibertyBuildpack::Util::ConfigurationUtils.load('components')
 
       @lib_directory = Buildpack.lib_directory app_dir
       @common_paths = LibertyBuildpack::Container::CommonPaths.new
@@ -157,24 +157,16 @@ def self.component_detections(components)
       compacted_tags.select { |tag| tag != '' }
     end
 
-    def self.components(logger)
-      Util::ConfigurationUtils.load('components')
-    end
-
-    def self.configuration(app_dir, type, logger)
+    def self.configure_context(basic_context, type)
       component_id = type.match(/^(?:.*::)?(.*)$/)[1].downcase
-      Util::ConfigurationUtils.load(component_id)
-    end
-
-    def self.configure_context(basic_context, type, logger)
       configured_context = basic_context.clone
-      configured_context[:configuration] = Buildpack.configuration(configured_context[:app_dir], type, logger)
+      configured_context[:configuration] = LibertyBuildpack::Util::ConfigurationUtils.load(component_id)
       configured_context
     end
 
-    def self.construct_components(components, type, basic_context, logger)
+    def self.construct_components(components, type, basic_context)
       components[type].map do |component|
-        component.constantize.new(Buildpack.configure_context(basic_context, component, logger))
+        component.constantize.new(Buildpack.configure_context(basic_context, component))
       end
     end
 
@@ -267,12 +259,12 @@ def initialize_components(components, basic_context)
       raise "No components of type #{JRE_TYPE} defined in components configuration.  At least one must be defined" if components[JRE_TYPE].nil?
 
       # finds the first jre component and its version that doesn't return false.  Just need one jre component.
-      jres = Buildpack.construct_components(components, JRE_TYPE, basic_context, @logger)
+      jres = Buildpack.construct_components(components, JRE_TYPE, basic_context)
       @jre = jres.find { |jre| @jre_version = jre.detect }
       @logger.error("JRE component did not detect a valid version. It's possible that the JVM environment variable needs to be set or its value needs to be corrected.") if @jre.nil?
 
-      @frameworks = Buildpack.construct_components(components, FRAMEWORK_TYPE, basic_context, @logger)
-      @containers = Buildpack.construct_components(components, CONTAINER_TYPE, basic_context, @logger)
+      @frameworks = Buildpack.construct_components(components, FRAMEWORK_TYPE, basic_context)
+      @containers = Buildpack.construct_components(components, CONTAINER_TYPE, basic_context)
     end
 
     def self.initialize_env(dir)

spec/liberty_buildpack/buildpack_spec.rb

@@ -73,7 +73,7 @@ module LibertyBuildpack
 
     it 'should not write VCAP_SERVICES credentials as debug info',
        log_level: 'DEBUG' do
-      ENV['VCAP_SERVICES'] = "{\"type\":\n[\n{\"credentials\":\n\"VERY SECRET PHRASE\", \"plain\":\n\"PLAIN DATA\"}]}"
+      ENV['VCAP_SERVICES'] = '{"type":[{"credentials":"VERY SECRET PHRASE","plain":"PLAIN DATA"}]}'
       log_content = with_buildpack do |buildpack|
         app_dir = File.dirname buildpack.instance_variable_get(:@lib_directory)
         File.read LibertyBuildpack::Diagnostics.get_buildpack_log app_dir