NEWS

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# OLFS Version 1.18.15 (15 January 2025)
#

* Updated logo.
* Added logback appender for AWS CloudWatch.
* Updated landing page on NGAP.

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# OLFS Version 1.18.14 (23 January 2024)
#

* OLFS now uses Java17 (jdk17)
* Changed maxResponseSize type from int to long.
* Units scalar set to 2^10 (kilobytes) when the
    units value is junk.
* preformatted text for 400 errors
* Reduced beslistener pool from 200 to 25 for NGAP deployments
* Fixes EDL token handling so that redirected are not issued
    and session (aka cookie) management is not required of the client.

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# OLFS Version 1.18.13 (25 July 2022)
#

* Setting BES timeOut to 55 seconds (thanks CloudFront) for the NGAP deployment
* Added Attributes with DAP4 types to the "No NetCDF-3 Downloads For You"
  feature.
* Updated ncWMS to version 2.5.2
* Migrated to Java-11
* Retiring various XSLT stuff
* Fixed dap4Contents.xsl so that the viewers link works.
* Migrated the schema.org annotations from the node_contents.xsl to
  dap4Contents.xsl
* Rewrote schema.org links to https.
* Removed commented out code.
* Fixed page formatting with new css type

# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# OLFS Version 1.18.12 (09 July 2022)
#

Nothing to see here


# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# OLFS Version 1.18.11 (08 July 2022)
#

## Configuration and behavior updates.

### Added code to detect client protcol change (HYRAX-141).
Updated ReqInfo.getRequestUrlPath() so that it utilizes the request headers:
* - "CloudFront-Forwarded-Proto"
* - "X-Forwarded-Proto"
* - "X-Forwarded-Port"
In reconstructing the "request url" dereferenced by the initiating client. This
means the if some upstream entity rewrites the URL (Like CloudFront does when
it drops the https protocol in favor of http for internal access) it can be
detected and the links built by the server and returned to the client are now
correct.

### Patched bug where unexpected Authentication headers would trigger a redirect
loop.

### Fixed broken service for delivering flat (not-data) files to clients.

## Made the "Get As NetCDF-3" an "Get As DAP2 Binary" buttons on the DAP4 Data
Request form context sensitive. If the dataset in question contains variables
whose data types are found in DAP4 and not in DAP2/NetCDF-3 then the buttons are
disabled. A more complete solution is envisioned where the projected variables
are assessed and if only DAP2/NetCDF-3 types are selected then the buttons would
would be enabled. This fix is only a step in that more dynamic direction.


# -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
# OLFS Version 1.18.10 (04 January 2022)
#

## Configuration and behavior updates.
In order to make the server's behavior more understandable to users, and its configuration
more understandable to admins we have changed the way that the server responds to client
requests for the unadorned Dataset URL and the way that the server generates Data Request
Form links in its catalog pages. There are new configuration parameters to control these
behaviors.

### <UseDAP2ResourceUrlResponse />
This configuration parameter has been DEPRECATED - Use the <DatasetUrlResponse /> and
<DataRequestForm /> elements to configure this behavior (see below).

### <DatasetUrlResponse type="..." />
The DatasetUrlResponse element is used to configure the type of response
that the server will generate when a client attempts to access the Dataset URL.
The type of response is controlled by the value of the type attribute.

Allowed Values:
* dsr - The dap4 DSR response will be returned for the dataset URL. Note: This
setting is not compatible with DataRequestForm type of "dap2" as the DSR response
URL collides with the DAP2 Data Request Form URL.
* download - If the configuration parameter AllowDirectDataSourceAccess is set
(present) thenthe source data file will be returned for the dataset URL. If the
configuration parameter AllowDirectDataSourceAccess is not present then a 403
forbidden will be returned for the dataset URL. (This is basically a file retrieval
service, any constraint expression submitted with the unadorned dataset URL will be
ignored.)
* requestForm - The Hyrax Data Request Form Page will be returned for the dataset URL.

Default: download
<DatasetUrlResponse type="download"/>

### <DataRequestForm type="..." />
The DataRequestForm element defines the target DAP data model for the dataset links
in the "blue-bar" catalog.html pages. These links point to the DAP Data Request Form
for each dataset. This element also determines the type  of Data request form page
returned when the DatasetUrlResponse type="requestForm" and the request is for the
Dataset URL. Allowed type values are: dap2 and dap4.

Default: dap4
<DataRequestForm type="dap4" />

### <AllowDirectDataSourceAccess />
When enabled users will be able to use Hyrax as a file server and download
the underlying data files/granules/objects directly, without utilizing the DAP APIs.

Default: disabled.
<!--AllowDirectDataSourceAccess / -->

### <ForceDataRequestFormLinkToHttps />
The presence of this element will cause the Data Request Form interfaces to
"force" the dataset URL to HTTPS. This is useful for situations where the sever
is sitting behind a connection management tool (like AWS CloudFront) whose outward
facing connections are HTTPS but Hyrax is not using HTTPS. Thus the internal URLs
being received by Hyrax are on HTTP. When these URLs are exposed via the Data
Request Forms they can cause some client's to have issues with session dropping
because the protocols are not consistent.

Default: disabled
<!-- ForceDataRequestFormLinkToHttps / -->

## Dependency Library Updates
* Updated the dependency libraries as follows:
  - Upgraded gson-2.3.1 to gson-2.8.9
  - Upgraded slf4j-1.7.16 to slf4j-1.7.32
  - Upgraded logback-core-1.1.11 to logback-core-1.2.9
  - Upgraded logback-classic-1.2.0 to logback-classic-1.2.9

## Log Sanitization
* Log entries for User-Agent, URL path, and query string are now scrubbed.

## DAP2 Data Request Form
* Dropped link to in DAP2 Data Request Form to the no longer supported BES
generated request form.

# OLFS Version 1.18.9 (15 September 2021)

## NASA/NGAP Updates
* Fixed gradle dependancies according snyk scan.
* Session serialization 
* Added session manager jars to ngap resources.
* Updated production rules to include session manager code in the ngap application build.
* Made history_json output as JSONArray.
* New landing page for NGAP service.
* Added UNIX time to OLFS logs in NGAP.

## Bug Fixes
* Fixed bug in computation of dimension sizes in the dap4 ifh xsl.
* Dropped DSR response for just the dataset_url.
* Fixed broken json error output.
* Changed the broken naming pattern for the PPT strings.
* Enabled ChunkedInputStream debugging and cleaned up the messages in support of stream pollution problem.
* Fixed broken css and deployment context links construction in DSR HTML page.

## Performance Improvements
* Added file size and last modified headers to flat file transfer.
* Added auth.log independant of debug log.
* First pass at patching POST request handling.

# OLFS Version 1.18.8 (30 December 2020)

## NASA/NGAP Updates
* Changes to our packaging of the ngap-snapshot docker images.
* Added support for EDL token chaining (in concert with BES modifications)

## Performance Improvements
* Improved BES connection lifecycle reliability improvements.

## Bug Fixes
* Improved the servers response for HEAD requests, now the server correctly
  returns 405 Method Not Allowed
* Fixed bug in request routing for DAP4 requests.
* Fixed the JSON-LD content in the Data Request Form so that the required 
  description property is generated from the data set using a heuristic, or 
  if that fails a generic string is used. The length of this mandatory 
  property is now correctly limited to [50 < length < 5000]

## Technical Debt
* Started developing Gradle production rules.

# OLFS Version 1.18.7 (27 April 2020)

## Features
* Added an ngap service endpoint in support of the NASA NGAP project.

## Bugs Fixed
* Improved content encoding/scrubbing on outbound error messages.
* Fixed bugs in bootstrap init and init for all the various init() components.
* Stabilized the order of output in WCS-2.0


# OLFS Version 1.18.6 (10 February 2020)

## Features

* Added (alpha) support support for server authentication credentials
  * Environment injection support for besstandalone and besd
  * A credentials configuration file for multiple sets, in which credentials are
  associated with URL prefixes and multiple credentials can be defined in a
  bes.conf environment.

* Combined Logs: The server can now be configured to add the OLFS request log content to the BES log.

* Earthdata Login User Authentication Support

* The Data Request Form requires data selection is now a configuration parameter,
    &lt;RequireUserSelection /&gt;  in the olfs.xml file.

* NoDynamicNavigation - Added a trial configuration parameter &lt;NoDynamicNavigation/&gt;
	to the olfs.xml file. If this element is uncommented in the olfs.xml file then
	all of the dynamically generated catalog/navigation pages will be disabled.
	When enabled the server admin must either supply and maintain THREDDS
	catalogs to be read by the server, or they have to provide their
	own system of navigation and discovery to generate links to the
	dataset endpoints (ex. Data Request Form)

## Performance Improvements
* No DAS construction for data response.

## Bugs Fixed
* Memory leak fixed and unneeded copy removed in awsv4 code
* CovJson patch.
* ncml_handler: When an NCML aggregation named a dataset that the server could not read... crsh. Fixed..
* Server cache stabilization
* hdf4_handler: Fixed memory leaks
* hdf5_handler: Fixed memory leaks

# OLFS Version 1.18.5 (08 July 2019)

## DMR++ Aggregations

## Bug Fixes

OPeNDAP Data Request Forms and catalgo pages.
 - Corrected JSON-LD encoding in the IFH generated pages.
 - Corrected javascript code generation problems.

Path construction bug (causing double "/" characters) in THREDDS catalog
presentation pages. (HK-338)

Security analysis and response.
 - Sonar Scan
 - Coverity Scan

# OLFS Version 1.18.4 (25 March 2019)

## Features

- Added in mempory LRU cache for BES showNode responses. This is a performance
  enhancement. (HK-366)

- Added a site map cache file so that site maps for large instances of 
  Hyrax(>500k datasets) can be navigated in a few seconds and not hours. 
  (HK-375)

- Updated Docker containers so that Tomcat is producing compressed responses 
  when the requesting client indicates that this is acceptable. (HK-329)

- Added response size logging to all metadata and data responses. Error 
  responses do not have ssociated response size values in the log.
  
## Bug Fixes

- Added missing code to support all DAP2/4 atomic numeric types in WCS-2.0 
  (HK-372)


# OLFS Version 1.18.3 (25 February 2019)

## Features:

- Added a context/request sensitive message to all support email links.

- Added support email links to directory/collection pages in addition to error
  pages.
  
- Decoupled support email configuration from site/server administrator 
  information.

- Added sitemap generation and an (optional) ROOT.war service that adds a 
  robots.txt service for said sitemap.

- Enable THREDDS catalog brokering for remote catalogs.

## Bug Fixes

- Corrected the CovJson response Content-Type headers

- Repaired incompatibility between IFH Request forms and CentOS-6

- Fixed odd port problem with redirect code.

- Repaired Viewers pages to correctly process encoded URL content.


# OLFS Version 1.18.2 (10 December 2018)

## Features:

- Added new (beta) dynamic siteMap and robots.txt service.

- Added confguration (to BES) for Site admin contact information that
  also gets used in the JSON-LD DataCatalog content. 

# OLFS Version 1.18.1 (21 November 2018)

- Minor changes mostly a release to support changes in the BES.

# OLFS Version 1.18.0 (26 September 2018)

## Features:

- NEW! Improved! The OPeNDAP Data Request Form!

- CovJson responses for compatible datasets.

- Simplified the OLFS configuration file, olfs.xml (If you are upgrading 
  you will need to replace your existing file with the new default one and 
  edit to recover your changes.)

- JSON-LD now in every contents.html page and in every Data Request Form.

- Hyrax now supports the browsing of remote THREDDS catalogs. This feature
  is disabled by default, and when enabled will only be allowed to access 
  remote systems based on the Gateway.Whitelist in the BES.

- Added optional login controls and framework for adding URS (or other) 
  authentication and identification providers.

- Switched to the OWASP string encoder/decoder.

- Added an additional default configuration location '/use/share/olfs'

## Bug Fixes:

- Added URI encoding to requests generated by the Data Access Forms so that
  special characters such as '[' and ']' are now encoded correctly.

- Applied a work-around patch to the THREDDS catalogs generated by Hyrax  
  to accomodate a java-netcdf problem in which no timezone abbreviations
  other than Z (or none at all) are allowed. The patch maps known GMT
  equivalents to Z and all other timzones are simpley dropped (which is
  also an allowed representation by java-netcdf)

- Repaired Gateway Service UI so that both DAP2 and DAP4 response are 
  represented and functioning.

- Updated URS client implementation to work with current code base.

# OLFS Version 1.17.0  (16 October 2017)

- Added WCS service.
  - Works using templates associated by regex.

- Made the OLFS authentication aware in its http
  activities by having it utilize the ~/.netrc credentials 
  source or one of the same format specified in the configuration. 

- Fixed bug in the generation of the "viewers" pages.

# OLFS Version 1.16.4  (09 August 2017)

- Repaired broken RDF responses.

- Updated dependecy Jars.

- Enabled ThreddsCatalogUtil to authenticate against Earthdata Login

- Merged eo-wcs features into master

# OLFS Version 1.16.3 (05 Jun 2017)

- Server timeout conditions now return HTTP status 504 (and not 418).

- Bug fixes in the DAP4 Data Request Form pages.

# OLFS Version 1.16.2 (10 Jan 2017)

- Improved logging, including log rotation.

- Fixed a null pointer exception in the AggregationServlet (the
  component that performs user-specified aggregations).

- Repaired viewers.xml configuration for Godiva3

- Issues with the netcdf file response History attribute were fixed,
  once again...

- Prompted by user requests we have changed the HTTP status returned when 
  the server encounters a BES timeout error from 418 (I'm a teapot) to 504 
  (Gateway Timeout) We really liked the teapot because we felt applying the 
  504 (Gateway Timeout) status to this error strains the formal definition 
  of the 504 status code and could lead to confusion down the road. 
  Time will tell...


# OLFS Version 1.16.1  (02 Nov 2016)

- Updated OLFS configuration defaults in viewers.xml file to reflect
  changes for ncWMS2 and Godiva3
    
- Patched a problem with the way the XSL transforms are carried out
  that could have left the process running in an unexpected directory.
      

# OLFS Version 1.16.0  (06 June 2016)

 - Added code to ammend the CF "history" attribute so that
    fileout_netcdf responses contain the ammended value that shows
    that Hyrax produced it along withg the constraint and id
    information of the dataset.

 - Added new rules for the Content-Disposition header download file
    names. Now the old behavior (add the netcdf file suffix to very
    file's basename) can be replaced so that if the suffix will end up
    as a duplicate (for example file.nc.nc or file.nc4.nc4), nothing
    is appended.

 - Added memory cache for BES Catalog responses. This allows the OLFS
    to stop asking the same things over and over again. The size and
    refresh interval are specified by the "<CatalogCache> element in
    the olfs.xml file. If the element is missing then the memory cache
    will be disabled.

 - Increased the number of simultaneous requests from 10 to 200 ,
    which is the default limit for Tomcat.

 -  Bug fixes
    - Large file issues in w10n.
    - Fixed build for WCS-2.0


# OLFS Version 1.15.0  (08 February 2016)

 -  The OLFS regression test harness has been migrated from DejaGnu to AutoTest. This makes writing and maintaining
    tests much easier. All of the DejaGnu tests have been migrated and many additional tests added. To configure the
    tests:
        autoconf -vif
        ./configure
    To run the tests:
        make check

 -  Error handling has been refactored. Error messages from libdap and the BES are now correctly transmitted to the
    user. Additionally the HTTP status codes and associated pages are now working correctly and there should be a
    significant reduction in the number of "superman" (Internal Error) pages encountered in normal operation.

 -  The default size of the BES client pool has been increased from 10 to 200 in response to numerous performance
    evaluators that failed to recognize that this could be done to increase performance in high traffic environments.

 -  The OLFS now keeps the BES on a timeout leash so that BES requests that take longer than the time out interval
    stated in the olfs.xml will actually timeout and return an associated error to the OLFS.

 -  !!!!! The location of the OLFS configuration directory has changed !!!!
    This change should resolve installation problems with Tomcat instances that are installed from RPM.
    By default the OLFS will utilize it's bundled configuration in the directory
      $CATALINA_HOME/webapps/opendap/WEB-INF/conf
    In order to localize the configuration so that your configuration changes are persistent you will need to do
    one of the following:
      * For the user that will be running the OLFS (the Tomcat user), set the environment variable OLFS_CONFIG_DIR to
        an existing directory to which the Tomcat user has both read and write privileges.
      OR
      * Create the directory /etc/olfs and set it's permissions/ownership so that the Tomcat user has both read and
        write permission.
    (If both of these steps are done then priority is given to the environment variable.)
    Restart Tomcat. When it starts the OLFS will check these locations and then install a copy of its default
    configuration into the new spot.
    Edit the configuration files as needed.
    If, for example, your beslistener is not running on localhost:10022 then you'll need to edit the olfs.xml
    file in the configuration directory and adjust the <host> and <port>  values to reflect your situation.


# OLFS Version 1.14.1 (28 October 2015)

  Worked under the hood to ensure that Hyrax's character encoding is actually UTF-8.

# OLFS Version 1.14.0 (21 September 2015)

  Rollback from Java 8 requirement. Now only Java 7 is needed for the OLFS and Hyrax.
  Many bug fixes.
  Removal of unused parts of the API.


# OLFS Version 1.13.2 (07 May 2015)

  New patched bugs in datasetScan implementation.


# OLFS Version 1.13.1 (24 April 2015)

  New datasetScan implementation for THREDDS catalogs provides for all datasetScan activities as described here:
      http://www.unidata.ucar.edu/software/thredds/v4.6/tds/reference/DatasetScan.html
  As of the day of this writing.


# OLFS Version 1.13.0 (07 April 2015)

  New w10n navigation and data access service.
  Updated ncWMS integration.
  Bug fix in THREDDS catalog inherited metadata injection.
  New Swath data aggregation service.


# OLFS Version 1.12.1 (05 January 2015)

  Disabled bug in our viewers page that was preventing some valid WMS datasets from receiving WMS and Godiva links.


# OLFS Version 1.12.0 (03 December 2014)

 Refactored OLFS configuration file. Installing into
 an existing Hyrax instance will require installing
 the new opendap.war file; restarting tomcat;
 manually copying the file:
   $CATALINA_HOME/opendap/initialContent/olfs.xml
 to:
   $CATALINA_HOME/content/opendap/olfs.xml
 and restarting Tomcat again.
 Added security patches.


# OLFS Version 1.11.5 (20 October 2014)

 Repaired bug that was introducing whitespace
 characters into the MIME Type strings extracted
 from the clients Accept header, and thus causing '
 the server to incorrectly return a HTTP status of 406

# OLFS Version 1.11.4 (14 August 2014)

Refactored OLFS configuration.
Added JSON and IJSN responses.
Added WMS support via ncWMS integration.

# OLFS Version 1.11.3 (02 July 2014)

Repaired bug that was breaking Hyrax's ability to
serve plain files (not otherwise recognized as
'data' by the BES)".


# OLFS Version 1.11.2 (06 June 2014)

Repaired broken nciso production rule.


# OLFS Version 1.11.1 (22 May 2014)

Fixed a bug in the WebStart system.


# OLFS Version 1.11.0 (1 April 2014)

Internal (but as yet not fully released support) for DAP4 responses,
along with stored_result/asynchronous behavior
Bug fixes.
Major changes to internal API.


# OLFS Version 1.10.0 (11 September 2013)


Support for geotiff and jpeg2000 responses.
Bug fixes.
Cruft code removal from code tree.
Support for constraint expressions as POST bodies.


# OLFS Version < 1.10 (06 January 2013)

I created the NEWS file with version 1.10.0 so no
prior versions are represented here.