Version v0.1.1 updates

mathworks-ref-arch · Sep 7, 2022 · 9cea72f · 9cea72f
1 parent 0f0cba8
commit 9cea72f
Show file tree

Hide file tree

Showing 17 changed files with 348 additions and 48 deletions.
diff --git a/.github/workflows/pages.yaml b/.github/workflows/pages.yaml
@@ -0,0 +1,26 @@
+name: Generate Documentation and Publish to Pages
+on: 
+  push:
+    branches:
+      - main
+jobs:
+  pages:
+    name: Generate Documentation and Publish to Pages 
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v2
+      - uses: actions/setup-python@v3
+        with:
+          python-version: '3.9'
+      - name: Install Sphinx Python Dependencies
+        run: pip install -r requirements.txt
+        working-directory: Documentation
+      - name: Build Documentation
+        run: make html
+        working-directory: Documentation  
+      - name: Deploy Documentation to Pages
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./Documentation/_build/html
diff --git a/Documentation/Deployment.md b/Documentation/Deployment.md
@@ -0,0 +1,102 @@
+# MATLAB Compiler (SDK) Deployment
+
+When compiling MATLAB™ code, in general, [MATLAB Compiler™](https://www.mathworks.com/products/compiler.html) will do a dependency analysis of the code and automatically include all the necessary files. However in the case of this package, because Java components are used [which need to be loaded on the static Java class path](Installation.md#configuring-the-matlab-java-class-path), additional steps are required.
+
+Three options are discussed below for making the correct JAR-files available to the deployed component.
+
+*   The [first option](#option-one-compile-the-jar-file-into-the-standalone-component) is the easiest but will add the JAR-file to the *end* of the static class path; see the [installation documentation](Installation.md#configuring-the-matlab-java-class-path) to learn more about what limitations this may introduce depending on the platform.
+
+*   The [second option](#option-two-modified-javaclasspathtxt-and-distribute-jar-file-next-to-component) can add the JAR-file to the *front* of the static Java class path but is more involved.
+
+*   The [last option](#option-three-make-jar-file-available-in-matlab-runtime) adds the JAR-file to the MATLAB Runtime installation rather than include it with the component, making it available to all standalone components using that MATLAB Runtime installation. 
+
+```{hint}
+Contact us at [[email protected]](mailto:[email protected]) if you require additional advice on your specific use-case of deploying MATLAB Code which makes use of the "MATLAB Interface *for Azure Services*". 
+```
+
+## Option One: Compile the JAR-file *into* the standalone component
+
+Any JAR-file which is compiled into a MATLAB Compiler (SDK) standalone component will automatically be added to the Java static<sup>[1](#option-three-make-jar-file-available-in-matlab-runtime)</sup> class path at runtime of the component. This will add the JAR-file to the *end* of the Java class path though, which on Windows should not be a problem but may introduce [limitations on Linux](Installation.md#configuring-the-matlab-java-class-path).
+
+To compile the JAR-files into the component, in the Application- or Library Compiler App under "Files required for your application to run" (or when working with `mcc`, using the `-a` argument), explicitly add the following files to the component, they will not be added by automatic dependency analysis:
+
+- `matlab-azure-services/Software/MATLAB/lib/jar/azure-common-sdk-0.2.0.jar`
+- `matlab-azure-services/Software/MATLAB/lib/jar/log4j.properties`
+- `matlab-azure-services/Software/MATLAB/lib/jar/log4j2.xml`
+- `$MATLABROOT/java/jarext/slf4j/slf4j-api.jar`
+- `$MATLABROOT/java/jarext/slf4j/slf4j-log4j12.jar` 
+
+Where `$MATLABROOT` stands for the MATLAB installation directory as returned by running the `matlabroot` function in the MATLAB Command Window.
+
+## Option Two: Modified `javaclasspath.txt` and distribute JAR-file *next to* component
+
+During the [installation](Installation.md) of the package, a [`javaclasspath.txt` will have been created](Installation.md#configuring-the-matlab-java-class-path) in the MATLAB preferences directory, with the following content:
+
+```xml
+<before>
+/myfiles/matlab-azure-services/Software/MATLAB/lib/jar/azure-common-sdk-0.2.0.jar
+```
+
+This `javaclasspath.txt` from the preferences directory *can* be included in MATLAB Compiler (SDK) standalone components and *will* then actually be used at runtime of the component. The problem however is that this typically refers to an absolute path which exists on the development machine but which will likely *not* exist on target machines to which the standalone component is deployed.
+
+However, the `javaclasspath.txt` file can be updated to the following before compiling the standalone component:
+
+```{code-block} xml
+---
+emphasize-lines: 3
+---
+<before>
+/myfiles/matlab-azure-services/Software/MATLAB/lib/jar/azure-common-sdk-0.2.0.jar
+./azure-common-sdk-0.2.0.jar
+```
+
+Where the line which added at the bottom basically says: load `azure-common-sdk-0.2.0.jar` from `./` which stands for "the current directory".
+
+Now when compiling a standalone component with this updated `javaclasspath.txt`, that component can load the JAR-file from either the specified absolute location *or* "the current directory at runtime of the standalone component". Where "the current directory at runtime" is typically quite simply equal to the directory where the main executable is located.
+
+Further, for this workflow, in the Application- or Library Compiler App under "Files required for your application to run" (or when working with `mcc`, using the `-a` argument), explicitly add the following files to the component:
+
+- `matlab-azure-services/Software/MATLAB/lib/jar/log4j.properties`
+- `matlab-azure-services/Software/MATLAB/lib/jar/log4j2.xml`
+- `$MATLABROOT/java/jarext/slf4j/slf4j-api.jar`
+- `$MATLABROOT/java/jarext/slf4j/slf4j-log4j12.jar` 
+- `$PREFDIR/javaclasspath.txt`
+
+Where `$PREFDIR` stands for the MATLAB preferences directory as returned by running the `prefdir` function in the MATLAB Command Windows. Depending on the exact MATLAB Compiler version, *some* versions may already include `$PREFDIR/javaclasspath.txt` *automatically*; by adding it *explicitly* though, this approach should work in *all* supported releases.
+
+And then, if working with [MATLAB Compiler (SDK) packaged installers](https://www.mathworks.com/help/compiler/files-generated-after-packaging-application-compiler.html), under "Files installed for your end user", add:
+
+- `matlab-azure-services/Software/MATLAB/lib/jar/azure-common-sdk-0.2.0.jar`
+
+The packaged installer will then place the JAR-file next to the standalone component during installation. Alternatively, if not working with the packaged installers, simply manually distribute `azure-common-sdk-0.2.0.jar` next to the standalone component itself, in the same directory.
+
+## Option Three: Make JAR-file available in MATLAB Runtime
+
+JAR-files can only be added to the *static* class path upon initialization of the MATLAB Runtime. In use cases where multiple standalone components are used in a single application, there will be only *one* MATLAB Runtime instance which is instantiated upon first interaction with the first component. If a component is loaded after initial initialization of the MATLAB Runtime, its JAR-files are added to the *dynamic* class path rather than the static class path. Unfortunately `azure-common-sdk-0.2.0.jar` *must* be loaded on the *static* class path though.
+
+In some situation, for example when working with multiple MATLAB Compiler Java/.NET/Python modules in a single Java/.NET/Python application, this is simply something to "keep in mind" and it may be possible to ensure that the right component is loaded first. However, this can not always be *guaranteed*, especially in [MATLAB Production Server](https://www.mathworks.com/products/matlab-production-server.html) workflows, where it is not possible to predict which component will be called first inside which worker process. In such situations an option could be to add the JAR-file to the MATLAB Runtime such that it is *always* loaded, regardless of which exact component instantiated this runtime first.
+
+First, at compile time, again add the following two files to the "Files required for your application to run" (or using `mcc`'s `-a` flag):
+
+- `matlab-azure-services/Software/MATLAB/lib/jar/log4j.properties`
+- `matlab-azure-services/Software/MATLAB/lib/jar/log4j2.xml`
+
+Then, manually copy the following three files to the target machine onto which the component will be deployed:
+
+- `matlab-azure-services/Software/MATLAB/lib/jar/azure-common-sdk-0.2.0.jar`
+- `$MATLABROOT/java/jarext/slf4j/slf4j-api.jar`
+- `$MATLABROOT/java/jarext/slf4j/slf4j-log4j12.jar`
+
+The files can basically be placed anywhere on this machine as long as they are accessible by the runtime. Lastly, also on the target machine where the MATLAB Runtime has been installed, open `$MCRROOT/toolbox/local/classpath.txt` (where `$MCRROOT` stands for the installation directory of the MATLAB Runtime) in a text editor and add the full absolute locations of the three files to the *front* of the list of JAR-files and directories in the text file.
+
+````{note}
+The `toolbox/local/classpath.txt` file contains a notification:
+
+```
+#DO NOT MODIFY THIS FILE.  IT IS AN AUTOGENERATED FILE.
+```
+
+For this particular use-case, this can partly be ignored, the file *may* be edited but do indeed keep in mind that it may be changed or overwritten when reinstalling the MATLAB Runtime or installing an Update to the runtime. The modification may have to be reapplied afterwards.
+````
+
+[//]: #  (Copyright 2022 The MathWorks, Inc.)
diff --git a/Documentation/Installation.md b/Documentation/Installation.md
@@ -35,16 +35,18 @@ the static java class path if `SharedTokenCacheCredential` and
 `TokenCachePersistenceOptions` are used, if not, the jar file can be in any
 position on the static java class path.
 
-> **_NOTE:_** when making use of MathWorks features which can automatically add
-> jar files to the static class path, these typically add them to then *end* of
-> the static class path. For example when working with a [packaged custom
-> toolbox](https://www.mathworks.com/help/matlab/matlab_prog/create-and-share-custom-matlab-toolboxes.html)
-> the included jar file is added to the *end* of the static path in the end user
-> MATLAB installation. Or if working with MATLAB Compiler (SDK) standalone
-> components the jar file which was packaged into the component are
-> automatically added to the *end* of the static class path at runtime. However
-> there may be situations in which this is not possible and then these features
-> may add the jar file to the dynamic class path.
+```{note}
+When making use of MathWorks features which can automatically add
+jar files to the static class path, these typically add them to then *end* of
+the static class path. For example when working with a [packaged custom
+toolbox](https://www.mathworks.com/help/matlab/matlab_prog/create-and-share-custom-matlab-toolboxes.html)
+the included jar file is added to the *end* of the static path in the end user
+MATLAB installation. Or if working with MATLAB Compiler (SDK) standalone
+components the jar file which was packaged into the component are
+automatically added to the *end* of the static class path at runtime. However
+there may be situations in which this is not possible and then these features
+may add the jar file to the dynamic class path.
+```
 
 In general the recommended approach to add the jar file to the static java class
 path in a local MATLAB installation is to add an entry to the

diff --git a/Documentation/README.md b/Documentation/README.md
@@ -1,16 +1,21 @@
-# MATLAB Interface *for Azure Services*
-
-## Contents
-
-* [Overview](../README.md)
-* [Installation](Installation.md)
-* [Configuration](Configuration.md)
-* [Authentication](Authentication.md)
-* Service specific documentation
-    * [Azure Data Lake Storage Gen2](DataLakeStorageGen2.md)
-    * [Azure Key Vault](KeyVault.md)
-* [Full API Reference](APIReference.md)
-* [FAQ](FAQ.md)
-* [References](References.md)
-
-[//]: #  (Copyright 2021-2022 The MathWorks, Inc.)
+# MATLAB Interface *for Azure Services*
+
+**Refer to the HTML documentation for the latest version of this help information with enhanced navigation, discoverability, and readability.**
+
+**<https://mathworks-ref-arch.github.io/matlab-azure-services>**
+
+## Contents
+
+* [Overview](../README.md)
+* [Installation](Installation.md)
+* [Configuration](Configuration.md)
+* [Authentication](Authentication.md)
+* Service specific documentation
+    * [Azure Data Lake Storage Gen2](DataLakeStorageGen2.md)
+    * [Azure Key Vault](KeyVault.md)
+* [MATLAB Compiler (SDK) Deployment](Deployment.md)
+* [Full API Reference](APIReference.md)
+* [FAQ](FAQ.md)
+* [References](References.md)
+
+[//]: #  (Copyright 2021-2022 The MathWorks, Inc.)
diff --git a/Documentation/conf.py b/Documentation/conf.py
@@ -98,6 +98,14 @@
 #
 # html_theme_options = {}
 
+html_context = {
+    'display_github': True,
+    'github_user': 'mathworks-ref-arch',
+    'github_repo': 'matlab-azure-services',
+    'github_version': 'main',
+    'conf_py_path': '/Documentation/'
+}
+
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".

diff --git a/Documentation/index.rst b/Documentation/index.rst
@@ -6,21 +6,34 @@ MATLAB Interface *for Azure Services*
 
 .. toctree::
    :maxdepth: 2
-   :caption: Contents
-   :name: README
-   :numbered:
+   :caption: Overview
 
    Overview
 
+.. toctree::
+   :maxdepth: 2
+   :caption: Installation & Configuration
+
    Installation
    Configuration
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Usage
+
    Authentication
 
    Services
 
    APIReference   
-
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Advanced Topics
+
+   Deployment
    Testing
    FAQ
 
    References
+
diff --git a/Documentation/requirements.txt b/Documentation/requirements.txt
@@ -0,0 +1,5 @@
+myst-parser==0.17.0
+Sphinx==4.4.0
+sphinx-markdown-tables==0.0.15
+sphinx-rtd-theme==1.0.0
+attrs==21.4.0
diff --git a/README.md b/README.md
@@ -2,11 +2,11 @@
 
 ## Introduction
 
-This package offers MATLAB interfaces that connect to various Microsoft Azure®
+This package offers MATLAB™ interfaces that connect to various Microsoft Azure®
 Services it currently supports:
 
-* [Azure Data Lake Storage Gen2](Documentation/DataLakeStorageGen2.md)
-* [Azure Key Vault](Documentation/KeyVault.md)
+* [Azure Data Lake Storage Gen2](https://mathworks-ref-arch.github.io/matlab-azure-services/DataLakeStorageGen2.html)
+* [Azure Key Vault](https://mathworks-ref-arch.github.io/matlab-azure-services/KeyVault.html)
 
 > Note, very many of MATLAB's IO operations support Blob Storage via builtin functions.
 > For example ```dir``` supports accessing remote data:
@@ -30,9 +30,15 @@ Services it currently supports:
 
 This package is primarily tested using Ubuntu™ 20.04 and Windows® 10.
 
+## Documentation
+
+The main documentation for this package is available at:
+
+<https://mathworks-ref-arch.github.io/matlab-azure-services>
+
 ## Usage
 
-Once [installed](Documentation/Installation.md) the interface is added to the MATLAB path
+Once [installed](https://mathworks-ref-arch.github.io/matlab-azure-services/Installation.html) the interface is added to the MATLAB path
 by running `startup.m` from the `Software/MATLAB` directory.
 
 ### Azure Data Lake Storage Gen2
@@ -70,7 +76,7 @@ dataLakeFileClient = createStorageClient('FileSystemName','myFileSystem',...
 tf = dataLakeFileClient.exists();
 ```
 
-For further details see: [Azure Data Lake Storage Gen2](Documentation/DataLakeStorageGen2.md)
+For further details see: [Azure Data Lake Storage Gen2](https://mathworks-ref-arch.github.io/matlab-azure-services/DataLakeStorageGen2.html)
 
 ### Azure Key Vault
 
@@ -95,22 +101,22 @@ properties = keyClient.listPropertiesOfKeys();
 name = propList(1).getName();
 ```
 
-For further details see: [Azure Key Vault](Documentation/KeyVault.md)
+For further details see: [Azure Key Vault](https://mathworks-ref-arch.github.io/matlab-azure-services/KeyVault.html)
 
 ### Configuration
 
 The package offers a `loadConfigurationSettings` function which allows reading
 configuration settings from a short JSON format file. This offers a convenient
 way for you to configure various settings (like endpoint URLs) as well as
 authentication configurations without having to hardcode these into your MATLAB
-code. For more details see: [Documentation/Configuration.md](Documentation/Configuration.md)
+code. For more details see: [Configuration](https://mathworks-ref-arch.github.io/matlab-azure-services/Configuration.html)
 
 ### Authentication
 
 Virtually all interactions with Azure will require some form of authentication.
 The authentication workflows are common to all services. The package offers
 various Builder classes as well as a higher-level function `configureCredentials`
-to aid performing the authentication. For more details see: [Documentation/Authentication.md](Documentation/Authentication.md)
+to aid performing the authentication. For more details see: [Authentication](https://mathworks-ref-arch.github.io/matlab-azure-services/Authentication.html)
 
 ## License
 

diff --git a/RELEASENOTES.md b/RELEASENOTES.md
@@ -2,6 +2,12 @@
 
 ## Release Notes
 
+## Release 0.1.1 September 7th 2022
+
+* Published documentation to GitHub pages.
+* Fixed relative path issues with `uploadFromFile` and `downloadToFile`.
+* Documented deployment workflows.
+
 ## Release 0.1.0 May 18th 2022
 
 * Initial release to GitHub.com

diff --git a/Software/Java/pom.xml b/Software/Java/pom.xml
@@ -22,7 +22,7 @@
         <dependency>
           <groupId>com.azure</groupId>
           <artifactId>azure-sdk-bom</artifactId>
-          <version>1.2.0</version>
+          <version>1.2.5</version>
           <type>pom</type>
           <scope>import</scope>
         </dependency>
@@ -133,6 +133,10 @@
                                     <pattern>com.fasterxml.woodstox</pattern>
                                     <shadedPattern>shaded.com.fasterxml.woodstox</shadedPattern>
                                 </relocation>
+                                <relocation>
+                                    <pattern>com.ctc.wstx</pattern>
+                                    <shadedPattern>shaded.com.ctc.wstx</shadedPattern>
+                                </relocation>                                
                                 <relocation>
                                     <pattern>io.netty</pattern>
                                     <shadedPattern>shaded.io.netty</shadedPattern>

diff --git a/Software/MATLAB/app/system/+azure/+storage/+blob/@BlobClient/downloadToFile.m b/Software/MATLAB/app/system/+azure/+storage/+blob/@BlobClient/downloadToFile.m
@@ -21,10 +21,29 @@ function downloadToFile(obj, filePath, varargin)
 % To be extended
 parse(p,varargin{:});
 
-if p.Results.overwrite
-    obj.Handle.downloadToFile(filePath, p.Results.overwrite);
-else
-    obj.Handle.downloadToFile(filePath);
+% Check whether file exists, if so throw an error if overwrite not set such that
+% all other code below can then simply just always assume overwriting is fine.
+if isfile(filePath) && ~p.Results.overwrite
+    logObj = Logger.getLogger();
+    write(logObj,'error','File already exists and overwrite is set to false.');
 end
 
+% Determine the parent directory of where to save the file
+[d,f,e] = fileparts(filePath);
+% Get attributes of this directory
+[status,info] = fileattrib(d);
+% If this location does not exist, throw an error
+if status == false
+    logObj = Logger.getLogger();
+    write(logObj,'error',sprintf('Specified download location "%s" does not exist.',d));
+end    
+
+% Form file name based on absolute location of directory with filename and
+% extension appended
+absPath = fullfile(info.Name,[f e]);
+
+% Now call Java downloadToFile with absolute path and overwrite simply always
+% set to true.
+obj.Handle.downloadToFile(absPath, true);
+
 end