Skip to content

Commit

Permalink
Merge pull request #1023 from eclipse/ag_snapshot-updates
Browse files Browse the repository at this point in the history
 Update versions for snapshot testing
  • Loading branch information
agibsonccc authored Jul 27, 2021
2 parents 5e9d1f6 + f744cf3 commit 57a99ad
Show file tree
Hide file tree
Showing 62 changed files with 968 additions and 3,052 deletions.
31 changes: 10 additions & 21 deletions .github/workflows/example-sanity-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,39 +25,28 @@ jobs:
with:
access_token: ${{ github.token }}
- uses: actions/checkout@v2
- uses: ./.github/actions/update-deps-linux
- uses: ./.github/actions/install-protobuf-linux
- uses: ./.github/actions/install-cmake-linux
- name: Set up Java for publishing to GitHub Packages
uses: actions/setup-java@v1
with:
java-version: 1.8
server-id: sonatype-nexus-snapshots
server-username: MAVEN_USERNAME
server-password: MAVEN_PASSWORD
gpg-private-key: ${{ secrets.SONATYPE_GPG_KEY }}
gpg-passphrase: MAVEN_GPG_PASSPHRASE
- name: Build on linux-arm64
shell: bash
env:
DEBIAN_FRONTEND: noninteractive
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DEPLOY: 1
BUILD_USING_MAVEN: 1
TARGET_OS: linux
CURRENT_TARGET: arm64
PUBLISH_TO: ossrh
DEPLOY_TO: ossrh
MAVEN_USERNAME: ${{ secrets.SONATYPE_USER_1 }}
MAVEN_PASSWORD: ${{ secrets.SONATYPE_USER1_PASS }}
MAVEN_GPG_PASSPHRASE: ${{ secrets.PACKAGES_GPG_PASS }}

run: |
mvn --version
cmake --version
protoc --version
for folder in `ls`; do
mvn clean test
if [[ -d "$folder" ]]; then
cd "$folder"
if test -f "pom.xml"; then
mvn clean test
fi
cd ..
fi
done
Empty file.
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ We do not monitor the github issues of this repository very often.
The **Eclipse Deeplearning4J** (DL4J) ecosystem is a set of projects intended to support all the needs of a JVM based deep learning application. This means starting with the raw data, loading and preprocessing it from wherever and whatever format it is in to building and tuning a wide variety of simple and complex deep learning networks.

The DL4J stack comprises of:
- **DL4J**: High level API to build MultiLayerNetworks and ComputationGraphs with a variety of layers, including custom ones. Supports importing Keras models from h5, including tf.keras models (as of 1.0.0-beta7) and also supports distributed training on Apache Spark
- **DL4J**: High level API to build MultiLayerNetworks and ComputationGraphs with a variety of layers, including custom ones. Supports importing Keras models from h5, including tf.keras models (as of 1.0.0-M1.1) and also supports distributed training on Apache Spark
- **ND4J**: General purpose linear algebra library with over 500 mathematical, linear algebra and deep learning operations. ND4J is based on the highly-optimized C++ codebase LibND4J that provides CPU (AVX2/512) and GPU (CUDA) support and acceleration by libraries such as OpenBLAS, OneDNN (MKL-DNN), cuDNN, cuBLAS, etc
- **SameDiff** : Part of the ND4J library, SameDiff is our automatic differentiation / deep learning framework. SameDiff uses a graph-based (define then run) approach, similar to TensorFlow graph mode. Eager graph (TensorFlow 2.x eager/PyTorch) graph execution is planned. SameDiff supports importing TensorFlow frozen model format .pb (protobuf) models. Import for ONNX, TensorFlow SavedModel and Keras models are planned. Deeplearning4j also has full SameDiff support for easily writing custom layers and loss functions.
- **DataVec**: ETL for machine learning data in a wide variety of formats and files (HDFS, Spark, Images, Video, Audio, CSV, Excel etc)
Expand All @@ -40,6 +40,17 @@ Users can also refer to the [simple sample project provided](./mvn-project-templ

Build tools are considered standard software engineering best practice. Besides this the complexities posed by the projects in the DL4J ecosystem make dependencies too difficult to manage manually. All the projects in the DL4J ecosystem can be used with other build tools like Gradle, SBT etc. More information on that can be found [here](https://deeplearning4j.konduit.ai/config/buildtools).

## Support

For help with the examples, please go to our [support forum](https://community.konduit.ai/)

Note for users of 1.0.0-beta7 and prior, some examples and modules have been removed to reflect
changes in the framework's direction. Please see and comment on our post [here](https://community.konduit.ai/t/upcoming-removal-of-modules-and-roadmap-changes/1240)

If you would like a workaround for something you may be missing,
please feel free to post on the forums, and we will do what we can to help you.


## Example Content
Projects are based on what functionality the included examples demonstrate to the user and not necessarily which library in the DL4J stack the functionality lives in.

Expand Down Expand Up @@ -69,9 +80,6 @@ This project contains a set of examples that demonstrate how raw data in various
- [nd4j-ndarray-examples](nd4j-ndarray-examples/README.md)
This project contains a set of examples that demonstrate how to manipulate NDArrays. The functionality of ND4J demonstrated here can be likened to NumPy.

- [arbiter-examples](arbiter-examples/README.md)
This project contains a set of examples that demonstrate usage of the Arbiter library for hyperparameter tuning of Deeplearning4J neural networks.

- [rl4j-examples](rl4j-examples/README.md)
This project contains examples of using RL4J, the reinforcement learning library in DL4J.

Expand Down
8 changes: 0 additions & 8 deletions android-examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,6 @@ The build configuration for Android can be found in [app/build.gradle](./app/bui

[Go back](../README.md) to the main repository page to explore other features/functionality of the **Eclipse Deeplearning4J** ecosystem. File an issue [here](https://github.com/eclipse/deeplearning4j-examples/issues) to request new features.

# Known Issues
Due to an unfortunately timed change, the 1.0.0-beta7 release doesn't work well with most Android devices. For this reason, this example project uses the SNAPSHOT version.

Again unfortunately, gradle has issues retrieving SNAPSHOT versions when combined with classifiers, which are used here to retrieve only the android specific backend dependencies.

If you want to run this example locally, you will therefore need to have Maven installed in order to download those specific dependencies.

Once you have installed Maven, you will need to run the following commands from the command line to download the correct android backend files:

```
mvn dependency:get -DremoteRepositories=snapshots::::https://oss.sonatype.org/content/repositories/snapshots -Dartifact=org.nd4j:nd4j-native:1.0.0-SNAPSHOT:jar:android-arm
Expand Down
21 changes: 11 additions & 10 deletions android-examples/app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ android {
versionCode 1
versionName "1.0"

testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}

buildTypes {
Expand All @@ -27,10 +27,10 @@ android {
}

dependencies {
def dl4jVersion = '1.0.0-beta7'
def openblasVersion = '0.3.9-1.5.3'
def opencvVersion = '4.3.0-1.5.3'
def leptonicaVersion = '1.79.0-1.5.3'
def dl4jVersion = '1.0.0-M1.1'
def openblasVersion = '0.3.10-1.5.5'
def opencvVersion = '4.5.1-1.5.5'
def leptonicaVersion = '1.80.0-1.5.5'

implementation fileTree(dir: 'libs', include: ['*.jar'])

Expand All @@ -45,13 +45,14 @@ dependencies {
exclude group: 'org.bytedeco', module: 'leptonica-platform'
exclude group: 'org.bytedeco', module: 'hdf5-platform'
exclude group: 'org.nd4j', module: 'nd4j-base64'
exclude group: 'org.nd4j', module: 'nd4j-api'
}

implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-arm"
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-arm64"
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-x86"
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-x86_64"
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-arm"
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-arm64"
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-x86"
implementation group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-x86_64"
implementation group: 'org.bytedeco', name: 'openblas', version: openblasVersion
implementation group: 'org.bytedeco', name: 'openblas', version: openblasVersion, classifier: "android-arm"
implementation group: 'org.bytedeco', name: 'openblas', version: openblasVersion, classifier: "android-arm64"
Expand All @@ -69,4 +70,4 @@ dependencies {
implementation group: 'org.bytedeco', name: 'leptonica', version: leptonicaVersion, classifier: "android-x86_64"

annotationProcessor group: 'org.projectlombok', name: 'lombok', version: '1.18.4'
}
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package com.example.androidDl4jClassifier;
package com.example.androidclassifier;

import android.content.Context;

Expand Down
6 changes: 3 additions & 3 deletions android-examples/app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.androidDl4jClassifier">
package="com.example.androidclassifier">

<application
android:allowBackup="true"
Expand All @@ -10,7 +10,7 @@
android:supportsRtl="true"
android:theme="@style/Theme.AppCompat.Light.DarkActionBar"
android:fullBackupContent="@xml/backup_descriptor">
<activity android:name="com.example.androidDl4jClassifier.MainActivity">
<activity android:name="com.example.org.deeplearning4j.examples.androidclassifier.MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN" />

Expand All @@ -19,4 +19,4 @@
</activity>
</application>

</manifest>
</manifest>
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
package com.example.androidDl4jClassifier;
package org.deeplearning4j.examples.androidclassifier;

import androidx.appcompat.app.AppCompatActivity;

import android.os.Bundle;
import android.view.Menu;
import android.view.MenuItem;
import com.example.androidclassifier.R;


public class MainActivity extends AppCompatActivity {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package com.example.androidDl4jClassifier;
package org.deeplearning4j.examples.androidclassifier;

import android.content.Context;
import android.graphics.Canvas;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package com.example.androidDl4jClassifier;
package org.deeplearning4j.examples.androidclassifier;

import org.deeplearning4j.nn.conf.MultiLayerConfiguration;
import org.deeplearning4j.nn.conf.NeuralNetConfiguration;
Expand Down
6 changes: 3 additions & 3 deletions android-examples/app/src/main/res/layout/activity_main.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
tools:context="org.deeplearning4j.examples.androidclassifier.MainActivity">


<view
android:id="@+id/id_scatterview"
class="com.example.androidDl4jClassifier.ScatterView"
class="org.deeplearning4j.examples.androidclassifier.ScatterView"
android:layout_width="fill_parent"
android:layout_height="0dp"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintHorizontal_bias="0.0"
app:layout_constraintStart_toStartOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>
</androidx.constraintlayout.widget.ConstraintLayout>

This file was deleted.

26 changes: 0 additions & 26 deletions cuda-specific-examples/README.md

This file was deleted.

Loading

0 comments on commit 57a99ad

Please sign in to comment.