Skip to content

Commit

Permalink
Releasing 0.4.2
Browse files Browse the repository at this point in the history
  • Loading branch information
EnricoMi committed Jul 28, 2020
1 parent 830f640 commit 61978be
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 7 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [UNRELEASED] - YYYY-MM-DD
## [0.4.2] - 2020-07-28

### Fixed
- Fixed dependency conflicts between connector dependencies and Spark.
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,15 @@ above will be addressed in the near future.

The Spark Dgraph Connector is available for Spark 2.4 and Spark 3.0, both with Scala 2.12.
Use Maven artifact ID `spark-dgraph-connector_2.12`. The Spark version is part of the package version,
e.g. 0.4.1-2.4 and 0.4.1-3.0, respectively.
e.g. 0.4.2-2.4 and 0.4.2-3.0, respectively.
Minor versions are kept in sync between those two packages, such that identical minor versions contain identical feature sets (where supported by the respective Spark version).

### SBT

Add this line to your `build.sbt` file to use the latest version for Spark 2.4:

```sbt
libraryDependencies += "uk.co.gresearch.spark" %% "spark-dgraph-connector" % "0.4.1-2.4"
libraryDependencies += "uk.co.gresearch.spark" %% "spark-dgraph-connector" % "0.4.2-2.4"
```

### Maven
Expand All @@ -64,19 +64,19 @@ Add this dependency to your `pom.xml` file to use the latest version:
<dependency>
<groupId>uk.co.gresearch.spark</groupId>
<artifactId>spark-dgraph-connector_2.12</artifactId>
<version>0.4.1-2.4</version>
<version>0.4.2-2.4</version>
</dependency>
```

### PySpark Shell and Python script

Launch the Python Spark REPL (pyspark 2.4.2 and ≥3.0) with the Spark Dgraph Connector dependency (version ≥0.4.2) as follows:

pyspark --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.4.1-3.0 --conf spark.driver.userClassPathFirst=true
pyspark --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.4.2-3.0 --conf spark.driver.userClassPathFirst=true

Run your Python script that uses PySpark (pyspark 2.4.2 and ≥3.0) and the Spark Dgraph Connector (version ≥0.4.2) via `spark-submit`:

spark-submit --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.4.1-3.0 --conf spark.driver.userClassPathFirst=true [script.py]
spark-submit --packages uk.co.gresearch.spark:spark-dgraph-connector_2.12:0.4.2-3.0 --conf spark.driver.userClassPathFirst=true [script.py]

The `--conf spark.driver.userClassPathFirst=true` is required to avoid the following exception:

Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>uk.co.gresearch.spark</groupId>
<artifactId>spark-dgraph-connector_2.12</artifactId>
<version>0.4.2-2.4-SNAPSHOT</version>
<version>0.4.2-2.4</version>
<name>Spark Dgraph Connector</name>
<description>A Spark connector for Dgraph databases</description>
<inceptionYear>2020</inceptionYear>
Expand Down

0 comments on commit 61978be

Please sign in to comment.