Skip to content

Patch for NIS export V7

Compare
Choose a tag to compare
@derrickoswald derrickoswald released this 23 Aug 12:00
· 507 commits to master since this release

This fix tries a workaround for a threading issue when registering the CHIM classes (and hence does not apply to the Spark 1.4.1 branch).

Theory

Sometimes when registering the many RDDs of case classes as tables we run into a Scala 2.10 thread safety issue (described by Michael Armbrust who closed https://issues.apache.org/jira/browse/SPARK-2178 because it depends on https://issues.scala-lang.org/browse/SI-6240 - which will never be fixed.

The following are examples of the strange things that can happen - although not consistently:

  • scala.reflect.runtime.ReflectError: value ch is not a package
  • java.lang.RuntimeException: error reading Scala signature of ch.ninecode.model.BusBarSectionInfo: value model is not a package
  • java.lang.NullPointerException

with this in the stack trace somewhere:

 at ch.ninecode.model.Parseable.register(CHIM.scala:179)
 at ch.ninecode.model._AssetInfo$.register(CHIM.scala:1396)
 at ch.ninecode.model.CHIM.<init>(CHIM.scala:6279)