Skip to content

CIMApplication-2.11-2.2.0-2.3.5

Compare
Choose a tag to compare
@derrickoswald derrickoswald released this 23 Nov 15:49
· 1575 commits to master since this release

This release provides a Docker image that allows using an external Spark cluster.
This is currently only tested with the sandbox cluster found in CIMReader.

To start the images, add the Spark master host (sandbox) to the start-docker command:

cd $code/CIMReader/src/test/resources
docker-compose up --scale worker=2&
cd $code/CIMApplication
./start-docker sandbox

Major changes:

  • change version numbering to scala_version-spark_version-cimapplication_version similar to CIMReader to be able to release for multiple Spark versions, so 2.11-2.2.0-2.3.5 means Scala 2.11.x, Spark 2.2.0, CIMApplication 2.3.5
  • upgrade to CIMReader-2-11-2.2.0-2.5.0
  • added themes and legends to the map view; out-of-the-box it includes basic, voltage level, topological island, and service status themes
  • added support for CGMES sample files (requires about processing and normalization using the CIMReader and un-checking the 0-based point sequence View option)
  • added the ability to load CSV files into Spark directly, there are many Load options for separator, header, etc. - and most importantly timestamp formats for time-series ingestion
  • added the ability to save an SQL query as a View, which can be used in subsequent queries
  • added Cassandra
    • the spark-docker Docker image has Cassandra installed and running
    • the cimapplication Docker image is based on the spark-docker image, Cassandra is installed but not running - just used for cqlsh: to create the initial schema and for command line access
    • the query tab has been extended to be able to query Cassandra and save (a very carefully crafted Spark query) to the measured_value_by_day Cassandra table