SUB #255: Spark-Notebook Glue Code Prototype

[FRosner]

Problem

For the first prototype of Spark-Notebook-DDS-Integration we need to write some glue code.

Solution

• Server implementing transformation of servable case class to Chart magic case class. We might need to change the way the Chart magic classes are defined to make them more general. - @Gerrrr, @FRosner, @andypetrella)
• JS implementation to plug into Spark-Notebook front-end (webworkers, etc.) - @Gerrrr

[andypetrella]

ah this is the one I guess @FRosner :-P

[FRosner]

Roadmap

• We give webworkers a try, reactivity is not a must for now
• Later, we might want to have javascript context to export a function as JS snippet string
• Implement a DDS server that returns a renderer (or renderable by implicit conversion) to the notebook

Notes

• JsWorld is center of reactivity for JS
• Connection is reactivity for Scala
• Renderer is very basic and the only thing that would be required to render something in the notebook

[andypetrella]

sketch on how snippet can be used:

:javascript[myFunction]
function() {
  //worker....

  container.append()

}

→ REPL creates a val myFunction = s""" ... """

Use it as snippet

[...]
  override val snippets = List(s"""|{
                                   |  "f": $myFunction, 
                                   |  "o": {
                                   |    "category": "$category",
                                   |    "value": "$value",
                                   |    "date": "$date",
                                   |    "date_format": "$dateFormat"
                                   |  }
                                   |}
                                  """.stripMargin)