Skip to content

Commit

Permalink
add code examples using flink-scala-api (#142)
Browse files Browse the repository at this point in the history
* add example section
* add code examples using flink-scala-api
* split main code with example module

---------

Co-authored-by: Alexey Novakov <[email protected]>
  • Loading branch information
geazi-anc and novakov-alexey authored Aug 26, 2024
1 parent acda432 commit 3bae623
Show file tree
Hide file tree
Showing 136 changed files with 2,513 additions and 1 deletion.
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,8 @@ target
.metals
.vscode
metals.sbt
.scala-build
.scala-build
.ammonite/*
*.jar
*checkpoint
sink-table
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,17 @@ If you want to create new project easily check this __Giter8 template__ out: [no

We suggest to remove the official `flink-scala` and `flink-streaming-scala` dependencies altogether to simplify the migration and do not to mix two flavors of API in the same project. But it's technically possible and not required.

## Examples

There is a wide range of [code examples](https://github.com/flink-extended/flink-scala-api/tree/master/modules/examples) to introduce you to flink-scala-api, both using Scala scripts and multimodule applications. These examples include:

- Flink jobs built using Scala 3 with Ammonite and Scala CLI;
- A complete application for fraud detection;
- Examples using Datastream and Table APIs;
- Simple job developed interactively via Jupyter notebooks;
- Word count reading text from a web socket;
- Example usage of DataGen connector and Kafka sink;
- And more;

## Differences with the Official Flink Scala API

Expand Down
39 changes: 39 additions & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ lazy val rootScalaVersion = "3.3.3"
lazy val flinkVersion = System.getProperty("flinkVersion", "1.18.1")

lazy val root = (project in file("."))
.aggregate(`scala-api`, `examples`)
.settings(
publish / skip := true
)

lazy val `scala-api` = (project in file("modules/scala-api"))
.settings(ReleaseProcess.releaseSettings(flinkVersion) *)
.settings(
name := "flink-scala-api",
Expand Down Expand Up @@ -108,3 +114,36 @@ lazy val root = (project in file("."))
mdocIn := new File("README.md")
)
.enablePlugins(MdocPlugin)

val flinkMajorAndMinorVersion =
flinkVersion.split("\\.").toList.take(2).mkString(".")

lazy val `examples` = (project in file("modules/examples"))
.settings(
scalaVersion := rootScalaVersion,
Test / fork := true,
libraryDependencies ++= Seq(
"org.flinkextended" %% "flink-scala-api" % "1.18.1_1.1.6",
"org.apache.flink" % "flink-runtime-web" % "1.18.1" % Provided,
"org.apache.flink" % "flink-clients" % "1.18.1" % Provided,
"org.apache.flink" % "flink-state-processor-api" % "1.18.1" % Provided,
"org.apache.flink" % "flink-connector-kafka" % "3.0.2-1.18" % Provided,
"org.apache.flink" % "flink-connector-files" % "1.18.1" % Provided,
"org.apache.flink" % "flink-table-runtime" % "1.18.1" % Provided,
"org.apache.flink" % "flink-table-planner-loader" % "1.18.1" % Provided,
"io.bullet" %% "borer-core" % "1.14.0" % Provided,
"ch.qos.logback" % "logback-classic" % "1.4.14" % Provided,
"org.apache.flink" % "flink-test-utils" % "1.18.1" % Test,
"org.apache.flink" % "flink-streaming-java" % "1.18.1" % Test classifier "tests",
"org.scalatest" %% "scalatest" % "3.2.15" % Test
),
Compile / run := Defaults
.runTask(
Compile / fullClasspath,
Compile / run / mainClass,
Compile / run / runner
)
.evaluated,
Compile / run / fork := true
)
.enablePlugins(ProtobufPlugin)
Loading

0 comments on commit 3bae623

Please sign in to comment.