Skip to content

Commit

Permalink
describe case class schema evolution
Browse files Browse the repository at this point in the history
  • Loading branch information
novakov-alexey committed Jun 8, 2024
1 parent 133dc97 commit a8482b3
Showing 1 changed file with 18 additions and 0 deletions.
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ implicit val mapper2: TypeMapper[WrappedString, String] = new TypeMapper[Wrapped

### Schema evolution

#### ADT
For the child case classes being part of ADT, the serializers use a Flink's `ScalaCaseClassSerializer`, so all the compatibility rules
are the same as for normal case classes.

Expand All @@ -245,6 +246,23 @@ For the sealed trait membership itself, this library uses own serialization form
* you cannot remove ADT members
* you cannot replace ADT members

#### Case Class Changes

On a case class level, this library supports new field addition(s) with default value(s). This allows to restore a Flink job from a savepoint created using previous case class schema.
For example:

1. A Flink job was stopped with a savepoint using below case class schema:
```scala
case class Click(id: String, inFileClicks: List[ClickEvent])
```
2. Now the Click case class is changed to:
```scala
case class Click(id: String, inFileClicks: List[ClickEvent],
fieldInFile: String = "test1",
fieldNotInFile: String = "test2")
```
3. Launch the same job with new case class schema version from the last savepoint. Job restore should work successfully.

### Compatibility

This project uses a separate set of serializers for collections, instead of Flink's own TraversableSerializer. So probably you
Expand Down

0 comments on commit a8482b3

Please sign in to comment.