Snapshot Testing for Integration Tests

[feniljain]

Hey everyone! 👋🏻

Creating this issue to continue discussion from these messages on possibility of using snapshot testing for integration tests:

• test: append partition data file #742 (comment)
• test: append partition data file #742 (comment)

[feniljain]

@ZENOTME you mentioned:

A creative idea is to support Avro format files, allowing us to create snapshots of the entire Iceberg metadata, which can then be used for quick comparisons in end-to-end tests.

Do you mean using avro metadata files directly as snapshots? I am not sure how that addresses your concern of:

One thing we need to address maybe filter out some random field.

Am I understanding something wrong here? Can you please elaborate a bit more :)

[ZENOTME]

@ZENOTME you mentioned:

A creative idea is to support Avro format files, allowing us to create snapshots of the entire Iceberg metadata, which can then be used for quick comparisons in end-to-end tests.

Do you mean using avro metadata files directly as snapshots? I am not sure how that addresses your concern of:

One thing we need to address maybe filter out some random field.

Am I understanding something wrong here? Can you please elaborate a bit more :)

Sorry, I didn't have a clear expression here. Actually, they are two things independently here.

[xxchan]

I'm also a fan of snapshot testing. In this PR #822, I used expect-test to do it.