Add compression config for Parquest files

[orinciog]

I've added a config for this plugin in order to specify the compression for parquest files.

The setting is called 'parquet.codec' and can have the following values: none, snappy, gzip, brotli, lz4, lzo, zstd.

[ghost]

It looks like @orinciog hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.

Once you've signed reply with [clabot:check] to prove it.

Appreciation of efforts,

clabot

[orinciog]

[clabot:check]

[ghost]

@confluentinc It looks like @orinciog just signed our Contributor License Agreement. 👍

Always at your service,

clabot

[sonupillai]

@orinciog Could you please add details on the testing done? Also if you could add the test coverage, it would be great and easy for us to review this. Thank you!

[ilanjiR]

@orinciog Overall, this change looks good. Can you add some tests/describe your testing methodology here?

[orinciog]

@ilanjiR Thank you for your message. We already use a patched version of kafka connect hdfs which includes these changes in it.

We are using parquet.codec=gzip setting in our kafka-connect configuration.

All parquet files in hdfs are compressed gzip.

Thank you,

[RuiFG]

@levzem levzemWhy is this PR not merged, but parquet.codec is listed in the document description
https://docs.confluent.io/kafka-connect-hdfs/current/configuration_options.html#connector last configuration description

[orinciog]

@levzem What extra details should I provide in order to merge this PR? Thank you.

[cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

❌ oranciog-bd
❌ orinciog
_{You have signed the CLA already but the status is still pending? Let us recheck it.}