[Enhancement] - Enable Unloading Redshift Tables to S3 in either JSON, PARQUET or CSV format

[NirTatcher]

• Adding the format parameter to the unload from redshift to S3 function. We can use either CSV or JSON or PARQUET (highly recommended for large tables, using PARQUET files are smaller than CSVs, can be read and written much faster compared to CSV), currently we only enable to unload as a TXT file by default.
• Including only supported options to the unload query.
• Unloading as a CSV does not support ADDQUOTES.
• Unloading as a PARQUET does not support DELIMITER, ADDQUOTES, ESCAPE, NULL AS, HEADER, GZIP (compression) options.
• Unloading as a JSON does not support DELIMITER, HEADER, ADDQUOTES, ESCAPE, NULL AS options.

Hope this will be beneficial to anyone except me and pass all of the live tests. @shaunagm please let me know if there are any changes that need to be made to get this implemented.
Thank you!
More on PARQUET here.
More on UNLOAD here.

[shaunagm]

Thanks so much @NirTatcher! This looks good to me but I'd love it if someone with more Redshift knowledge gave it a look too. Maybe @austinweisgrau or @Jason94?

[NirTatcher]

Thanks so much @NirTatcher! This looks good to me but I'd love it if someone with more Redshift knowledge gave it a look too. Maybe @austinweisgrau or @Jason94?

Sure thing @shaunagm!
If it would make it easier I can just make all of that in separated functions so like unload (as text by default), unload as csv, unload as json, unload as parquet functions.
Also, I believe it's much more powerful (at least for my use) to unload from redshidt to s3 as csv/json/parquert than in text format.
So I'll be waiting and would be happy to receive any feedback from @Jason94 or @austinweisgrau so thank you all in advance!

[NirTatcher]

@austinweisgrau once you or anyone else gets to it I will be glad to get feedback on this so we can start letting people to unload tables into S3 buckets in another format than TXT.
Thank you in advance!

[shaunagm]

This looks good to me, let's wait and see if there's a bit more availability to review right now.

[shaunagm]

So I don't know the underlying tooling at all, but reading this the only thing that concerns me is that the ordering of the string created by the new version is different. Is order relatively flexible here?