-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding native support of AWS Cloudtrail input format #24479
base: master
Are you sure you want to change the base?
Adding native support of AWS Cloudtrail input format #24479
Conversation
This pull request has gone a while without any activity. Tagging for triage help: @mosabua |
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time. |
We had @zhaner08 and others attend the contributor call to ask about help with this PR. @pettyjamesm and @dain and @electrum will be able to help driving this so I added stale-ignore since we know this PR will come to completion over time. |
@zhaner08 let us know if you have any further questions or work planned on this PR or if you are waiting for first review beyond the input we provided during the contributor call. |
Will work on another revision of this this week |
This PR is ready to be reviewed, as discussed during the call, this currently only supports CloudTrail + Hive Json combination. |
Description
Adding native support of AWS Cloudtrail input format
Additional context and related issues
Publishing this revision out to get some feedbacks while testing is ongoing and tests are being added
Specific question regarding the implementation:
I saw the current code more about supporting multiple input format mapped to single SerDe instead of single InputFormat mapped to multiple SerDe like in this CR, is there a better way to do this? Or we want to pass the input format down to
TextLineReaderFactory
so it can create Cloudtrail line reader at that the creation time.Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( X) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: