You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
S3 Sink users would like to be able to group Events together and send them to dynamic partitions of their S3 bucket. Users would like to track different groups based on what is in the Events to group them into different S3 “folders”.
Describe the solution you'd like
Proposal with path_prefix and object_name
sink:
- s3:
object_key:
path_prefix: "folder-${/partition_key}/"
object_name: "my-object-${/some_key}";
threshold:
// We flush and remove groups when they reach 1000 events
event_count: 1000
// Alternatively, flush and remove groups on time when event_count isn't reached
event_collect_timeout: "30s"
When dynamic values are used in either the path_prefix or the object_name, we will group Events together based on the expression evaluation result.
Describe alternatives you've considered (Optional)
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
@graytaylor0 , I believe this request is already expressed in #3310. Then we also have #4099 for the configurable object name. I'll close this one as a duplicate.
Is your feature request related to a problem? Please describe.
S3 Sink users would like to be able to group Events together and send them to dynamic partitions of their S3 bucket. Users would like to track different groups based on what is in the Events to group them into different S3 “folders”.
Describe the solution you'd like
Proposal with path_prefix and object_name
When dynamic values are used in either the path_prefix or the object_name, we will group Events together based on the expression evaluation result.
Describe alternatives you've considered (Optional)
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: