[BUG] S3 Sink Avro Output issues

[engechas]

Describe the bug
A few bugs found for S3 Sink Avro output codecs:

1. With compression: gzip specified, the files are still uploaded with just a .avro suffix. This should be .avro.gz
2. The contents of the avro file show null for all fields:

{"version":null,"srcport":null,"dstport":null,"accountId":null,"interfaceId":null,"srcaddr":null,"dstaddr":null,"start":null,"end":null,"protocol":null,"packets":null,"bytes":null,"action":null,"logStatus":null}
{"version":null,"srcport":null,"dstport":null,"accountId":null,"interfaceId":null,"srcaddr":null,"dstaddr":null,"start":null,"end":null,"protocol":null,"packets":null,"bytes":null,"action":null,"logStatus":null}
{"version":null,"srcport":null,"dstport":null,"accountId":null,"interfaceId":null,"srcaddr":null,"dstaddr":null,"start":null,"end":null,"protocol":null,"packets":null,"bytes":null,"action":null,"logStatus":null}
{"version":null,"srcport":null,"dstport":null,"accountId":null,"interfaceId":null,"srcaddr":null,"dstaddr":null,"start":null,"end":null,"protocol":null,"packets":null,"bytes":null,"action":null,"logStatus":null}

3. The timestamp in the file name is 12 hours behind UTC. I am not sure what the "correct" time is but UTC seems logical

To Reproduce
Sink config:

sink:
    - s3:
        aws:
          region: "us-west-2"
          sts_role_arn: "<my role>"
        bucket: "my-sink-bucket"
        object_key:
          path_prefix: "s3-sink"
        threshold:
          event_collect_timeout: 600s
        compression: "gzip"
        codec:
          avro:
            schema: >
              {
                "type" : "record",
                "namespace" : "org.opensearch.dataprepper.examples",
                "name" : "VpcFlowLog",
                "fields" : [
                  { "name" : "version", "type" : ["null", "string"]},
                  { "name" : "srcport", "type": ["null", "int"]},
                  { "name" : "dstport", "type": ["null", "int"]},
                  { "name" : "accountId", "type" : ["null", "string"]},
                  { "name" : "interfaceId", "type" : ["null", "string"]},
                  { "name" : "srcaddr", "type" : ["null", "string"]},
                  { "name" : "dstaddr", "type" : ["null", "string"]},
                  { "name" : "start", "type": ["null", "int"]},
                  { "name" : "end", "type": ["null", "int"]},
                  { "name" : "protocol", "type": ["null", "int"]},
                  { "name" : "packets", "type": ["null", "int"]},
                  { "name" : "bytes", "type": ["null", "int"]},
                  { "name" : "action", "type": ["null", "string"]},
                  { "name" : "logStatus", "type" : ["null", "string"]}
                ]
              }

Screenshots
File name times example:

[dlvenable]

The Avro codec does not work with union types, nor is there any support for it in the code.

data-prepper/data-prepper-plugins/avro-codecs/src/main/java/org/opensearch/dataprepper/plugins/codec/avro/AvroOutputCodec.java

Lines 192 to 227 in 60f69b3

	if (PRIMITIVE_TYPES.contains(fieldType)) {
	switch (fieldType) {
	case "string":
	finalValue = rawValue.toString();
	break;
	case "int":
	finalValue = Integer.parseInt(rawValue.toString());
	break;
	case "float":
	finalValue = Float.parseFloat(rawValue.toString());
	break;
	case "double":
	finalValue = Double.parseDouble(rawValue.toString());
	break;
	case "long":
	finalValue = Long.parseLong(rawValue.toString());
	break;
	case "bytes":
	finalValue = rawValue.toString().getBytes(StandardCharsets.UTF_8);
	break;
	default:
	LOG.error("Unrecognised Field name : '{}' & type : '{}'", field.name(), fieldType);
	break;
	}
	} else {
	if (fieldType.equals("record") && rawValue instanceof Map) {
	finalValue = buildAvroRecord(field.schema(), (Map<String, Object>) rawValue);
	} else if (fieldType.equals("array") && rawValue instanceof List) {
	GenericData.Array<String> avroArray =
	new GenericData.Array<>(((List<String>) rawValue).size(), field.schema());
	for (String element : ((List<String>) rawValue)) {
	avroArray.add(element);
	}
	finalValue = avroArray;
	}
	}

[engechas]

For the timestamp in the file name, the same behavior is seen with ndjson so it doesn't look related to avro, rather S3 sink as a whole

[engechas]

#3171 fixes number 3

[dlvenable]

PR #3194 fixes item 2.

[dlvenable]

Item 1 should be resolved by #3196.