Squashed commit of the following:

commit e2a688fbb1c8712ba0cad243713146867ac2f986 Author: milan-elastic <[email protected]> Date: Wed May 1 15:43:52 2024 +0530 Squashed commit of the following: commit a17de73 Author: milan-elastic <“[email protected]”> Date: Wed May 1 15:29:41 2024 +0530 Squashed commit of the following: commit fccdb1f Author: milan-elastic <[email protected]> Date: Wed May 1 14:58:41 2024 +0530 add global filter on dashboard level for hadoop commit 686e49b Merge: 024d864 01201a7 Author: “milan-elastic” <“[email protected]”> Date: Wed May 1 11:38:59 2024 +0530 Merge branch 'main' of github.com:milan-elastic/integrations into mongodb-atlas-database-logs commit 01201a7 Author: Eric Forte <[email protected]> Date: Tue Apr 30 10:46:55 2024 -0400 [Security Rules] Update security rules package to v8.13.5 (elastic#9762) * [Security Rules] Update security rules package to v8.13.5 * Add changelog entry for 8.13.5 --------- Co-authored-by: protectionsmachine <[email protected]> commit c9d1f1b Author: Eric Forte <[email protected]> Date: Tue Apr 30 09:30:30 2024 -0400 [Security Rules] Update security rules package to v8.13.5-beta.1 (elastic#9758) * [Security Rules] Update security rules package to v8.13.5-beta.1 * Add changelog entry for 8.13.5-beta.1 --------- Co-authored-by: protectionsmachine <[email protected]> commit a79f813 Author: Tetiana Kravchenko <[email protected]> Date: Tue Apr 30 11:32:37 2024 +0200 [kubernetes] Remove deprecated fields, add missing status.last_terminated_reason metric (elastic#9736) * remove deprecated fields Signed-off-by: Tetiana Kravchenko <[email protected]> * Update changelog.yml * add missing metric: last_terminated_reason; update description of the status.reason field Signed-off-by: Tetiana Kravchenko <[email protected]> --------- Signed-off-by: Tetiana Kravchenko <[email protected]> commit b1627a3 Author: ShourieG <[email protected]> Date: Tue Apr 30 13:03:29 2024 +0530 [integrations][http_endpoint] - Converted HTTP Endpoint Integration to input type (elastic#9732) * converted http_endpoint to input package type * updated changelog * updated original event in sample event commit 3a9b508 Author: Lalit Satapathy <[email protected]> Date: Tue Apr 30 11:49:09 2024 +0530 Remove separate codeowners for system package kibana paths. (elastic#9731) commit c90e817 Author: Krishna Chaitanya Reddy Burri <[email protected]> Date: Tue Apr 30 11:32:17 2024 +0530 [Crowdstrike,Azure] Fix flaky tests with ECS fields (elastic#9738) * Fix flaky pipeline tests. * `azure.graphactivitylogs`: Add missing ECS field definitions. * `crowdstrike.falcon`: Update `geoip` processor to `destination` instead of `source`. commit ace8fb4 Author: Aliabbas Attarwala <[email protected]> Date: Mon Apr 29 16:37:23 2024 +0530 [O11y][AWS] Rally benchmark `aws.cloudtrail` (elastic#9448) commit d4e4aa4 Author: niraj-elastic <[email protected]> Date: Mon Apr 29 14:45:46 2024 +0530 [Apache] Update grok pattern for accepting user-identity (elastic#9632) * update grok pattern * update changelog * address review comments * address review comments Co-authored-by: muthu-mps <[email protected]> * address review comments * address review comment --------- Co-authored-by: muthu-mps <[email protected]> commit dce5699 Author: Mario Rodriguez Molins <[email protected]> Date: Mon Apr 29 10:33:19 2024 +0200 Enable publishing packages from integrations-publish pipeline (elastic#9712) Enable publishing packages from integrations-publish pipeline, and remove corresponding step from the main pipeline. commit c7bc530 Author: Chema Martínez <[email protected]> Date: Sat Apr 27 08:57:55 2024 +0200 [zscaler_zia] Fix mapping of source.ip and source.nat.ip (elastic#9727) * Fix mapping of source.ip and source.nat.ip * Update changelog * updated web datastream pipeline tests --------- Co-authored-by: Shourie Ganguly <[email protected]> commit 4750ea8 Author: Mario Rodriguez Molins <[email protected]> Date: Fri Apr 26 13:09:53 2024 +0200 [nginx] Update nginx config to listen in ipv6 too (elastic#9720) commit 25b0988 Author: Mario Rodriguez Molins <[email protected]> Date: Fri Apr 26 10:45:03 2024 +0200 [Buildkite] Update filter to use api source (elastic#9717) commit 45327cf Author: Mario Rodriguez Molins <[email protected]> Date: Fri Apr 26 10:13:22 2024 +0200 [Buildkite] Update filter condition to allow just from webhook source (elastic#9714) commit 024d864 Author: milan-elastic <[email protected]> Date: Fri Apr 26 13:00:47 2024 +0530 add dashboard level filter for apache tomcat commit 1cb5fad Author: Dan Kortschak <[email protected]> Date: Fri Apr 26 16:23:35 2024 +0930 entityanalytics_ad: new package for Active Directory user collection (elastic#9485) commit 37c598f Author: CarsonHrusovsky <[email protected]> Date: Thu Apr 25 18:13:26 2024 -0500 [BBOT] New integration for Black Lantern Security scanner (elastic#9651) commit d13e474 Author: Mario Rodriguez Molins <[email protected]> Date: Thu Apr 25 11:55:39 2024 +0200 [Buildkite] Skip install package command in serverless builds for some packages (elastic#9686) commit 0c2198b Author: Mario Rodriguez Molins <[email protected]> Date: Thu Apr 25 11:41:42 2024 +0200 [Buildkite] Add retry suffix for logs (elastic#9703) commit d932e79 Author: Simon Kötting <[email protected]> Date: Thu Apr 25 07:35:45 2024 +0200 [Exchange Server] GA of Integration, Add Dashbord Panel Titles & System Tests (elastic#9560) * Add Dashboard Titles * Add Dashboard Titles * Change Version to GA * adjust PR in Changelog * Add System Tests to all datstreams * fix imap system test config * remove Folder structure out of system tests sample logs * Fix mapping * Add convert for inode field * specify numeric_keyword_fields in system tests commit dba2901 Author: Dan Kortschak <[email protected]> Date: Thu Apr 25 10:21:30 2024 +0930 rapid7_insightvm: canonicalize host.name to lower case and map subdomain to host.hostname (elastic#9665) commit 4284262 Author: Panos Koutsovasilis <[email protected]> Date: Wed Apr 24 20:34:13 2024 +0300 fix(fim): add auto option for backend and make it the default one (elastic#9702) commit c563bb3 Author: Panos Koutsovasilis <[email protected]> Date: Wed Apr 24 19:40:04 2024 +0300 [juniper_netscreen]: include log.file.device_id and log.file.inode in base-fields (elastic#9658) * fix(juniper_netscreen): include log.file.device_id and log.file.inode in base-fields.yml * fix(juniper_netscreen): update README.md commit f187d0d Author: Panos Koutsovasilis <[email protected]> Date: Wed Apr 24 19:11:28 2024 +0300 [juniper_junos]: include log.file.device_id and log.file.inode in base-fields (elastic#9657) * fix(juniper_junos): include log.file.device_id and log.file.inode in base-fields.yml * fix(juniper_junos): update README.md
milan-elastic · May 1, 2024 · 8cf8f03 · 8cf8f03
1 parent 4e459d1
commit 8cf8f03
Show file tree

Hide file tree

Showing 165 changed files with 11,855 additions and 5,881 deletions.
diff --git a/.buildkite/pipeline.publish.yml b/.buildkite/pipeline.publish.yml
@@ -37,7 +37,8 @@ steps:
       memory: "8G"
     env:
       ARTIFACTS_FOLDER: "artifacts-to-sign"
-      DRY_RUN: "true"
+      # by default it will publish packages
+      DRY_RUN: "${DRY_RUN:-false}"
     depends_on:
       - step: "check"
         allow_failure: false

diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml
@@ -1,5 +1,4 @@
 # yaml-language-server: $schema=https://raw.githubusercontent.com/buildkite/pipeline-schema/main/schema.json
-
 env:
   SETUP_GVM_VERSION: "v0.5.2"
   LINUX_AGENT_IMAGE: "golang:${GO_VERSION}"
@@ -29,22 +28,6 @@ steps:
       cpu: "8"
       memory: "4G"
 
-  - label: ":package: Build packages"
-    key: "build-packages"
-    command: ".buildkite/scripts/build_packages.sh"
-    agents:
-      image: "${LINUX_AGENT_IMAGE}"
-      cpu: "8"
-      memory: "8G"
-    env:
-      ARTIFACTS_FOLDER: "artifacts-to-sign"
-      DRY_RUN: "false"
-    depends_on:
-      - step: "check"
-        allow_failure: false
-    artifact_paths:
-      - artifacts-to-sign/*.zip
-
   - label: "Trigger integrations"
     key: "test-integrations"
     command: ".buildkite/scripts/trigger_integrations_in_parallel.sh"

diff --git a/.buildkite/scripts/common.sh b/.buildkite/scripts/common.sh
@@ -309,7 +309,6 @@ create_kind_cluster() {
     kind create cluster --config "${WORKSPACE}/kind-config.yaml" --image "kindest/node:${K8S_VERSION}"
 }
 
-
 delete_kind_cluster() {
     echo "--- Delete kind cluster"
     kind delete cluster || true
@@ -410,7 +409,6 @@ is_package_excluded() {
     return 1
 }
 
-
 is_supported_capability() {
     if [ "${SERVERLESS_PROJECT}" == "" ]; then
         return 0
@@ -755,6 +753,19 @@ build_zip_package() {
     return 0
 }
 
+skip_installation_step() {
+    local package=$1
+    if ! is_serverless ; then
+        return 1
+    fi
+
+    if [[ "$package" == "security_detection_engine" ]]; then
+        return 0
+    fi
+
+    return 1
+}
+
 install_package() {
     local package=$1
     echo "Install package: ${package}"
@@ -814,10 +825,13 @@ run_tests_package() {
         fi
     fi
 
-    echo "--- [${package}] test installation"
-    if ! install_package "${package}" ; then
-        return 1
+    if ! skip_installation_step "${package}" ; then
+        echo "--- [${package}] test installation"
+        if ! install_package "${package}" ; then
+            return 1
+        fi
     fi
+
     echo "--- [${package}] run test suites"
     if is_serverless; then
         if ! test_package_in_serverless "${package}" ; then
@@ -877,6 +891,10 @@ upload_safe_logs_from_package() {
     fi
 
     local package=$1
+    local retry_count="${BUILDKITE_RETRY_COUNT:-"0"}"
+    if [[ "${retry_count}" -ne 0 ]]; then
+        package="${package}_retry_${retry_count}"
+    fi
     local build_directory=$2
 
     local parent_folder="insecure-logs"

diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -97,6 +97,7 @@
 /packages/azure_metrics/data_stream/storage_account @elastic/obs-ds-hosted-services
 /packages/barracuda @elastic/security-service-integrations
 /packages/barracuda_cloudgen_firewall @elastic/security-service-integrations
+/packages/bbot @elastic/security-service-integrations
 /packages/beaconing @elastic/ml-ui @elastic/sec-applied-ml
 /packages/beat @elastic/stack-monitoring
 /packages/bitdefender @elastic/security-service-integrations
@@ -146,6 +147,7 @@
 /packages/elastic_package_registry @elastic/ecosystem
 /packages/elasticsearch @elastic/stack-monitoring
 /packages/enterprisesearch @elastic/stack-monitoring
+/packages/entityanalytics_ad @elastic/security-service-integrations
 /packages/entityanalytics_entra_id @elastic/security-service-integrations
 /packages/entityanalytics_okta @elastic/security-service-integrations
 /packages/eset_protect @elastic/security-service-integrations
@@ -302,7 +304,6 @@
 /packages/system/changelog.yml @elastic/obs-infraobs-integrations @elastic/sec-linux-platform @elastic/sec-windows-platform
 /packages/system/data_stream/auth @elastic/sec-windows-platform
 /packages/system/data_stream/security @elastic/sec-linux-platform @elastic/sec-windows-platform
-/packages/system/kibana @elastic/elastic-agent-data-plane @elastic/kibana-visualizations
 /packages/system/manifest.yml @elastic/obs-infraobs-integrations @elastic/sec-linux-platform @elastic/sec-windows-platform
 /packages/system_audit @elastic/sec-linux-platform
 /packages/tanium @elastic/security-service-integrations

diff --git a/catalog-info.yaml b/catalog-info.yaml
@@ -50,7 +50,7 @@ spec:
         build_tags: false
         filter_enabled: true
         filter_condition: >-
-          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null)
+          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null && build.source == 'api')
       repository: elastic/integrations
       cancel_intermediate_builds: true
       cancel_intermediate_builds_branch_filter: '!main !backport-*'
@@ -99,7 +99,7 @@ spec:
         build_tags: false
         filter_enabled: true
         filter_condition: >-
-          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null)
+          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null && build.source == 'api')
       repository: elastic/integrations
       cancel_intermediate_builds: true
       cancel_intermediate_builds_branch_filter: '!main'
@@ -146,7 +146,7 @@ spec:
         build_tags: false
         filter_enabled: true
         filter_condition: >-
-          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null)
+          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null && build.source == 'api')
       repository: elastic/integrations
       cancel_intermediate_builds: true
       cancel_intermediate_builds_branch_filter: '!main'
@@ -188,7 +188,7 @@ spec:
         build_tags: false
         filter_enabled: true
         filter_condition: >-
-          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null)
+          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null && build.source == 'api')
       repository: elastic/integrations
       cancel_intermediate_builds: true
       cancel_intermediate_builds_branch_filter: '!main'
@@ -232,7 +232,7 @@ spec:
         build_tags: false
         filter_enabled: true
         filter_condition: >-
-          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null)
+          build.pull_request.id == null || (build.creator.name == 'elasticmachine' && build.pull_request.id != null && build.source == 'api')
       repository: elastic/integrations
       cancel_intermediate_builds: true
       cancel_intermediate_builds_branch_filter: '!main !backport-*'

diff --git a/packages/apache/changelog.yml b/packages/apache/changelog.yml
@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "1.17.1"
+  changes:
+    - description: Update grok for accepting user-identity.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/9632
 - version: "1.17.0"
   changes:
     - description: Limit request tracer log count to five.

diff --git a/packages/apache/data_stream/access/_dev/test/pipeline/test-access-basic.log b/packages/apache/data_stream/access/_dev/test/pipeline/test-access-basic.log
@@ -7,4 +7,5 @@ monitoring-server - - [29/May/2017:19:02:48 +0000] "GET /status HTTP/1.1" 200 61
 monitoring-server - - [29/May/2017:19:02:48 +0000] "GET /A%20Beka%20G1%20Howe/029_AND_30/15%20reading%20elephants.mp4 HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2" X-Forwarded-For="-"
 89.160.20.112 - - [29/May/2017:19:02:48 +0000] "GET /A%20Beka%20G1%20Howe/029_AND_30/15%20reading%20elephants.mp4 HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2" X-Forwarded-For="10.0.0.2,10.0.0.1"
 2a02:cf40:add:4002:91f2:a9b2:e09a:6fc6 - - [29/May/2017:19:02:48 +0000] "GET /A%20Beka%20G1%20Howe/029_AND_30/15%20reading%20elephants.mp4 HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2" X-Forwarded-For="10.225.192.17, 10.2.2.121"
-monitoring-server - - [17/May/2022:21:41:43 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.79.1" X-Forwarded-For="192.168.0.2"
+monitoring-server - - [17/May/2022:21:41:43 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.79.1" X-Forwarded-For="192.168.0.2"
+127.0.0.1 user-identity frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326
diff --git a/packages/apache/data_stream/access/_dev/test/pipeline/test-access-basic.log-expected.json b/packages/apache/data_stream/access/_dev/test/pipeline/test-access-basic.log-expected.json
@@ -15,7 +15,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409634501Z",
+                "ingested": "2024-04-26T05:46:25.296250288Z",
                 "kind": "event",
                 "original": "::1 - - [26/Dec/2016:16:16:29 +0200] \"GET /favicon.ico HTTP/1.1\" 404 209",
                 "outcome": "failure"
@@ -63,7 +63,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409644668Z",
+                "ingested": "2024-04-26T05:46:25.296284705Z",
                 "kind": "event",
                 "original": "192.168.33.1 - - [26/Dec/2016:16:22:13 +0000] \"GET /hello HTTP/1.1\" 404 499 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:50.0) Gecko/20100101 Firefox/50.0\"",
                 "outcome": "failure"
@@ -124,7 +124,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409645876Z",
+                "ingested": "2024-04-26T05:46:25.296289743Z",
                 "kind": "event",
                 "original": "::1 - - [26/Dec/2016:16:16:48 +0200] \"-\" 408 -",
                 "outcome": "failure"
@@ -160,7 +160,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409646876Z",
+                "ingested": "2024-04-26T05:46:25.296293311Z",
                 "kind": "event",
                 "original": "172.17.0.1 - - [29/May/2017:19:02:48 +0000] \"GET /stringpatch HTTP/1.1\" 404 612 \"-\" \"Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2\" \"-\"",
                 "outcome": "failure"
@@ -221,7 +221,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409647793Z",
+                "ingested": "2024-04-26T05:46:25.296296691Z",
                 "kind": "event",
                 "original": "monitoring-server - - [29/May/2017:19:02:48 +0000] \"GET /status HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2\" \"-\"",
                 "outcome": "success"
@@ -282,7 +282,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409648793Z",
+                "ingested": "2024-04-26T05:46:25.296300048Z",
                 "kind": "event",
                 "original": "127.0.0.1 - - [02/Feb/2019:05:38:45 +0100] \"-\" 408 152 \"-\" \"-\"",
                 "outcome": "failure"
@@ -331,7 +331,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409649793Z",
+                "ingested": "2024-04-26T05:46:25.296303835Z",
                 "kind": "event",
                 "original": "monitoring-server - - [29/May/2017:19:02:48 +0000] \"GET /A%20Beka%20G1%20Howe/029_AND_30/15%20reading%20elephants.mp4 HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2\" X-Forwarded-For=\"-\"",
                 "outcome": "success"
@@ -398,7 +398,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409650668Z",
+                "ingested": "2024-04-26T05:46:25.296310193Z",
                 "kind": "event",
                 "original": "89.160.20.112 - - [29/May/2017:19:02:48 +0000] \"GET /A%20Beka%20G1%20Howe/029_AND_30/15%20reading%20elephants.mp4 HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2\" X-Forwarded-For=\"10.0.0.2,10.0.0.1\"",
                 "outcome": "success"
@@ -486,7 +486,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409651543Z",
+                "ingested": "2024-04-26T05:46:25.296313609Z",
                 "kind": "event",
                 "original": "2a02:cf40:add:4002:91f2:a9b2:e09a:6fc6 - - [29/May/2017:19:02:48 +0000] \"GET /A%20Beka%20G1%20Howe/029_AND_30/15%20reading%20elephants.mp4 HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20120716 Firefox/15.0a2\" X-Forwarded-For=\"10.225.192.17, 10.2.2.121\"",
                 "outcome": "success"
@@ -564,7 +564,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.409652876Z",
+                "ingested": "2024-04-26T05:46:25.296316938Z",
                 "kind": "event",
                 "original": "monitoring-server - - [17/May/2022:21:41:43 +0000] \"GET / HTTP/1.1\" 200 45 \"-\" \"curl/7.79.1\" X-Forwarded-For=\"192.168.0.2\"",
                 "outcome": "success"
@@ -607,6 +607,55 @@
                 "original": "curl/7.79.1",
                 "version": "7.79.1"
             }
+        },
+        {
+            "@timestamp": "2000-10-10T20:55:36.000Z",
+            "apache": {
+                "access": {
+                    "identity": "user-identity",
+                    "remote_addresses": [
+                        "127.0.0.1"
+                    ]
+                }
+            },
+            "ecs": {
+                "version": "8.5.1"
+            },
+            "event": {
+                "category": "web",
+                "created": "2020-04-28T11:07:58.223Z",
+                "ingested": "2024-04-26T05:46:25.296320274Z",
+                "kind": "event",
+                "original": "127.0.0.1 user-identity frank [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 200 2326",
+                "outcome": "success"
+            },
+            "http": {
+                "request": {
+                    "method": "GET"
+                },
+                "response": {
+                    "body": {
+                        "bytes": 2326
+                    },
+                    "status_code": 200
+                },
+                "version": "1.0"
+            },
+            "source": {
+                "address": "127.0.0.1",
+                "ip": "127.0.0.1"
+            },
+            "tags": [
+                "preserve_original_event"
+            ],
+            "url": {
+                "extension": "gif",
+                "original": "/apache_pb.gif",
+                "path": "/apache_pb.gif"
+            },
+            "user": {
+                "name": "frank"
+            }
         }
     ]
 }
diff --git a/packages/apache/data_stream/access/_dev/test/pipeline/test-access-darwin.log-expected.json b/packages/apache/data_stream/access/_dev/test/pipeline/test-access-darwin.log-expected.json
@@ -15,7 +15,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.483539043Z",
+                "ingested": "2024-04-26T05:46:25.447843628Z",
                 "kind": "event",
                 "original": "::1 - - [26/Dec/2016:16:16:28 +0200] \"GET / HTTP/1.1\" 200 45",
                 "outcome": "success"
@@ -62,7 +62,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.483550209Z",
+                "ingested": "2024-04-26T05:46:25.447895323Z",
                 "kind": "event",
                 "original": "::1 - - [26/Dec/2016:16:16:29 +0200] \"GET /favicon.ico HTTP/1.1\" 404 209",
                 "outcome": "failure"
@@ -110,7 +110,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.483551501Z",
+                "ingested": "2024-04-26T05:46:25.447905030Z",
                 "kind": "event",
                 "original": "::1 - - [26/Dec/2016:16:16:48 +0200] \"-\" 408 -",
                 "outcome": "failure"
@@ -146,7 +146,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.483552501Z",
+                "ingested": "2024-04-26T05:46:25.447912585Z",
                 "kind": "event",
                 "original": "89.160.20.156 - - [26/Dec/2016:18:23:35 +0200] \"GET / HTTP/1.1\" 200 45",
                 "outcome": "success"
@@ -211,7 +211,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.483553418Z",
+                "ingested": "2024-04-26T05:46:25.447919912Z",
                 "kind": "event",
                 "original": "89.160.20.156 - - [26/Dec/2016:18:23:41 +0200] \"GET /notfound HTTP/1.1\" 404 206",
                 "outcome": "failure"
@@ -276,7 +276,7 @@
             "event": {
                 "category": "web",
                 "created": "2020-04-28T11:07:58.223Z",
-                "ingested": "2022-12-08T15:09:52.483554501Z",
+                "ingested": "2024-04-26T05:46:25.447927217Z",
                 "kind": "event",
                 "original": "89.160.20.156 - - [26/Dec/2016:18:23:45 +0200] \"GET /hmm HTTP/1.1\" 404 201",
                 "outcome": "failure"