Skip to content

Commit

Permalink
[Amazon Security Lake] - OCSF v1.1 update with major refactor & addin…
Browse files Browse the repository at this point in the history
…g support for dynamic template and mappings & system tests (#10405)

* Added  support for OCSF v1.1.0. with major pipeline refactor and dynamic mapping support.
  • Loading branch information
ShourieG authored Oct 23, 2024
1 parent 23776ce commit 4d6643d
Show file tree
Hide file tree
Showing 124 changed files with 34,099 additions and 20,744 deletions.
4 changes: 3 additions & 1 deletion packages/amazon_security_lake/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The Amazon Security Lake integration can be used in two different modes to colle

## Compatibility

This module follows the latest OCSF Schema Version **v1.0.0**.
This module follows the OCSF Schema Version **v1.1.0**.

## Data streams

Expand All @@ -19,6 +19,8 @@ The Amazon Security Lake integration collects logs from both [Third-party servic
### **NOTE**:
- The Amazon Security Lake integration supports events collected from [AWS services](https://docs.aws.amazon.com/security-lake/latest/userguide/internal-sources.html) and [third-party services](https://docs.aws.amazon.com/security-lake/latest/userguide/custom-sources.html).

- Due to the nature and structure of the OCSF schema, this integration has limitations on how deep the mappings run. Some important objects like 'Actor', 'User' and 'Product' have more fleshed-out mappings compared to others which get flattened after the initial 2-3 levels of nesting to keep them maintainable and stay within field mapping [limits](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-settings-limit.html). This will evolve as needed.

## Requirements

- Elastic Agent must be installed.
Expand Down
5 changes: 5 additions & 0 deletions packages/amazon_security_lake/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "2.0.0"
changes:
- description: Updated to support OCSF v1.1.0. with major pipeline rework and dynamic mapping support.
type: enhancement
link: https://github.com/elastic/integrations/pull/10405
- version: "1.5.0"
changes:
- description: Re-added SQS notification settings which were removed due to a prior update error.
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
- name: ocsf
type: group
fields:
- name: api
type: group
description: Describes details about a typical API (Application Programming Interface) call.
fields:
- name: operation
type: keyword
description: Verb/Operation associated with the request.
- name: group
type: group
description: The information pertaining to the API group.
fields:
- name: desc
type: text
description: The group description.
- name: domain
type: keyword
description: The domain where the group is defined. For example, the LDAP or Active Directory domain.
- name: name
type: keyword
description: The group name.
- name: privileges
type: keyword
description: The group privileges.
- name: type
type: keyword
description: The type of the group or account.
- name: uid
type: keyword
description: The unique identifier of the group. For example, for Windows events this is the security identifier (SID) of the group.
- name: request
type: group
description: Details pertaining to the API request.
fields:
- name: uid
type: keyword
description: The unique request identifier.
- name: containers
type: group
description: When working with containerized applications, the set of containers which write to the standard the output of a particular logging driver.
fields:
- name: hash
type: flattened
description: Commit hash of image created for docker or the SHA256 hash of the container.
- name: image
type: flattened
description: The container image used as a template to run the container.
- name: name
type: keyword
description: The container name.
- name: network_driver
type: keyword
description: The network driver used by the container. For example, bridge, overlay, host, none, etc.
- name: orchestrator
type: keyword
description: The orchestrator managing the container, such as ECS, EKS, K8s, or OpenShift.
- name: pod_uuid
type: keyword
description: The unique identifier of the pod (or equivalent) that the container is executing on.
- name: runtime
type: keyword
description: The backend running the container, such as containerd or cri-o.
- name: size
type: integer
description: The size of the container image.
- name: tag
type: keyword
description: The tag used by the container. It can indicate version, format, OS.
- name: uid
type: keyword
description: The full container unique identifier for this instantiation of the container.
- name: data
type: flattened
description: The additional data that is associated with the api request.
- name: flags
type: keyword
description: The list of communication flags, normalized to the captions of the flag_ids values.
- name: response
type: group
description: Details pertaining to the API response.
fields:
- name: code
type: integer
description: The numeric response sent to a request.
- name: containers
type: group
description: When working with containerized applications, the set of containers which write to the standard the output of a particular logging driver.
fields:
- name: hash
type: flattened
description: Commit hash of image created for docker or the SHA256 hash of the container.
- name: image
type: flattened
description: The container image used as a template to run the container.
- name: name
type: keyword
description: The container name.
- name: network_driver
type: keyword
description: The network driver used by the container. For example, bridge, overlay, host, none, etc.
- name: orchestrator
type: keyword
description: The orchestrator managing the container, such as ECS, EKS, K8s, or OpenShift.
- name: pod_uuid
type: keyword
description: The unique identifier of the pod (or equivalent) that the container is executing on.
- name: runtime
type: keyword
description: The backend running the container, such as containerd or cri-o.
- name: size
type: integer
description: The size of the container image.
- name: tag
type: keyword
description: The tag used by the container. It can indicate version, format, OS.
- name: uid
type: keyword
description: The full container unique identifier for this instantiation of the container.
- name: data
type: flattened
description: The additional data that is associated with the api response.
- name: error
type: keyword
description: Error Code.
- name: error_message
type: text
description: Error Message.
- name: flags
type: keyword
description: The list of communication flags, normalized to the captions of the flag_ids values.
- name: message
type: text
description: The description of the event/finding, as defined by the source.
- name: service
type: group
description: The information pertaining to the API service.
fields:
- name: labels
type: keyword
description: The list of labels associated with the service.
- name: name
type: keyword
description: The name of the service.
- name: uid
type: keyword
description: The unique identifier of the service.
- name: version
type: keyword
description: The version of the service.
- name: version
type: keyword
description: The version of the API service.
Loading

0 comments on commit 4d6643d

Please sign in to comment.