[servicenow_cmdb] Initial release of ServiceNow CMDB

.github/CODEOWNERS

@@ -316,6 +316,7 @@
 /packages/security_detection_engine @elastic/protections
 /packages/sentinel_one @elastic/security-service-integrations
 /packages/sentinel_one_cloud_funnel @elastic/security-service-integrations
+/packages/servicenow_cmdb @elastic/security-service-integrations
 /packages/slack @elastic/security-service-integrations
 /packages/snort @elastic/sec-deployment-and-devices
 /packages/snyk @elastic/security-service-integrations

packages/servicenow_cmdb/_dev/build/build.yml

@@ -0,0 +1,3 @@
+dependencies:
+  ecs:
+    reference: [email protected]

packages/servicenow_cmdb/_dev/build/docs/README.md

@@ -0,0 +1,147 @@
+# ServiceNow Configuration Management Database
+
+## Overview
+
+The [ServiceNow CMDB](https://www.servicenow.com/products/servicenow-platform/configuration-management-database.html#benefits) integration helps organizations keep track of all their IT assets: computers, software and network devices, and shows how these items are related to each other. By having this information in one place, it is easier to manage changes, fix problems, and ensure everything is compliant with regulations. Essentially, it's a way to stay organized and know exactly what IT resources are available and how they work together.
+
+The ServiceNow CMDB integration can be used in three different modes to collect logs:
+- AWS S3 polling mode: ServiceNow CMDB writes data to S3, and Elastic Agent polls the S3 bucket by listing its contents and reading new files. Refer to the [ServiceNow documentation](https://www.servicenow.com/community/now-platform-forum/aws-s3-integration-with-servicenow/td-p/1121852) for how to integrate AWS S3 with ServiceNow for retrieving logs into an S3 bucket.
+- AWS S3 SQS mode: ServiceNow CMDB writes data to S3; S3 sends a notification of a new object to SQS; the Elastic Agent receives the notification from SQS and then reads the S3 object. Multiple agents can be used in this mode.
+- REST API mode: ServiceNow CMDB offers table APIs to retrieve data from its tables; the Elastic Agent polls these APIs to list their contents and read any new data. Visit this [link](https://developer.servicenow.com/dev.do#!/reference/api/washingtondc/rest/c_TableAPI#table-GET) for additional information about REST APIs.
+
+## Compatibility
+
+This module has been tested against the latest (updated Aug 1, 2024) ServiceNow CMDB API.
+
+## Data streams
+
+The ServiceNow integration supports both custom tables and the default tables offered by ServiceNow. Additionally, both types of tables are included in the data stream labeled `event`.
+
+Below is a list of the default ones.
+
+- **alm_hardware**
+- **change_request**
+- **change_task**
+- **cmdb**
+- **cmdb_ci**
+- **cmdb_ci_app_server**
+- **cmdb_ci_appl**
+- **cmdb_ci_business_app**
+- **cmdb_ci_computer**
+- **cmdb_ci_db_instance**
+- **cmdb_ci_esx_server**
+- **cmdb_ci_hardware**
+- **cmdb_ci_hyper_v_server**
+- **cmdb_ci_infra_service**
+- **cmdb_ci_linux_server**
+- **cmdb_ci_server**
+- **cmdb_ci_service**
+- **cmdb_ci_vm**
+- **cmdb_ci_win_server**
+- **cmdb_rel_ci**
+- **cmn_department**
+- **cmn_location**
+- **incident**
+- **kb_knowledge**
+- **problem**
+- **sc_req_item**
+- **sys_user**
+- **sys_user_grmember**
+- **sys_user_group**
+- **task_ci**
+
+**Note**:
+
+1. This integration currently supports ECS mapping for default ServiceNow tables listed above. For custom tables created by users, ECS mapping is not automatically provided. If you want to add mappings for custom tables, please refer to this [tutorial guide](https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html).
+2. For each table, a tag will be added based on the name of the table from which data is fetched.
+
+## Requirements
+
+- Elastic Agent must be installed.
+- You can install only one Elastic Agent per host.
+- Elastic Agent is required to stream data through the REST API or AWS S3/SQS and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines.
+
+### Installing and managing an Elastic Agent:
+
+You have a few options for installing and managing an Elastic Agent:
+
+### Install a Fleet-managed Elastic Agent (recommended):
+
+With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.
+
+### Install Elastic Agent in standalone mode (advanced users):
+
+With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.
+
+### Install Elastic Agent in a containerized environment:
+
+You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes.
+
+There are some minimum requirements for running Elastic Agent. For more information, refer to the Elastic Agent [installation guide](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).
+
+## Setup
+
+### To collect logs through REST API, follow the below steps:
+
+- Your instance URL will serve as your base URL and will be formatted as follows: https://<instancename>.service-now.com
+- Additionally, the username and password you use to log into your instance will be required to fetch logs in our integration.
+
+### To collect logs through AWS S3, follow the below steps:
+
+- With an AWS S3 bucket that has been set up, you can configure it with ServiceNow CMDB by integrating it using your AWS S3 credentials.
+
+### To collect logs through AWS SQS, follow the below steps:
+
+1. Assuming you've already set up a connection to push data into the AWS bucket you can follow the below steps; if not, see the section above.
+2. To set up an SQS queue, follow "Step 1: Create an Amazon SQS Queue" as described in the [Amazon S3 user guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ways-to-add-notification-config-to-bucket.html).
+   - While creating an access policy, use the bucket name configured to create a connection for AWS S3 in ServiceNow CMDB.
+3. Configure event notifications for an S3 bucket according to the [Amazon S3 user guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-event-notifications.html).
+   - While creating `event notification` select the event type as s3:ObjectCreated:*, destination type SQS Queue, and select the queue name created in Step 2.
+
+### Time Zone Selection:
+- In the Data Collection section, use the `Timezone of ServiceNow Instance` dropdown to select your preferred timezone. The `.value` field for date data will always be in UTC, while the `.display_value` field can reflect your instance's selected timezone. The system default is set to America/Los_Angeles, but you can change this in your ServiceNow profile settings.
+- Steps to See/Update the timezone in ServiceNow Instance:
+  1. Click the user icon in the top-right corner of the ServiceNow interface.
+  2. Select Profile from the dropdown menu.
+  3. In your Profile settings, locate the Timezone option.
+
+### Enabling the integration in Elastic:
+
+1. In Kibana go to Management > Integrations.
+2. In "Search for integrations" search bar, type ServiceNow CMDB.
+3. Click on the "ServiceNow CMDB" integration from the search results.
+4. Click on the "Add ServiceNow CMDB" button to add the integration.
+5. While adding the integration, if you want to collect logs via REST API, then you have to put the following details:
+   - API URL
+   - username
+   - password
+   - table name
+   - timezone
+   - collect logs via REST API toggled on
+
+   or if you want to collect logs via AWS S3, then you have to put the following details:
+   - access key id
+   - secret access key
+   - bucket arn
+   - timezone
+   - collect logs via S3 Bucket toggled on
+
+   or if you want to collect logs via AWS SQS, then you have to put the following details:
+   - access key id
+   - secret access key
+   - queue url
+   - timezone
+   - collect logs via S3 Bucket toggled off
+6. Click on "Save and Continue" to save the integration.
+
+## Logs Reference
+
+### Event
+
+This is the `Event` dataset.
+
+#### Example
+
+{{event "event"}}
+
+{{fields "event"}}

packages/servicenow_cmdb/_dev/deploy/docker/docker-compose.yml

@@ -0,0 +1,15 @@
+version: '2.3'
+services:
+  servicenow_cmdb:
+    image: docker.elastic.co/observability/stream:v0.15.0
+    hostname: servicenow_cmdb
+    ports:
+      - 8090
+    volumes:
+      - ./files:/files:ro
+    environment:
+      PORT: '8090'
+    command:
+      - http-server
+      - --addr=:8090
+      - --config=/files/config.yml