Home

Splunk AWS GDI Toolkit

The Splunk AWS GDI Toolkit is a set of resources designed to help organizations easily ingest data of AWS into Splunk to improve visibility, observability, and monitoring from AWS accounts. The toolkit consists of a series of CloudFormationt templates to enable service and get the data from those services into Splunk. These templates can also serve as a reference or starting point for organizations looking to implement use-cases that the Splunk AWS GDI Toolkit doesn't exactly meet.

The goals of this toolkit are to:

• Augment the Splunk Cloud Data Manager with CloudFormation templates to enable pre-requisites for getting data in from single AWS accounts.
• Follow AWS best practices for getting data in from multiple AWS accounts utilizing AWS Organizations.
• Combine the best of Splunk Cloud Data Manager, Splunk Trumpet, and Grand Central.

PRs are open, and feel free to reach out to me over the Splunk Usergroups Slack if you have questions, comments, or concerns!

Use Case Library

• Send CloudTrail, ELB Access, and/or VPC Flow logs to Splunk - https://github.com/pdreeves/splunk-aws-gdi-toolkit/wiki/S3-SQS-Lambda-Firehose-Method
• Send events from a custom sourcetype from S3 buckets to Splunk - https://github.com/pdreeves/splunk-aws-gdi-toolkit/wiki/S3-SQS-Lambda-Firehose-Method
• Implement SCDM pre-requistes using CloudFormation - https://github.com/pdreeves/splunk-aws-gdi-toolkit/wiki/Single-AWS-Account-Ingestion-with-SCDM

Wiki Index

• Single AWS Account Ingestion with SCDM
• S3-SQS-Lambda-Firehose Method
• Splunk Cloud Data Manager

When to use Splunk Data Manager or Splunk AWS GDI Toolkit

Both the Splunk Cloud Data Manager (SCDM) and the CloudFormation templates in this toolkit can be used to pull in some of the same events form AWS, like CloudTrail events. The SCDM prioritizes event latency, while the CloudFormation templates here prioritize following AWS best-practices and cost.

Mermaid code:

graph TD;
	start([Start here])
	usingOrg{Are you using AWS Organization and/or Landing Zones?}
	costOrLtency{Do you want to minimize lowering cost or data latency?}
	SCDM([Use Data Manager, referencing the CloudFormation Templates here for prerequisites])
	sAWSGDITK([Use the CoudFormation Templates here])
	start-->usingOrg
	usingOrg-->|No|costOrLtency
	costOrLtency-->|Minimize latency|SCDM
	costOrLtency-->|Minimize cost|sAWSGDITK
	usingOrg-->|Yes|sAWSGDITK

Loading