Home

Vault Spark is a message queue system that allows developers to use the Vault Java SDK to send and receive messages from durable queue. It can be used to allow for loosely coupled, asynchronous integration within a vault, between different vaults, or between a vault and an external system. This page describes the example use cases of the integration between a vault and an external 3rd party system along with sample code, that you can use with your vault to step through the code in the Vault Java SDK Debugger.

Introduction

The vsdk-spark-external-aws-sample project contains an example of how to send a spark messages from a vault to external Amazon Web Services (AWS) SQS queue for processing. The project will step through:

• Creation of an sample 3rd party AWS Application
  • Amazon Simple Queue Service (SQS) - store Spark messages in AWS for processing
  • Amazon API Gateway - point of entry from Vault to AWS
  • Amazon Lambda - verify, enqueue and process Spark messages
• Vault
  • External messaging - send a simple message from a source vault to AWS for processing

Business Use Case

Loan Approval

A user needs to request a loan for an item, which is done within Vault with the creation of a Loan Approval record. Basic information about the loan request must be entered such as the name of the person requiring the loan, the item, the amount and the loan duration. Once saved a request is sent to an external Finance System outside within AWS for approval or rejection based on the specified loan criteria. The loan approval record within Vault is then updated with an approval decision, and if approved with details of the loan.

Subsequent loan approval requests can be made for the same item via a Loan re-quote record action.

General Guidelines

Vault to External Setup

Prerequisites

1. An AWS account. If you don't have one sign up for an account at AWS Free Tier.

There are three main steps to setup this project for use.

1. Create AWS Application Services in the external system that will process the messages from vault.
2. Create Vault Connection record in the vault that links it to AWS.
3. Import Vault Packages to create necessary components and Spark Message Queues.
4. Create Vault Queue object in the vault for the Spark Messages.
5. Configure Public Key within the lambda function vsdkSparkSampleValidateAndEnqueueMessage for message signature verification.

The project contains a vault packages (VPK) in the "deploy-vpk" directory with the necessary Objects, Queues, and Vault Java SDK code.

Create AWS Application Services

Create AWS IAM Role

1. In the Amazon IAM console, click Roles in the menu on the left hand side of the page.
2. Click Create role.
3. Select AWS service as the type of trusted entity and then select the service Lambda.
4. Click Next: Permissions.
5. Add the following permissions policies by searching for the policy name and when it appears clicking the check mark next to it:
   • AWSLambdaBasicExecutionRole
   • AWSLambdaSQSQueueExecutionRole
   • AmazonSQSFullAccess
6. Click Next: Tags.
7. Click Next: Review.
8. For Role Name, enter SparkSampleAwsRole.
9. Click Create role.

Create SQS Queue vsdk-spark-sample-external-queue

1. In the Amazon SQS console, click Create New Queue.
2. Configure the new Queue:
   • For Queue Name, enter vsdk-spark-sample-external-queue.
   • For Region, choose your Region.
   • For type of queue, choose Standard Queue.
3. Select Quick-Create Queue.
4. With the vsdk-spark-sample-external-queue SQS queue selected, select the Permissions tab.
5. Click Add a permission.
6. Set the following values:
   • For Effect, check Allow.
   • For Principle, choose (click) the check mark next to Everybody (*).
   • For Actions, choose (click) the check mark next to the following:
     • DeleteMessage
     • GetQueueURL
     • ReceiveMessage
     • SendMessage
7. Select Add Permission.
8. With the vsdk-spark-sample-external-queue SQS queue selected, select the Details tab.
9. Record the URL displayed in the Details tab, as this will be needed later in configuring AWS in the section Create Lambda function vsdkSparkSampleValidateAndEnqueueMessage.

Create Lambda function vsdkSparkSampleValidateAndEnqueueMessage

First, you need to download the project

1. Clone or download the sample Maven project vSDK Spark AWS External Sample project from GitHub, and save on your local machine.
2. In the AWS Lambda console, click Create function.
3. Set the following values:
   • Check Author from scratch.
   • For Name, enter vsdkSparkSampleValidateAndEnqueueMessage.
   • For Runtime, select Node.js 8.10.
   • For Role, select Choose an existing role.
   • For Existing role, select SparkSampleAwsRole that was created earlier.
4. Click Create function.
5. Scroll to the Function code section and in the Code entry type dropdown, select Upload a .zip file.
6. Click Upload.
7. From the sample Maven project vSDK Spark AWS External Sample project downloaded in step 1, select the \aws-lambda-samples\vsdkSparkSampleValidateAndEnqueueMessage.zip file.
8. Click Save.
9. Scroll down to the Environment Variables section and add the following:
   • Key: VAULT_SAMPLE_SQS_QUEUE_URL; Value: URL for the queue recorded in the section Create SQS Queue vsdk-spark-sample-external-queue.
   • Key: VAULT_HOSTNAME; Value: {YourVault}, e.g. your-vault.veeva.com.
   • Key: VAULT_USER; Value: Your Vault Integration service account's username.
   • Key: VAULT_PASSWORD; Value: Your Vault Integration service account's password.
10. Scroll down to the Basic settings section and set the Timeout to 0 mins and 20 sec.
11. Click Save.
12. Click Functions in the breadcrumb trial at the top the page to return to the main functions page.

Create Lambda function vsdkSparkSampleProcessMessage

1. Whilst still in the AWS Lambda console, click Create function.
2. Set the following values:
   • Check Author from scratch.
   • For Name, enter vsdkSparkSampleProcessMessage.
   • For Runtime, select Python 3.7.
   • For Role, select Choose an existing role.
   • For Existing role, select SparkSampleAwsRole that was created earlier.
3. Click Create function.
4. Scroll to the Function code section and in the Code entry type dropdown, select Upload a .zip file.
5. Click Upload.
6. From the sample Maven project vSDK Spark AWS External Sample project downloaded in step 1, select the \aws-lambda-samples\vsdkSparkSampleProcessMessage.zip file.
7. Click Save.
8. Scroll down to the Environment Variables section and add the following:
   • Key: CLIENT_ID; Value: vsdk-sparksample-aws-process-message.
   • Key: VAULT_API_BURST_LIMIT_CUTOFF; Value: 200.
   • Key: VAULT_VQL_PAGE_LIMIT; Value: 500.
   • Key: VAULT_REST_API_BASE_URL; Value: https://{YourVault}/api/v18.3/.
   • Key: VAULT_USER; Value: Your Vault Integration service account's username.
   • Key: VAULT_PASSWORD; Value: Your Vault Integration service account's password.
9. Scroll down to the Basic settings section and set the Timeout to 0 mins and 30 sec.
10. Click Save.
11. Scroll up to the Designer section and in the Add triggers section on the left hand side of the screen, select SQS
12. In the Configure triggers section, select the SQS Queue vsdk-spark-sample-external-queue.
13. Make sure Enable trigger is checked and click Add.

Note: Now this trigger is enabled background processing will be running in your AWS environment which will counts towards your billable resources. To stop this at any time, simply disable this trigger and press save.

Create API Gateway vsdk-spark-sample-external-queue

1. Whilst still in the API Gateway console, click Create API.
2. Configure the integration point:
   • For protocal, choose REST.
   • For type, choose New API.
   • For API name, enter vSdkSparkSampleAPIGateway.
3. Select Create API.
4. Select Resources in the navigation pane, open the Actions menu, and then choose Create Resource.
5. For Resource Name, enter message, and then choose Create Resource.
6. Open Actions, and then choose Create Method.
7. From the drop-down menu under /message, choose POST, and then choose (click) the check mark.
8. Configure the integration point:
   • For Integration type, choose Lambda Function.
   • For Lambda Region, choose your Region.
   • For Lambda Function, enter vsdkSparkSampleValidateAndEnqueueMessage.
   • For User Default Timeout, choose (click) the check mark.
9. Select Save.
10. On the message Add Permission to Lambda Function click OK.
11. Select Integration Request.
12. Scroll to the bottom of the page and expand Mapping Templates.
13. Click Add mapping template.
14. For Content-Type, enter application/json, and then choose (click) the check mark.
15. On the message Change passthrough behavior, click Yes, secure this integration.
16. Paste the following code into the template body. { "method": "$context.httpMethod", "body" : $input.json('$'), "headers": { #foreach($param in $input.params().header.keySet()) "$param": "$util.escapeJavaScript($input.params().header.get($param))" #if($foreach.hasNext),#end #end } }
17. Select Save.
18. From the Actions menu, choose Deploy API.
19. For Deployment stage, choose [New Stage], and then enter a Stage name demo.
20. Choose Deploy.
21. Record the URL displayed next to Invoke URL at the top of the Stage Editor, as this will be needed later in configuring Vault in the step Create Connection.

The creation of API Gateway vsdk-spark-sample-external-queue is now complete.

Create Vault Connection

The Connection object is used to create records that define connections between different vaults or between a vault and an external system.

In this use case, we will create Vault to External records to link a source vault to an external system.

In vault

1. Log in and navigate to Admin > Business Admin > Connections and click Create
2. Choose the External connection type. Select Continue.
3. Enter AWS Queue Sample API Gateway in the Name field.
4. Enter vsdk_aws_queue_sample_api_gateway in the API Name field.
5. Enter your API Gateway Invoke URL in the URL field. This will be the value you were asked to record when creating the AWS API Gateway earlier and will look something like https://12abcdefg2.execute-api.eu-west-2.amazonaws.com/demo. Important - make sure /message is not included at the end of the URL.
6. Select Save.

The connection is now ready for use!

Import Vault Packages

The VPKs need to be deployed to your vault prior to debugging these use cases.

First, you need to retrieve the project

1. Retrieve the sample Maven project vSDK Spark AWS External Sample project that was previously downloaded from GitHub in section Create Lambda function vsdkSparkSampleValidateAndEnqueueMessage.
2. Run through the Getting Started guide to set up your deployment environment.

Next, deploy the required code and components to your vault

1. In the vault, log in and navigate to Admin > Deployment > Inbound Packages and click Import:

   Deploy vault components and code: Select the \deploy-vpk\vault\vsdk-spark-external-aws-sample-components\vsdk-spark-external-aws-sample-components.vpk file.

2. From the Actions menu, select Review & Deploy. Vault displays a list of all components in the package.

3. Review the prompts to deploy the package. You will receive an email when the deployment is complete.

Once the package has been deployed, you will want to review the configuration and understand the normal behavior so you can observe the effects of the sample trigger code.

Below is a rundown of the included components.

Objects

• Loan approval (vsdk_loan_approval__c)

Picklists

• Loan Period (Months) (vsdk_loan_period_months1__c)
• Approval Status (vsdk_approval_status1__c)

VAULT JAVA SDK - Records Triggers

• com.veeva.vault.custom.triggers.vSdkSparkExternalAwsSampleTrigger - AFTER INSERT (vSdkAwsQueueSampleTrigger.java)

VAULT JAVA SDK - Records Actions

• Loan re-quote (vSdkSparkExternalAwsSampleAction.java)

Create Vault Queue

A Queue needs to be configured to utilize the Spark messaging functionality. These queues will handle messages that are produced via the Vault Java SDK Message and QueueService.

In vault

1. Log in and navigate to Admin > Configuration > Queue Setup > Queues and click Create.
2. Set the following values:
   • Check Author from scratch.
   • For Label, enter vSDK AWS Queue Sample.
   • For Name, enter vsdk_aws_queue_sample.
   • For Queue Type, select Outbound.
3. Click Save
4. On the new Queue, scroll down to the Queue Connections section and click Create.
5. Select the vsdk_aws_queue_sample_api_gateway and click Save.

Configure Public Key

First, you create a loan approval record to fire a message containing the Vault API Signature's Certificate Id, as this will correspond to the certificate used within the Spark messages

1. Complete steps 1 to 4 in part 1 of the page Run the project, which creates a Loan Approval record which in turn triggers a Spark message to be sent to AWS. Step 5 won't complete since the public key hasn't been set up within AWS to validate the record against.
2. Delete the Loan Approval record that has just been created.

Next, you configure the public key within AWS

1. In the AWS Lambda console, click the function vsdkSparkSampleValidateAndEnqueueMessage.
2. At the top of the screen select Monitoring which will display a screen showing CloudWatch metrics at a glance
3. Click View logs in CloudWatch, which will open CloudWatch Management Console in a new tab.
4. Click the Log Stream with the Last Event Time.
5. Click the Text radio button, which will display a screen like the one above.
6. Make a note of the File Name and copy the File contents which are highlighted as this will be needed shortly.
7. Return to the AWS Lambda console tab and select Configuration
8. Scroll to the Function code section
9. In the inline code editor, select the folder PublicKeys and while it's highlighted select File then New File from the menu.
10. Paste the File contents copied in step 6, into the newly created Untitled1 file.
11. Select File then Save As from the menu, and enter the File Name noted in step 6. This must contain the .pem extension, e.g. 00001.pem.
12. Click Save.

Next Steps

• Run the project provides details of how to run the project.
• Code logic provides a detailed understanding of how the sample components work.