Skip to content

Commit

Permalink
Merge pull request #261 from aws-samples/feature/update-models
Browse files Browse the repository at this point in the history
Added claude-3 models for summarization and queries.

Got offline approval from Chris Lott to merge.
  • Loading branch information
kishd authored May 24, 2024
2 parents 5355062 + 54cbf0e commit 2d35e4e
Show file tree
Hide file tree
Showing 19 changed files with 1,445 additions and 146 deletions.
4 changes: 2 additions & 2 deletions patches/mediasearch/msfinder.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -331,7 +331,7 @@ Resources:
Type: AWS::Lambda::Function
Properties:
Handler: lambda_function.lambda_handler
Runtime: python3.8
Runtime: python3.11
Role: !GetAtt 'MediaLambdaRole.Arn'
Timeout: 300
Code: ../lambda/build-trigger
Expand Down Expand Up @@ -387,7 +387,7 @@ Resources:
Condition: EnableAccess
Properties:
Handler: lambda_function.lambda_handler
Runtime: python3.8
Runtime: python3.11
Role: !GetAtt 'TokenEnablerLambdaRole.Arn'
Timeout: 900
MemorySize: 1024
Expand Down
33 changes: 25 additions & 8 deletions pca-main-nokendra.template
Original file line number Diff line number Diff line change
Expand Up @@ -363,8 +363,10 @@ Parameters:

GenAIQueryBedrockModelId:
Type: String
Default: anthropic.claude-v2
Default: anthropic.claude-3-haiku-20240307-v1:0
AllowedValues:
- anthropic.claude-3-haiku-20240307-v1:0
- anthropic.claude-3-sonnet-20240229-v1:0
- amazon.titan-text-express-v1
- anthropic.claude-v1
- anthropic.claude-instant-v1
Expand Down Expand Up @@ -394,8 +396,10 @@ Parameters:

SummarizationBedrockModelId:
Type: String
Default: anthropic.claude-instant-v1
Default: anthropic.claude-3-haiku-20240307-v1:0
AllowedValues:
- anthropic.claude-3-haiku-20240307-v1:0
- anthropic.claude-3-sonnet-20240229-v1:0
- amazon.titan-text-express-v1
- anthropic.claude-v1
- anthropic.claude-instant-v1
Expand Down Expand Up @@ -685,10 +689,18 @@ Resources:
provider = modelId.split(".")[0]
request_body = None
if provider == "anthropic":
request_body = {
"prompt": prompt,
"max_tokens_to_sample": DEFAULT_MAX_TOKENS
}
if 'claude-3' in modelId:
request_body = {
"max_tokens": DEFAULT_MAX_TOKENS,
"messages": [{"role": "user", "content": prompt}],
"anthropic_version": "bedrock-2023-05-31"
}
else:
request_body = {
"prompt": prompt,
"max_tokens_to_sample": DEFAULT_MAX_TOKENS
}

request_body.update(parameters)
elif provider == "ai21":
request_body = {
Expand All @@ -713,8 +725,13 @@ Resources:
provider = modelId.split(".")[0]
generated_text = None
if provider == "anthropic":
response_body = json.loads(response.get("body").read().decode())
generated_text = response_body.get("completion")
if 'claude-3' in modelId:
response_raw = json.loads(response.get("body").read().decode())
generated_text = response_raw.get('content')[0].get('text')

else:
response_body = json.loads(response.get("body").read().decode())
generated_text = response_body.get("completion")
elif provider == "ai21":
response_body = json.loads(response.get("body").read())
generated_text = response_body.get("completions")[0].get("data").get("text")
Expand Down
33 changes: 25 additions & 8 deletions pca-main.template
Original file line number Diff line number Diff line change
Expand Up @@ -365,8 +365,10 @@ Parameters:

GenAIQueryBedrockModelId:
Type: String
Default: anthropic.claude-v2
Default: anthropic.claude-3-haiku-20240307-v1:0
AllowedValues:
- anthropic.claude-3-haiku-20240307-v1:0
- anthropic.claude-3-sonnet-20240229-v1:0
- amazon.titan-text-express-v1
- anthropic.claude-v1
- anthropic.claude-instant-v1
Expand Down Expand Up @@ -396,8 +398,10 @@ Parameters:

SummarizationBedrockModelId:
Type: String
Default: anthropic.claude-instant-v1
Default: anthropic.claude-3-haiku-20240307-v1:0
AllowedValues:
- anthropic.claude-3-haiku-20240307-v1:0
- anthropic.claude-3-sonnet-20240229-v1:0
- amazon.titan-text-express-v1
- anthropic.claude-v1
- anthropic.claude-instant-v1
Expand Down Expand Up @@ -816,10 +820,18 @@ Resources:
provider = modelId.split(".")[0]
request_body = None
if provider == "anthropic":
request_body = {
"prompt": prompt,
"max_tokens_to_sample": DEFAULT_MAX_TOKENS
}
if 'claude-3' in modelId:
request_body = {
"max_tokens": DEFAULT_MAX_TOKENS,
"messages": [{"role": "user", "content": prompt}],
"anthropic_version": "bedrock-2023-05-31"
}
else:
request_body = {
"prompt": prompt,
"max_tokens_to_sample": DEFAULT_MAX_TOKENS
}

request_body.update(parameters)
elif provider == "ai21":
request_body = {
Expand All @@ -844,8 +856,13 @@ Resources:
provider = modelId.split(".")[0]
generated_text = None
if provider == "anthropic":
response_body = json.loads(response.get("body").read().decode())
generated_text = response_body.get("completion")
if 'claude-3' in modelId:
response_raw = json.loads(response.get("body").read().decode())
generated_text = response_raw.get('content')[0].get('text')

else:
response_body = json.loads(response.get("body").read().decode())
generated_text = response_body.get("completion")
elif provider == "ai21":
response_body = json.loads(response.get("body").read())
generated_text = response_body.get("completions")[0].get("data").get("text")
Expand Down
2 changes: 1 addition & 1 deletion pca-server/cfn/lib/trigger.template
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ Resources:
Properties:
Code: ../../src/trigger
Handler: index.handler
Runtime: nodejs16.x
Runtime: nodejs18.x
Role: !GetAtt ConfigureBucketRole.Arn
Environment:
Variables:
Expand Down
4 changes: 3 additions & 1 deletion pca-server/cfn/pca-server.template
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,10 @@ Parameters:

SummarizationBedrockModelId:
Type: String
Default: anthropic.claude-instant-v1
Default: anthropic.claude-3-haiku-20240307-v1:0
AllowedValues:
- anthropic.claude-3-haiku-20240307-v1:0
- anthropic.claude-3-sonnet-20240229-v1:0
- amazon.titan-text-express-v1
- anthropic.claude-v1
- anthropic.claude-instant-v1
Expand Down
24 changes: 18 additions & 6 deletions pca-server/src/pca/pca-aws-sf-summarize.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,17 @@ def get_bedrock_request_body(modelId, parameters, prompt):
provider = modelId.split(".")[0]
request_body = None
if provider == "anthropic":
request_body = {
"prompt": prompt,
"max_tokens_to_sample": MAX_TOKENS
}
if 'claude-3' in modelId:
request_body = {
"max_tokens": MAX_TOKENS,
"messages": [{"role": "user", "content": prompt}],
"anthropic_version": "bedrock-2023-05-31"
}
else:
request_body = {
"prompt": prompt,
"max_tokens_to_sample": MAX_TOKENS
}
request_body.update(parameters)
elif provider == "ai21":
request_body = {
Expand All @@ -92,8 +99,13 @@ def get_bedrock_generate_text(modelId, response):
provider = modelId.split(".")[0]
generated_text = None
if provider == "anthropic":
response_body = json.loads(response.get("body").read().decode())
generated_text = response_body.get("completion")
if 'claude-3' in modelId:
response_raw = json.loads(response.get("body").read().decode())
generated_text = response_raw.get('content')[0].get('text')

else:
response_body = json.loads(response.get("body").read().decode())
generated_text = response_body.get("completion")
elif provider == "ai21":
response_body = json.loads(response.get("body").read())
generated_text = response_body.get("completions")[0].get("data").get("text")
Expand Down
31 changes: 16 additions & 15 deletions pca-server/src/trigger/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pca-server/src/trigger/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"description": "",
"main": "index.js",
"dependencies": {
"aws-sdk": "^2.757.0",
"aws-sdk": "^2.1620.0",
"cfn-response": "^1.0.1"
},
"devDependencies": {},
Expand Down
20 changes: 11 additions & 9 deletions pca-ui/cfn/lib/api.template
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,10 @@ Parameters:

GenAIQueryBedrockModelId:
Type: String
Default: anthropic.claude-v2
Default: anthropic.claude-3-haiku-20240307-v1:0
AllowedValues:
- anthropic.claude-3-haiku-20240307-v1:0
- anthropic.claude-3-sonnet-20240229-v1:0
- amazon.titan-text-express-v1
- anthropic.claude-v1
- anthropic.claude-instant-v1
Expand Down Expand Up @@ -124,7 +126,7 @@ Resources:
Policies:
- DynamoDBReadPolicy:
TableName: !Ref TableName
Runtime: nodejs16.x
Runtime: nodejs18.x

GetFunction:
Type: "AWS::Serverless::Function"
Expand All @@ -147,7 +149,7 @@ Resources:
BucketName: !Ref DataBucket
- S3ReadPolicy:
BucketName: !Ref AudioBucket
Runtime: nodejs16.x
Runtime: nodejs18.x

ListFunction:
Type: "AWS::Serverless::Function"
Expand All @@ -167,7 +169,7 @@ Resources:
Policies:
- DynamoDBReadPolicy:
TableName: !Ref TableName
Runtime: nodejs16.x
Runtime: nodejs18.x

SearchFunction:
Type: "AWS::Serverless::Function"
Expand All @@ -187,7 +189,7 @@ Resources:
Policies:
- DynamoDBReadPolicy:
TableName: !Ref TableName
Runtime: nodejs16.x
Runtime: nodejs18.x

EntitiesFunction:
Type: "AWS::Serverless::Function"
Expand All @@ -207,7 +209,7 @@ Resources:
Policies:
- DynamoDBReadPolicy:
TableName: !Ref TableName
Runtime: nodejs16.x
Runtime: nodejs18.x

LanguagesFunction:
Type: "AWS::Serverless::Function"
Expand All @@ -227,7 +229,7 @@ Resources:
Policies:
- DynamoDBReadPolicy:
TableName: !Ref TableName
Runtime: nodejs16.x
Runtime: nodejs18.x

SwapFunction:
Type: "AWS::Serverless::Function"
Expand All @@ -250,7 +252,7 @@ Resources:
BucketName: !Ref DataBucket
- DynamoDBCrudPolicy:
TableName: !Ref TableName
Runtime: nodejs16.x
Runtime: nodejs18.x

GenAIQueryFunction:
Type: "AWS::Serverless::Function"
Expand Down Expand Up @@ -367,7 +369,7 @@ Resources:
Policies:
- S3WritePolicy:
BucketName: !Ref AudioBucket
Runtime: nodejs16.x
Runtime: nodejs18.x

Outputs:
Uri:
Expand Down
Loading

0 comments on commit 2d35e4e

Please sign in to comment.