feat: generativeaionvertexai_embedding_batch #3913

gryczj · 2024-10-31T14:59:31Z

Description

Fixes #

Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.

Checklist

snippet-bot · 2024-10-31T15:07:05Z

Here is the summary of changes.

You are about to add 1 region tag.

ai-platform/snippets/create-batch-embedding.js:20, tag generativeaionvertexai_embedding_batch

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

Refresh this comment

iennae · 2024-11-21T06:41:40Z

@gemini review

code-review-assist-experimental

This pull request introduces a new snippet and its corresponding test for creating batch embeddings using the Vertex AI API. The code is well-structured and follows the project's contribution guidelines. However, there are a few areas that could be improved for clarity, maintainability, and best practices.

code-review-assist-experimental · 2024-11-21T06:57:23Z

ai-platform/snippets/create-batch-embedding.js

+  /**
+   * TODO(developer):  Uncomment/update these variables before running the sample.
+   */


The TODO comment asks the developer to uncomment/update these variables. It would be helpful to provide more specific instructions or examples of how these variables should be updated, including valid formats and constraints. For instance, specify the expected format for inputUri and outputUri, and whether jobName can contain special characters.

Suggested change

/**

* TODO(developer): Uncomment/update these variables before running the sample.

*/

/**

* TODO(developer): Uncomment and replace the following placeholders before running the sample.

* - projectId: Replace with your Google Cloud project ID.

* - inputUri: Replace with the URI of the input dataset. This can be a BigQuery table or a Google Cloud Storage file.

* Examples: "gs://[BUCKET]/[DATASET].jsonl" or "bq://[PROJECT].[DATASET].[TABLE]".

* - outputUri: Replace with the URI where the output will be stored. This can be a BigQuery table or a Google Cloud Storage file.

* Examples: "gs://[BUCKET]/[OUTPUT].jsonl" or "bq://[PROJECT].[DATASET].[TABLE]".

* - jobName: Replace with a descriptive name for the batch embedding job. Avoid special characters.

*/

This doesn't follow our style guidelines and would make sample management unwieldy

I understand your concern about style guidelines. Could you please clarify which specific aspects of the suggested changes violate your style guide? Knowing the specific rules will help me refine the suggestion to better align with your team's preferences. For example, are there length restrictions on comments, preferred ways to format URIs, or specific naming conventions for variables that are being violated?

ai-platform/snippets/create-batch-embedding.js

code-review-assist-experimental · 2024-11-21T06:57:23Z

ai-platform/snippets/create-batch-embedding.js

+  const textEmbeddingModel = 'text-embedding-005';
+  const location = 'us-central1';


Consider making textEmbeddingModel and location configurable parameters passed to the main function. This would make the snippet more flexible and reusable.

code-review-assist-experimental · 2024-11-21T06:57:23Z

ai-platform/snippets/create-batch-embedding.js

+    // Create batch prediction job request
+    const [response] = await jobServiceClient.createBatchPredictionJob(request);
+
+    console.log('Raw response: ', JSON.stringify(response, null, 2));


Instead of just logging the raw response, consider logging specific fields that are relevant to the user, such as the job ID, status, and output URI. This would make it easier for the user to track the job's progress and access the results.

Suggested change

console.log('Raw response: ', JSON.stringify(response, null, 2));

console.log('Batch prediction job created:', response.name);

console.log('Job ID:', response.name.split('/').pop());

console.log('Output URI:', response.outputConfig.gcsDestination.outputUriPrefix);

code-review-assist-experimental · 2024-11-21T06:57:23Z

ai-platform/snippets/test/create-batch-embedding.test.js

+  const location = 'us-central1';
+  const inputUri =
+    'gs://cloud-samples-data/generative-ai/embeddings/embeddings_input.jsonl';
+  let outputUri = 'gs://ucaip-samples-test-output/';


It's good practice to use a dedicated test project for integration tests to avoid conflicts with other resources. Consider using a separate project for testing and setting the project ID explicitly in the test configuration.

code-review-assist-experimental · 2024-11-21T06:57:23Z

ai-platform/snippets/test/create-batch-embedding.test.js

+
+  before(async () => {
+    const bucketName = `test-bucket-${uuid()}`;
+    // Create a Google Cloud Storage bucket for UsageReports


The comment mentions UsageReports, but this test creates a bucket for embedding output. Update the comment to reflect the actual purpose of the bucket.

Suggested change

// Create a Google Cloud Storage bucket for UsageReports

// Create a Google Cloud Storage bucket for embedding output

I think this suggestion is valid?

Yes, the suggestion is valid. The original comment incorrectly referred to UsageReports. The updated comment accurately reflects that the bucket is for embedding output, aligning with the test's purpose. The change improves clarity and accuracy.

@m-strzelczyk fixed

code-review-assist-experimental · 2024-11-21T06:57:24Z

ai-platform/snippets/test/create-batch-embedding.test.js

+
+      return jobServiceClient.deleteBatchPredictionJob(deleteRequest);
+    });
+    // Delete the Google Cloud Storage bucket created for usage reports.


Similar to the previous comment, this one also mentions usage reports. Update it to reflect the purpose of deleting the bucket.

Suggested change

// Delete the Google Cloud Storage bucket created for usage reports.

// Delete the Google Cloud Storage bucket created for embedding output.

I think this suggestion is valid?

The suggestion is valid and improves clarity. The comment now accurately reflects the purpose of deleting the bucket, which is to clean up resources after the test completes. This is good practice for resource management and prevents unnecessary costs.

@m-strzelczyk fixed

m-strzelczyk

Please change the comments about usageReport bucket, LGTM after that

m-strzelczyk · 2024-12-03T13:12:48Z

ai-platform/snippets/test/create-batch-embedding.test.js

+
+  before(async () => {
+    const bucketName = `test-bucket-${uuid()}`;
+    // Create a Google Cloud Storage bucket for UsageReports


I think this suggestion is valid?

m-strzelczyk · 2024-12-03T13:12:52Z

ai-platform/snippets/test/create-batch-embedding.test.js

+
+      return jobServiceClient.deleteBatchPredictionJob(deleteRequest);
+    });
+    // Delete the Google Cloud Storage bucket created for usage reports.


I think this suggestion is valid?

gryczj requested review from a team as code owners October 31, 2024 14:59

product-auto-label bot added samples Issues that are directly related to samples. api: aiplatform Issues related to the AI Platform API. labels Oct 31, 2024

gryczj added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Oct 31, 2024

gryczj force-pushed the generativeaionvertexai_embedding_batch branch from b5e809a to fc64882 Compare October 31, 2024 15:00

gryczj force-pushed the generativeaionvertexai_embedding_batch branch from fc64882 to 35afe3e Compare November 12, 2024 15:23

gryczj force-pushed the generativeaionvertexai_embedding_batch branch from 35afe3e to 32eb6c9 Compare November 20, 2024 12:33

code-review-assist-experimental bot reviewed Nov 21, 2024

View reviewed changes

gryczj force-pushed the generativeaionvertexai_embedding_batch branch from 111148a to d66ec9d Compare November 21, 2024 10:16

gryczj requested a review from iennae November 22, 2024 09:11

gryczj force-pushed the generativeaionvertexai_embedding_batch branch 3 times, most recently from 1c00638 to 0ad638a Compare December 3, 2024 08:28

m-strzelczyk approved these changes Dec 3, 2024

View reviewed changes

Joanna Grycz added 2 commits December 4, 2024 14:58

feat: generativeaionvertexai_embedding_batch

acbb009

fix typo

831a1d1

gryczj force-pushed the generativeaionvertexai_embedding_batch branch from 0ad638a to 831a1d1 Compare December 4, 2024 14:01

gryczj requested a review from m-strzelczyk December 4, 2024 14:01

BigBlackWolf self-requested a review December 5, 2024 11:55

Merge branch 'main' into generativeaionvertexai_embedding_batch

797cddf

iennae merged commit 936d05b into main Dec 5, 2024
133 of 134 checks passed

iennae deleted the generativeaionvertexai_embedding_batch branch December 5, 2024 22:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: generativeaionvertexai_embedding_batch #3913

feat: generativeaionvertexai_embedding_batch #3913

gryczj commented Oct 31, 2024 •

edited

Loading

snippet-bot bot commented Oct 31, 2024 •

edited

Loading

iennae commented Nov 21, 2024

code-review-assist-experimental bot left a comment

code-review-assist-experimental bot Nov 21, 2024

iennae Nov 21, 2024

code-review-assist-experimental bot Nov 21, 2024

code-review-assist-experimental bot Nov 21, 2024

code-review-assist-experimental bot Nov 21, 2024

code-review-assist-experimental bot Nov 21, 2024

code-review-assist-experimental bot Nov 21, 2024

m-strzelczyk Dec 3, 2024

code-review-assist-experimental bot Dec 3, 2024

gryczj Dec 4, 2024

code-review-assist-experimental bot Nov 21, 2024

m-strzelczyk Dec 3, 2024

code-review-assist-experimental bot Dec 3, 2024

gryczj Dec 4, 2024

m-strzelczyk left a comment

m-strzelczyk Dec 3, 2024

m-strzelczyk Dec 3, 2024

-  /**
-   * TODO(developer):  Uncomment/update these variables before running the sample.
-   */
+/**
+   * TODO(developer): Uncomment and replace the following placeholders before running the sample.
+   * - projectId: Replace with your Google Cloud project ID.
+   * - inputUri: Replace with the URI of the input dataset. This can be a BigQuery table or a Google Cloud Storage file.
+   *   Examples: "gs://[BUCKET]/[DATASET].jsonl" or "bq://[PROJECT].[DATASET].[TABLE]".
+   * - outputUri: Replace with the URI where the output will be stored. This can be a BigQuery table or a Google Cloud Storage file.
+   *   Examples: "gs://[BUCKET]/[OUTPUT].jsonl" or "bq://[PROJECT].[DATASET].[TABLE]".
+   * - jobName: Replace with a descriptive name for the batch embedding job. Avoid special characters.
+   */

		const textEmbeddingModel = 'text-embedding-005';
		const location = 'us-central1';

-    console.log('Raw response: ', JSON.stringify(response, null, 2));
+console.log('Batch prediction job created:', response.name);
+    console.log('Job ID:', response.name.split('/').pop());
+    console.log('Output URI:', response.outputConfig.gcsDestination.outputUriPrefix);

	// Create a Google Cloud Storage bucket for UsageReports
	// Create a Google Cloud Storage bucket for embedding output

	// Delete the Google Cloud Storage bucket created for usage reports.
	// Delete the Google Cloud Storage bucket created for embedding output.

feat: generativeaionvertexai_embedding_batch #3913

feat: generativeaionvertexai_embedding_batch #3913

Conversation

gryczj commented Oct 31, 2024 • edited Loading

Description

Checklist

snippet-bot bot commented Oct 31, 2024 • edited Loading

iennae commented Nov 21, 2024

code-review-assist-experimental bot left a comment

Choose a reason for hiding this comment

code-review-assist-experimental bot Nov 21, 2024

Choose a reason for hiding this comment

iennae Nov 21, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Nov 21, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Nov 21, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Nov 21, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Nov 21, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Nov 21, 2024

Choose a reason for hiding this comment

m-strzelczyk Dec 3, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Dec 3, 2024

Choose a reason for hiding this comment

gryczj Dec 4, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Nov 21, 2024

Choose a reason for hiding this comment

m-strzelczyk Dec 3, 2024

Choose a reason for hiding this comment

code-review-assist-experimental bot Dec 3, 2024

Choose a reason for hiding this comment

gryczj Dec 4, 2024

Choose a reason for hiding this comment

m-strzelczyk left a comment

Choose a reason for hiding this comment

m-strzelczyk Dec 3, 2024

Choose a reason for hiding this comment

m-strzelczyk Dec 3, 2024

Choose a reason for hiding this comment

gryczj commented Oct 31, 2024 •

edited

Loading

snippet-bot bot commented Oct 31, 2024 •

edited

Loading