Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audio Timestamp option missing in Generation Config #4511

Open
Waheguru-Anurag opened this issue Oct 5, 2024 · 3 comments
Open

Audio Timestamp option missing in Generation Config #4511

Waheguru-Anurag opened this issue Oct 5, 2024 · 3 comments
Assignees
Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API.

Comments

@Waheguru-Anurag
Copy link

Hi I was reading the vertex ai documentation - [https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/audio-understanding#:~:text=files%2C%20enable%20the-,audioTimestamp,-parameter%20in%20GenerationConfig]

Here it is mentioned:
2. Audio-only timestamps: To accurately generate timestamps for audio-only files, you must configure the audio_timestamp parameter in generation_config.

But I am not able to set this parameter in generation_config.

@product-auto-label product-auto-label bot added the api: vertex-ai Issues related to the googleapis/python-aiplatform API. label Oct 5, 2024
@jaycee-li jaycee-li self-assigned this Oct 7, 2024
@jaycee-li
Copy link
Contributor

Hi @Waheguru-Anurag, this field was recently added (last week) and is currently only available in the REST API. The Python SDK hasn't been updated to support it yet.

@tfriedel
Copy link

tfriedel commented Oct 8, 2024

I tried this parameter using the REST API but didn't notice an improvement.
For a 3 min long file timestamps suggested it was over 4 minutes long.

I used this prompt:

Translate the audio to english. Include timestamps and speakers. Use the following format:

<example>
[00:17] Agent (male): Yes, sir. So, you have a shop that sells medicines, fertilizers, and seeds?
[00:19] Customer (male): Hmm.
[00:21] Agent (male): Sir, I have this app, sir, for retailers.
</example>

@nithinsag
Copy link

I have tried the rest api as well, didn't help as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API.
Projects
None yet
Development

No branches or pull requests

4 participants