Fix max tokens for embedding models #18
Conversation
Based on other documentation, the max tokens for embedding models should be 8,192 but not 8,191. Reference: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=console
|
@nkgami : Thanks for your contribution! The author(s) have been notified to review your proposed change. |
|
Learn Build status updates of commit 605ee39: ✅ Validation status: passed
For more details, please refer to the build report. For any questions, please:
|
|
Can you review the proposed changes? Important: When the changes are ready for publication, adding a #label:"aq-pr-triaged" |
prmerger-automator
bot
added
the
aq-pr-triaged
documenting entra id option
|
@nkgami Let me get back to you on this. We previously had 8192 set everywhere and then were asked by the PG to update to 8191 so the location that still says 8192 was an oversight in that round of changes. 8191 is what is currently stated in OpenAI's docs as well: https://platform.openai.com/docs/guides/embeddings/embedding-models With that said I am aware and have tested that if you exceed the token limit both with OAI, and AOAI the error message that is returned says 8192 is the limit, and the 8192 value appears in various code comments in the OpenAI Python library. It is possible that the issue is with the error message stating 8192, but I haven't been able to track down a definitive answer yet. |
|
This pull request has been inactive for at least 14 days. If you are finished with your changes, don't forget to sign off. See the contributor guide for instructions. |
Based on other documentation, the max tokens for embedding models should be 8,192 but not 8,191. Reference: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=console