AttributeError("'dict' object has no attribute 'audio_tokens'")

[demux79]

• This is actually a bug report.
• I am not getting good LLM Results
• I have tried asking for help in the community on discord or discussions and have not received a response.
• I have tried searching the documentation and have not found an answer.

What Model are you using?

• gpt-4o-mini-2024-07-18

Describe the bug
After upgrading from Instructor 1.4 to 1.7.2 I reliably get this error message: AttributeError("'dict' object has no attribute 'audio_tokens'")

Instructor retry error (AttributeError("'dict' object has no attribute 'audio_tokens'"),): Secret question | Number of attempts: 2 | Last completion: ChatCompletion(id='chatcmpl-AwHgTkbW0O4qstBcBXV7tVbGnLqkj', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Bje6d4Fv05eM7UqnQ8r3Jqyk', function=Function(arguments='{"relevant":true,"message":"Secret Answer"}', name='ChatClassification'), type='function')]))], created=1738454365, model='gpt-4o-mini-2024-07-18', object='chat.completion', service_tier='default', system_fingerprint='fp_72ed7ab54c', usage=CompletionUsage(completion_tokens=95, prompt_tokens=326, total_tokens=421, completion_tokens_details={'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, prompt_tokens_details={'audio_tokens': 0, 'cached_tokens': 0}))

To Reproduce
Every time

Expected behavior
Do not fail on audio_token

[ivanleomk]

@demux79 could you provide a code snippet to reproduce this?

[ivanleomk]

ivanleo@Ivans-MacBook-Pro ~/D/c/test (main) [127]> uv pip list | grep -E "instructor|openai"
instructor 1.7.2
openai 1.61.0

I just tested this on the latest release of instructor and I don't get this error.

from openai import OpenAI
from pydantic import BaseModel
import instructor


class ChatClassification(BaseModel):
    relevant: bool
    message: str


client = instructor.from_openai(OpenAI())

resp = client.chat.completions.create(
    model="gpt-4o-mini-2024-07-18",
    messages=[{"role": "user", "content": "Is a secret answer relevant?"}],
    response_model=ChatClassification,
)

print(resp)

Closing this issue for now until there's a code snippet to reproduce this error and a list of installed packages for openai and instructor

[brycedrennan]

Reproduction:

from litellm import completion
from pydantic import BaseModel
import instructor


class ChatClassification(BaseModel):
    relevant: bool
    message: str


client = instructor.from_litellm(completion)

resp = client.chat.completions.create(
    model="gpt-4o-mini-2024-07-18",
    messages=[{"role": "user", "content": "Is a secret answer relevant?"}],
    response_model=ChatClassification,
)

print(resp)

instructor==1.7.2
litellm==1.44.10
openai==1.60.2

Traceback (most recent call last):
  File ".pyenv/versions/3.10.13/envs/job-listing-monitor-3.10.13/lib/python3.10/site-packages/instructor/retry.py", line 158, in retry_sync
    response = update_total_usage(
  File ".pyenv/versions/3.10.13/envs/job-listing-monitor-3.10.13/lib/python3.10/site-packages/instructor/utils.py", line 151, in update_total_usage
    ttd.audio_tokens = (ttd.audio_tokens or 0) + (rtd.audio_tokens or 0)
AttributeError: 'dict' object has no attribute 'audio_tokens'

[brycedrennan]

Looks like litellm doesn't fully convert the usage information to objects.
https://github.com/BerriAI/litellm/blob/c17342ac5bf0e9b281ffe2dbf6380411856eaf96/litellm/litellm_core_utils/llm_response_utils/convert_dict_to_response.py#L511-L514

[demux79]

Yes, that must be it. Now that I know it I understand why some of my projects work and some not. Those with litellm fail.