NullReferenceException when retrieving properties of a streaming chat chunk using Asynchronous Filter

[RogerBarreto]

Confirm this is not an issue with the underlying OpenAI API

• This is an issue with the Python library

Confirm this is not an issue with Azure OpenAI

• This is not an issue with Azure OpenAI

Describe the bug

When using StreamingChatCompletionUpdate with Asynchronous Filter enabled in azure, this class breaks.

openai-dotnet/src/Custom/Chat/StreamingChatCompletionUpdate.cs

Line 83 in 583e9f6

? Choices[0].Delta.Content

When a SSE data: doesn't have a delta (null) the Choice[0].Delta line throws NullReferenceException.

Final 3 streaming chunks of an Asynchornous Filter response.

data: {"choices":[{"delta":{},"finish_reason":"stop","index":0,"logprobs":null}],"created":1724860848,"id":"chatcmpl-A1FOCU4vamBSxxqjaKcvADYMlwlaC","model":"gpt-4o-2024-05-13","object":"chat.completion.chunk","system_fingerprint":"fp_abc28019ad"}

data: {"choices":[{"content_filter_offsets":{"check_offset":1576,"start_offset":1576,"end_offset":2318},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"finish_reason":null,"index":0}],"created":0,"id":"","model":"","object":""}

data: {"choices":[{"content_filter_offsets":{"check_offset":1576,"start_offset":1576,"end_offset":2318},"content_filter_results":{"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false}},"finish_reason":null,"index":0}],"created":0,"id":"","model":"","object":""}

This issue was reported in our Semantic Kernel Issues Here but actually seems to be a problem when Azure tries to use the OpenAI SDK for those chunks, this potentially can be handled by Azure SDK library in a non-ideal (try catch block) or in the back-end (providing always an empty delta) but I wonder would be quicker/simple getting a fix in here.

The problem also happen for other properties of the class as below:

To Reproduce

1. Enable Asynchronous Filter in Azure or OpenAI (if possible)
2. Make a streaming request and try to check the Role property of the last 2 chunks

Since the property is a pointer, we are unable to test it for null.

if (lastChunk.Role.HasValue) { } // throws
if (lastChunk.Role is null) { } // throws

Code snippets

No response

OS

Windows 11

.NET version

.net 8

Library version

beta.10

[sandrohanea]

The problem seems to be that the SDK assumes that first "Choices" object always have a not-null Delta, while for content_filter_offsets this is not true.

https://github.com/openai/openai-dotnet/blob/dce79fc17963d0550f790db7158cbb0df7c186ec/src/Custom/Chat/StreamingChatCompletionUpdate.cs#L110C11-L110C27

Propagating null should help to not throw in this case: Choices[0].Delta.FunctionCall => Choices[0].Delta?.FunctionCall

[joseharriaga]

Thank you for reaching out, @RogerBarreto ! We have merged and released a fix for this issue. It is available starting with version 2.0.0-beta.12.