Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AnthropicChatGenerator StreamingChunk "meta" is empty #1146

Open
wursterje opened this issue Oct 18, 2024 · 0 comments
Open

AnthropicChatGenerator StreamingChunk "meta" is empty #1146

wursterje opened this issue Oct 18, 2024 · 0 comments
Assignees
Labels
bug Something isn't working integration:anthropic P2

Comments

@wursterje
Copy link

wursterje commented Oct 18, 2024

I am looking for some meta information, similar to the OpenAIChatGenerator, to capture the stop event. Unfortunately, the meta field for the AnthropicChatGenerator is empty.

I added some print statements for debugging to AnthropicChatGenerator (line 221 and 227):

  # if streaming is enabled, the response is a Stream[MessageStreamEvent]
        if isinstance(response, Stream):
            chunks: List[StreamingChunk] = []
            stream_event, delta, start_event = None, None, None
            for stream_event in response:
                print(stream_event)
                if isinstance(stream_event, MessageStartEvent):
                    # capture start message to count input tokens
                    start_event = stream_event
                if isinstance(stream_event, ContentBlockDeltaEvent):
                    chunk_delta: StreamingChunk = self._build_chunk(stream_event.delta)
                    print(chunk_delta)
                    chunks.append(chunk_delta)

And here is the output where you can see that the meta field is empty:

RawContentBlockDeltaEvent(delta=TextDelta(text='d of test', type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content='d of test', meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text=" you're", type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content=" you're", meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text=' referring to? Once', type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content=' referring to? Once', meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text=' you give', type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content=' you give', meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text=' me more context or', type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content=' me more context or', meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text=' a specific question, I', type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content=' a specific question, I', meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text="'ll do my best to", type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content="'ll do my best to", meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text=' assist you', type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content=' assist you', meta={})
RawContentBlockDeltaEvent(delta=TextDelta(text='.', type='text_delta'), index=0, type='content_block_delta')
StreamingChunk(content='.', meta={})
RawContentBlockStopEvent(index=0, type='content_block_stop')
RawMessageDeltaEvent(delta=Delta(stop_reason='end_turn', stop_sequence=None), type='message_delta', usage=MessageDeltaUsage(output_tokens=93))
RawMessageStopEvent(type='message_stop')

Version: anthropic-haystack 1.1.0

@wursterje wursterje added the bug Something isn't working label Oct 18, 2024
@Amnah199 Amnah199 self-assigned this Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working integration:anthropic P2
Projects
None yet
Development

No branches or pull requests

3 participants