feat(llm-observability): spans #27933

skoob13 · 2025-01-27T19:50:28Z

Problem

Users want to see a tree of spans and generations with the input and output of each step.

Changes

Restore the generation tree.
Render the tree recursively.
Update taxonomy for spans.
Update the traces query runner to have a fallback for $ai_trace_name and return $ai_span events.

Light theme

Dark theme

Does this work well for both Cloud and self-hosted?

Yes

How did you test this code?

Unit tests and manual testing

github-actions · 2025-01-27T20:02:49Z

Size Change: +503 B (+0.04%)

Total Size: 1.16 MB

ℹ️ View Unchanged

Filename	Size	Change
`frontend/dist/toolbar.js`	1.16 MB	+503 B (+0.04%)

_{compressed-size-action}

…ervability-spans

…thog into feat/llm-observability-spans

Twixes

Tree overflow gets janky, otherwise all good

Twixes · 2025-01-28T15:53:51Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

                            </h3>
                        </div>
                        {isLLMTraceEvent(event) ? (
                            <MetadataHeader
+                                hasError={event.properties.$ai_is_error}


Naming nit: may be more consistent to do isError={event.properties.$ai_is_error}

Twixes · 2025-01-28T16:20:23Z

products/llm_observability/frontend/ConversationDisplay/ConversationMessagesDisplay.tsx

+        )
+    }
+)
+LLMMessageDisplay.displayName = 'LLMMessageDisplay'


This wouldn't be needed if the memo used function:

export const LLMMessageDisplay = React.memo( function LLMMessageDisplay(...) { } )

Twixes · 2025-01-28T16:40:29Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

-    topLevelTrace,
-    item,
-    isSelected,
+const TraceNode = React.memo(


So actually this is a TraceLeaf

Twixes · 2025-01-28T16:40:35Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

+)
+TraceNode.displayName = 'TraceNode'
+
+function RecursiveTreeDisplay({


And this is a TraceNode

AH, I was being raving mad, this is actually NodeChildren and the other one is TreeNode

Twixes · 2025-01-28T16:43:34Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

+                </span>
+            ),
+        ]
+        const hasChildren = children.find((child) => !!child)


Suggested change

const hasChildren = children.find((child) => !!child)

const hasChildren = children.some((child) => !!child)

~~find is enough here. We just need to know that at least one child exists.~~ You're right😄 It's what some does. I'm more used to find.

Twixes · 2025-01-28T16:46:06Z

products/llm_observability/frontend/llmObservabilityTraceDataLogic.ts

+    const idMap = new Map<any, LLMTraceEvent>()
+
+    // Map all events with parents to their parent IDs
+    events.forEach((event) => {


I always find for (... of ...) a bit more readable, but that's a nit which should be a linter rule

forEach used to be faster. I'm unsure though it's true now.

Twixes · 2025-01-28T16:55:24Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

 }): JSX.Element {
    return (
-        <aside className="border-border max-h-fit bg-bg-light border rounded overflow-hidden md:w-72">
+        <aside className="border-border max-h-full bg-bg-light border rounded overflow-hidden md:w-72">


Is it intentional we're back to filling the full height of the scene even if there's just a couple items in the trace? I believed max-h-fit would still work here

~~Yeah, if we set max-h-fit, the scroll won't work for larger content.~~

Twixes · 2025-01-28T16:56:37Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

            </NestingGroup>
        </aside>
    )
 }

 function NestingGroup({ level = 0, children }: { level?: number; children: React.ReactNode }): JSX.Element {
-    const listEl = <ul className={!level ? 'overflow-y-auto p-1 first:*:mt-0' : 'flex-1'}>{children}</ul>
+    const listEl = (
+        <ul className={!level ? 'hide-scrollbar overflow-y-auto p-1 first:*:mt-0 h-full' : 'flex-1'}>{children}</ul>


A few snags in scrolling:

It's a much worse experience without scrollbars, let's avoid hide-scrollbar

Horizontal overflow is a little awkward, it's generally hard to make scrolling a list in both axes feel good. Proposing we wrap for readability instead. This will also avoid mismatches width, like here:

I believe this should be max-h-full so avoid filling height with whitespace

Do we usually show scrollbars for overflow or not? The scrollbar doesn't look nice on Windows.

It's all fixed, but the scrollbar question is nice to know.

We do! On macOS it's worth setting "always show scrollbars" in system settings, to see them (though most macOS users won't)

Twixes · 2025-01-28T17:03:10Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

-    const usage = formatLLMUsage(item)
+    return (
+        <NestingGroup level={1}>
+            {tree.map(({ event, children }) => (


Something mysterious I'm seeing in Max AI traces: a span beyond what I can scroll to at the bottom

Thanks! I didn't notice it.

Twixes · 2025-01-28T17:03:43Z

products/llm_observability/frontend/LLMObservabilityTraceScene.tsx

+                        'p-2 text-xs border rounded',
+                        !raisedError ? 'bg-[var(--bg-fill-success-tertiary)]' : 'bg-[var(--bg-fill-warning-tertiary)]'


Let's use red to signal an error:

Suggested change

'p-2 text-xs border rounded',

!raisedError ? 'bg-[var(--bg-fill-success-tertiary)]' : 'bg-[var(--bg-fill-warning-tertiary)]'

'p-2 text-xs border rounded',

!raisedError ? 'bg-[var(--bg-fill-success-tertiary)]' : 'bg-[var(--bg-fill-danger-tertiary)]'

I used the danger color initially, but the contrast ratio is not great and is hard to read. When you look at those spans for a little longer, you actually want it to be plain and readable. I've thought about changing the header color instead of applying a background, as it serves the same indicative purpose, but the content is significantly easier to read.

Error

Danger

I'll switch to red, as I think it looks better now. Let's see.

Twixes

Awesome

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michael Matloka <[email protected]>

skoob13 added 6 commits January 27, 2025 12:48

feat: taxonomy

b2fad55

feat: output spans

3a6a7f2

feat: frontend tree

fd250f7

feat: format input and output

6e8da2c

feat: display error

88a0592

feat: render errors

56a3a46

github-actions bot and others added 13 commits January 27, 2025 20:03

Update query snapshots

2abb42a

chore: update taxonomy

eaef6f7

feat: fallback for $ai_trace_name

98f90b2

Merge branch 'master' of github.com:PostHog/posthog into feat/llm-obs…

c9c134d

…ervability-spans

fix: contrast ratio for inputs

3da9a30

feat: display errors in generations

330bb17

feat: hide scrollbars

1fb5159

fix: filter out feedback events

6ca0f1e

fix: argument display of messages

3266a49

fix: jest missing path for products

9ad7aca

test: tree

b2cd3a3

fix: danger -> warning for errors

71ba333

Merge branch 'feat/llm-observability-spans' of github.com:PostHog/pos…

323dbc6

…thog into feat/llm-observability-spans

skoob13 marked this pull request as ready for review January 28, 2025 10:31

skoob13 requested review from Twixes and k11kirky January 28, 2025 10:31

Update query snapshots

9eea910

Twixes requested changes Jan 28, 2025

View reviewed changes

skoob13 added 3 commits January 28, 2025 18:53

fix: comments

6e3bb78

fix: overflow

77dcdc8

fix: use fit instead of max-h-full

9d69ccc

skoob13 requested a review from Twixes January 28, 2025 18:22

Make sidebar slightly wider

1a958d8

Fix component naming

fc3ce81

Twixes approved these changes Jan 28, 2025

View reviewed changes

k11kirky merged commit d7758ee into master Jan 28, 2025
99 checks passed

k11kirky deleted the feat/llm-observability-spans branch January 28, 2025 21:45

Twixes mentioned this pull request Jan 28, 2025

feat(llm-observability): Make spans collapsible #27998

Merged

adamleithp pushed a commit that referenced this pull request Jan 29, 2025

feat(llm-observability): spans (#27933)

5bc5603

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Michael Matloka <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(llm-observability): spans #27933

feat(llm-observability): spans #27933

skoob13 commented Jan 27, 2025 •

edited

Loading

github-actions bot commented Jan 27, 2025 •

edited

Loading

Twixes left a comment

Twixes Jan 28, 2025

Twixes Jan 28, 2025

Twixes Jan 28, 2025

Twixes Jan 28, 2025

Twixes Jan 28, 2025

Twixes Jan 28, 2025

skoob13 Jan 28, 2025 •

edited

Loading

Twixes Jan 28, 2025

skoob13 Jan 28, 2025

Twixes Jan 28, 2025

skoob13 Jan 28, 2025 •

edited

Loading

Twixes Jan 28, 2025

skoob13 Jan 28, 2025 •

edited

Loading

skoob13 Jan 28, 2025 •

edited

Loading

Twixes Jan 28, 2025

Twixes Jan 28, 2025

skoob13 Jan 28, 2025

Twixes Jan 28, 2025

skoob13 Jan 28, 2025

skoob13 Jan 28, 2025

Twixes left a comment

	const hasChildren = children.find((child) => !!child)
	const hasChildren = children.some((child) => !!child)

		'p-2 text-xs border rounded',
		!raisedError ? 'bg-[var(--bg-fill-success-tertiary)]' : 'bg-[var(--bg-fill-warning-tertiary)]'

feat(llm-observability): spans #27933

feat(llm-observability): spans #27933

Conversation

skoob13 commented Jan 27, 2025 • edited Loading

Problem

Changes

Does this work well for both Cloud and self-hosted?

How did you test this code?

github-actions bot commented Jan 27, 2025 • edited Loading

Twixes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skoob13 Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skoob13 Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skoob13 Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

skoob13 Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Twixes left a comment

Choose a reason for hiding this comment

skoob13 commented Jan 27, 2025 •

edited

Loading

github-actions bot commented Jan 27, 2025 •

edited

Loading

skoob13 Jan 28, 2025 •

edited

Loading

skoob13 Jan 28, 2025 •

edited

Loading

skoob13 Jan 28, 2025 •

edited

Loading

skoob13 Jan 28, 2025 •

edited

Loading