Skip to content

docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumentation#53668

Open
richardsolomou wants to merge 9 commits intomasterfrom
docs/otel-onboarding-docs
Open

docs(llma): migrate onboarding docs to OpenTelemetry auto-instrumentation#53668
richardsolomou wants to merge 9 commits intomasterfrom
docs/otel-onboarding-docs

Conversation

@richardsolomou
Copy link
Copy Markdown
Member

@richardsolomou richardsolomou commented Apr 8, 2026

Problem

The in-app LLM analytics onboarding docs currently show PostHog SDK wrappers (posthog.ai.openai, @posthog/ai) as the primary integration method. We're moving toward standard OpenTelemetry auto-instrumentation as the recommended approach, keeping the wrappers available as a last resort for users who need them.

Changes

Migrates 28 provider onboarding guides from PostHog SDK wrapper approach to OpenTelemetry auto-instrumentation. Each migrated doc follows a consistent 3-step pattern:

  1. Install dependencies — OTel SDK + provider-specific instrumentation + provider SDK
  2. Set up OpenTelemetry tracingTracerProvider with OTLPSpanExporter (Python) or PostHogTraceExporter (Node)
  3. Call the provider — Native SDK usage, no posthog_ parameters

User identification uses the posthog.distinct_id resource attribute; custom properties like foo and conversation_id demonstrate the passthrough behavior.

Providers migrated (21 original + 7 more):

  • OpenAI-compatible: openai, groq, deepseek, mistral, cohere, fireworks-ai, xai, hugging-face, openrouter, together-ai, ollama, cerebras, perplexity
  • AI gateways: portkey, helicone, vercel-ai-gateway
  • Azure OpenAI, Anthropic
  • Frameworks: langchain, langgraph, llamaindex
  • Vercel AI (SimpleSpanProcessor update)
  • Newly migrated: autogen, instructor, mirascope, semantic-kernel, smolagents, pydantic-ai (already OTel-based in examples — docs just hadn't caught up)
  • Mastra (migrated to @mastra/posthog exporter — Mastra's native integration since there's no mature OTel path yet)

Kept as-is: google (no Node.js OTel instrumentation for @google/genai), aws-bedrock (already OTel), crewai, litellm, dspy, openai-agents, manual

Migration callout: Every migrated doc has a callout at the top linking to the full Node.js and Python examples on GitHub (current main/master), plus legacy wrapper examples pinned to the last commit before the migration. This helps both new users (who see the recommended OTel path) and existing users (who can find the old wrapper examples they're already using).

Manual capture doc: Split into multiple steps so it can have a useful table of contents (Capture LLM events manually → Event properties → Verify traces and generations).

Removed: "No proxy" callouts from all migrated docs — they made sense when wrapping SDK clients but don't apply to standard OTel tracing.

How did you test this code?

Agent-authored PR. Manually verified:

No runtime tests since these are in-app onboarding content (static TSX returning step definitions).

Publish to changelog?

No

Docs update

This IS the docs update. Companion PR: PostHog/posthog.com#16236 (updates TOC frontmatter in posthog.com).

🤖 LLM context

Co-authored with Claude Code. Related PRs:

…tion

Replace PostHog SDK wrapper approach with standard OpenTelemetry
auto-instrumentation across 22 provider onboarding guides. Each doc
now follows a 3-step pattern: install OTel deps, set up tracing,
call the provider with the native SDK.
- Remove user.id from Vercel AI resource (posthog_distinct_id in
  experimental_telemetry is the established mechanism for that SDK)
- Normalize service.name to 'my-app' in Vercel AI doc
- Add anonymous events Blockquote to langchain, langgraph, llamaindex
- Add Node tab to OpenAI embeddings step
@richardsolomou richardsolomou marked this pull request as ready for review April 8, 2026 09:42
@richardsolomou richardsolomou requested a review from a team April 8, 2026 09:42
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 8, 2026

Vulnerabilities

No security concerns identified. The changes are documentation-only, migrating onboarding code snippets from PostHog SDK wrappers to OpenTelemetry auto-instrumentation. API keys in the snippets are clearly marked as placeholders.

Comments Outside Diff (1)

  1. docs/onboarding/llm-analytics/portkey.tsx, line 24-43 (link)

    P1 Missing portkey-ai package in install instructions

    The "Install dependencies" step only lists the OpenAI SDK and OTel packages, but the "Call Portkey" step imports portkey_ai (Python) and portkey-ai (Node) which are not included. Users copying the install command will hit an ImportError / Module not found error at runtime.

    Python step (line 136–141) uses:

    from portkey_ai import PORTKEY_GATEWAY_URL

    Node step (line 158) uses:

    import { PORTKEY_GATEWAY_URL } from 'portkey-ai'

    The install blocks should include portkey-ai:

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: docs/onboarding/llm-analytics/portkey.tsx
    Line: 24-43
    
    Comment:
    **Missing `portkey-ai` package in install instructions**
    
    The "Install dependencies" step only lists the OpenAI SDK and OTel packages, but the "Call Portkey" step imports `portkey_ai` (Python) and `portkey-ai` (Node) which are not included. Users copying the install command will hit an `ImportError` / `Module not found` error at runtime.
    
    Python step (line 136–141) uses:
    ```python
    from portkey_ai import PORTKEY_GATEWAY_URL
    ```
    
    Node step (line 158) uses:
    ```typescript
    import { PORTKEY_GATEWAY_URL } from 'portkey-ai'
    ```
    
    The install blocks should include `portkey-ai`:
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: docs/onboarding/llm-analytics/portkey.tsx
Line: 24-43

Comment:
**Missing `portkey-ai` package in install instructions**

The "Install dependencies" step only lists the OpenAI SDK and OTel packages, but the "Call Portkey" step imports `portkey_ai` (Python) and `portkey-ai` (Node) which are not included. Users copying the install command will hit an `ImportError` / `Module not found` error at runtime.

Python step (line 136–141) uses:
```python
from portkey_ai import PORTKEY_GATEWAY_URL
```

Node step (line 158) uses:
```typescript
import { PORTKEY_GATEWAY_URL } from 'portkey-ai'
```

The install blocks should include `portkey-ai`:

```suggestion
                    <Markdown>Install the OpenTelemetry SDK, the OpenAI instrumentation, the OpenAI SDK, and the Portkey SDK.</Markdown>

                    <CodeBlock
                        blocks={[
                            {
                                language: 'bash',
                                file: 'Python',
                                code: dedent`
                                    pip install openai portkey-ai opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-openai-v2
                                `,
                            },
                            {
                                language: 'bash',
                                file: 'Node',
                                code: dedent`
                                    npm install openai portkey-ai @posthog/ai @opentelemetry/sdk-node @opentelemetry/resources @opentelemetry/instrumentation-openai
                                `,
                            },
                        ]}
                    />
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(llma): address review feedback on OT..." | Re-trigger Greptile

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

Docs from this PR will be published at posthog.com

Project Deployment Preview Updated (UTC)
posthog.com 🤷 Unknown Preview Apr 9, 2026, 8:16 AM

Preview will be ready in ~10 minutes. Click Preview link above to access docs at /handbook/engineering/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant