Add telemetry documentation

[rajeshkamal5050]

Documents the telemetry system end-to-end across azure-dev, azd-queries, and azure-dev-tools.

TLDR

azd telemetry spans 3 repos — instrumentation in azure-dev, pipeline/governance in azd-queries, analysis in azure-dev-tools. These docs capture the full picture: how data flows, what gets collected, how to add telemetry for new features, and where dashboards live.

Architecture

flowchart TB
    subgraph Instrumentation ["azure-dev (Instrumentation)"]
        CLI["azd CLI<br/>(Go + OpenTelemetry)"]
        VSC["VS Code Extension<br/>(@microsoft/vscode-azext-utils)"]
        EXT["Extensions<br/>(structured error reporting)"]
    end

    subgraph Export ["CLI Export Pipeline"]
        MW["Command Middleware<br/>cli/azd/cmd/middleware/telemetry.go"]
        OTel["OTel TracerProvider"]
        AIExp["App Insights Exporter<br/>SpanToEnvelope()"]
        DiskQ["Disk Queue<br/>~/.azd/telemetry/*.trn"]
        Upload["azd telemetry upload<br/>(background / deferred)"]
    end

    subgraph Ingestion ["Azure Monitor / Kusto"]
        AppInsights["Azure Application Insights"]
        Kusto["Azure Data Explorer (Kusto)<br/>DDAzureClients.DevCli"]
        RawTable["RawEventsAppRequests"]
    end

    subgraph Pipeline ["azd-queries (Pipeline & Governance)"]
        GDPR["GDPR Classify Pipeline<br/>eng/pipelines/classify.yml"]
        GDPRTool["gdpr tool<br/>(export → publish → ingest)"]
        GDPRAPI["GDPR API"]
        TableSync["Kusto Table Sync<br/>.github/workflows/ci.yml"]
        IngestScripts["Ingest Scripts<br/>(templates, template versions)"]
        KQLLib["KQL Query Library<br/>(core-usage, insights, aspire, vscode)"]
    end

    subgraph Analysis ["azure-dev-tools (Analysis)"]
        KustoFn["Kusto Functions<br/>(getAzdEvents, addTemplateColumns, etc.)"]
        PBI["Power BI Reports<br/>(KPIs, funnels, user journeys)"]
        Investigations["Ad-hoc Investigations"]
    end

    CLI --> MW --> OTel --> AIExp --> DiskQ --> Upload --> AppInsights
    VSC -->|VS Code telemetry framework| AppInsights
    EXT -->|structured errors via host| MW
    AppInsights --> Kusto --> RawTable

    GDPR -->|reads azure-dev source| GDPRTool --> GDPRAPI
    TableSync --> Kusto
    IngestScripts --> Kusto
    KQLLib -->|queries| RawTable

    KustoFn -->|deployed to DDAzureClients.DevCli| RawTable
    PBI -->|reads via| KustoFn
    Investigations -->|ad-hoc KQL| RawTable

Loading

What is added

• docs/architecture/telemetry.md — architecture and data flow
• docs/reference/telemetry-data.md — events, fields, schema, KQL patterns
• docs/reference/telemetry-dashboards.md — Kusto functions, Power BI reports, analysis tools
• docs/guides/feature-telemetry.md — how to instrument telemetry for new features
• docs/guides/telemetry-overview.md — product-facing overview of metrics and dashboards
• Updated docs/README.md with links to the above

Partially contributes to Azure/azure-dev-pr#1772.

Note: Once #8041 merges, will incorporate its extension/error attribute additions into the data reference.