Skip to content

Add telemetry support for HTTP request performance monitoring - #90

Draft
Burgyn with Copilot wants to merge 3 commits into
masterfrom
copilot/add-telemetry-support
Draft

Add telemetry support for HTTP request performance monitoring#90
Burgyn with Copilot wants to merge 3 commits into
masterfrom
copilot/add-telemetry-support

Conversation

Copilot AI commented Mar 22, 2026

Copy link
Copy Markdown
Contributor

Adds structured performance telemetry collection for HTTP requests, retry operations, and collection execution. Telemetry is opt-in via the scripting API and renders a Spectre.Console performance summary at the end of a run.

Data model

  • HttpRequestTelemetry — per-request metrics (method, URL, status, duration, retry count)
  • TelemetryData — aggregated stats with computed properties (success rate, throughput, fastest/slowest/most-retried)

Collection & extensibility

  • ITelemetryCollector / InMemoryTelemetryCollector — singleton collector, no-ops when disabled
  • ITelemetryProvider — extension point for external systems (App Insights, OpenTelemetry, etc.)
  • TelemetryOptions — toggle console output, plug in custom providers

Integration

  • ExecuteRequestStep wraps the resilience pipeline call with a Stopwatch and records request telemetry including retry attempt count
  • ReportTelemetrySummaryStep added to the default pipeline after test results reporting
  • DI wired in Setup.csAddTelemetry()

Scripting API

// In init.csx — default (in-memory + console summary)
tp.EnableTelemetry();

// With custom provider
tp.EnableTelemetry(options => {
    options.UseCustomProvider(new MyTelemetryProvider());
    options.EnableConsoleOutput = false;
});

Tests

29 new tests covering TelemetryData aggregation, InMemoryTelemetryCollector enable/disable/recording/provider-failure resilience, HttpRequestTelemetry success classification, and ReportTelemetrySummaryStep conditional execution.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.deps.json /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/testhost.dll --port 41577 --endpoint 127.0.0.1:041577 --role client --parentprocessid 4580 --telemetryoptedin false (dns block)
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.deps.json /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/testhost.dll --port 44463 --endpoint 127.0.0.1:044463 --role client --parentprocessid 5471 --telemetryoptedin false (dns block)
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.deps.json /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/testhost.dll --port 45169 --endpoint 127.0.0.1:045169 --role client --parentprocessid 6522 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>🚀 [NEW] Telemetry Support</issue_title>
<issue_description>## 🫴 Proposal

Add comprehensive telemetry support to TeaPie, enabling detailed performance monitoring and statistics collection for HTTP requests, retry operations, and overall collection execution through standard .NET telemetry APIs.
The goal is to provide users with actionable insights about their API test performance while maintaining seamless integration with external monitoring systems like Application Insights, OpenTelemetry, or custom telemetry providers.

Default In-Memory Implementation:

// In init.csx - telemetry enabled by default
tp.EnableTelemetry(); // Optional call - can be enabled by default

Application Insights Telemetry:

// In init.csx
tp.EnableTelemetry(options => {
    options.UseApplicationInsights("instrumentation-key");
    options.EnableDetailedTracing = true;
    options.SampleRate = 1.0; // 100% sampling for development
});

Custom Telemetry Provider:

// In init.csx  
tp.EnableTelemetry(options => {
    options.UseCustomProvider(new MyTelemetryProvider());
    options.TrackMetrics("http_requests", "retry_attempts", "test_duration");
    options.EnableConsoleOutput = false; // Disable default console summary
});

✨ Motivation

When running extensive API test collections, developers often need to answer questions like:

  • Which requests are consistently slow and need optimization?
  • How often do retries occur and what's their success rate?
  • What's the overall performance trend of the API under test?
  • Are there performance regressions between test runs?

Currently, TeaPie provides excellent test results and detailed logging, but lacks structured performance insights that could help:

  • Identify performance bottlenecks in API endpoints
  • Monitor retry patterns and optimize retry strategies
  • Track performance trends over time for regression detection
  • Generate reports for stakeholders about API reliability
  • Integrate with monitoring systems like Application Insights

This telemetry support would transform TeaPie from just a testing tool into a comprehensive API performance monitoring solution, providing valuable data for both development and operations teams without requiring any external tools for basic insights.

Example Console Output

┌─── 📊 Performance Summary ─────────────────────────────────────────────────────────────┐
│                                                                                      │
│  🌐 HTTP: 47 requests • 95.7% success rate • 234ms avg • 45ms-1.2s range              │
│  🔄 Retry: 12 attempts • 5 requests (10.6%) • 83.3% success rate                       │
│  ⏱️  Tests: 23 cases • 28.4s total • 1.65 req/s throughput                             │ 
│                                                                                         │
│  🚀 Fastest: GET /api/users/123 (45ms)                                                 │
│  🐌 Slowest: POST /api/data/bulk-import (1.2s)                                         │
│  🔄 Most retried: POST /api/orders/process (4x)                                        │
│                                                                                        │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
```</issue_description>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits March 22, 2026 17:50
…toring

- Add TelemetryData and HttpRequestTelemetry data models for tracking HTTP request metrics
- Add ITelemetryCollector interface and InMemoryTelemetryCollector default implementation
- Add ITelemetryProvider interface for custom telemetry provider extensibility
- Add TelemetryOptions for configuring telemetry behavior (console output, custom providers)
- Add SpectreConsoleTelemetryReporter for rendering performance summary tables
- Add ReportTelemetrySummaryStep pipeline step for end-of-run telemetry reporting
- Add TeaPieTelemetryExtensions with EnableTelemetry() extension methods for tp scripting API
- Integrate telemetry collection into ExecuteRequestStep (HTTP duration, status, retries)
- Register telemetry services in DI Setup.cs
- Wire telemetry reporting step into default pipeline
- Add 28 unit tests covering TelemetryData, InMemoryTelemetryCollector, HttpRequestTelemetry, and ReportTelemetrySummaryStep

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Kros-sk/TeaPie/sessions/7d1128d9-d0d6-47ac-8409-dd00ac57f095
…vider errors

- Change HttpRequestTelemetry.Timestamp default from DateTime.Now to DateTime.UtcNow
- Properly await custom provider's OnRequestRecordedAsync with exception handling
- Add test for custom provider failure resilience

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Kros-sk/TeaPie/sessions/7d1128d9-d0d6-47ac-8409-dd00ac57f095
Copilot AI changed the title [WIP] Add telemetry support for performance monitoring Add telemetry support for HTTP request performance monitoring Mar 22, 2026
Copilot AI requested a review from Burgyn March 22, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🚀 [NEW] Telemetry Support

2 participants