From 99927b6d0ece28ff6a391dd448c4d0e405443926 Mon Sep 17 00:00:00 2001 From: Jaroslav Bachorik Date: Tue, 7 Apr 2026 16:55:36 +0200 Subject: [PATCH 1/7] docs(profiler): mark Java live heap profiling as GA Co-Authored-By: Claude Sonnet 4.6 --- content/en/profiler/profile_types.md | 10 ++--- .../profiler/profiler_troubleshooting/java.md | 41 +++++-------------- 2 files changed, 14 insertions(+), 37 deletions(-) diff --git a/content/en/profiler/profile_types.md b/content/en/profiler/profile_types.md index 8e3c7acb20c..cb6faa0ef78 100644 --- a/content/en/profiler/profile_types.md +++ b/content/en/profiler/profile_types.md @@ -32,13 +32,9 @@ Allocated Memory : The amount of heap memory allocated by each method, including allocations which were subsequently freed.
_Requires: Java 11_ -Heap Live Objects (in Preview, 1.17.0+) -: The number of objects allocated by each method in heap memory that have not yet been garbage collected. This is useful for investigating the overall memory usage of your service and identifying potential memory leaks.
-_Requires: Java 11_
- -Heap Live Size (in Preview, 1.39.0+) -: The amount of heap memory allocated by each method that has not yet been garbage collected. This is useful for investigating the overall memory usage of your service and identifying potential memory leaks.
-_Requires: Java 11.0.23+, 17.0.11+, 21.0.3+ or 22+_
+Live Heap (v1.61.0+) +: The objects and memory allocated by each method that have not yet been garbage collected. This is useful for investigating the overall memory usage of your service and identifying potential memory leaks. The profiler automatically uses the most accurate method available for your JVM version.
+_Requires: Java 11+_ Wall Time in Native Code : The elapsed time spent by each method. Elapsed time includes time when code is running on CPU, waiting for I/O, and anything else that happens while the method is running. diff --git a/content/en/profiler/profiler_troubleshooting/java.md b/content/en/profiler/profiler_troubleshooting/java.md index 026969aa90b..b87725b6401 100644 --- a/content/en/profiler/profiler_troubleshooting/java.md +++ b/content/en/profiler/profiler_troubleshooting/java.md @@ -80,41 +80,20 @@ jdk.ObjectAllocationSample#enabled=true ### Live heap profiling -Live Heap Profiling is a Datadog profiler feature that helps identify memory leaks and understand heap usage by tracking which allocated objects survive garbage collection. Unlike standard allocation profiling (which shows what was allocated), live heap profiling shows what is still alive on the heap — making it more useful for diagnosing memory leaks. +Live heap profiling tracks which allocated objects survive garbage collection, making it useful for diagnosing memory leaks and understanding overall heap usage. Starting with dd-trace-java **v1.61.0**, it is **enabled by default** on Linux. The profiler automatically selects the most accurate engine available for your JVM version. -The live-heap profiler engine: +The live-heap profiler: - Useful for investigating the overall memory usage of your service and identifying potential memory leaks -- Samples allocations and keeps track of whether those samples survived the most recent garbage collection cycle -- Uses the number of surviving samples to estimate the number of live objects in the heap -- Limits the number of tracked samples to avoid unbounded growth of the profiler's memory usage -- Disabled by default +- Samples allocations and tracks whether those samples survived the most recent garbage collection cycle +- Automatically uses the Datadog profiler engine on JDK 11.0.23+, 17.0.11+, 21.0.3+, or 22+, and falls back to the JFR engine (`jdk.OldObjectSample`) on older supported JDK versions - Not available on Windows -- For JMC users, the Datadog live-heap event is `datadog.HeapLiveObject` +- For JMC users, the Datadog live-heap event is `datadog.HeapLiveObject`; the JFR fallback emits `jdk.OldObjectSample` -| Configuration | Environment Variable | System Property | -|---------------|---------------------|-----------------| -| **Enable** (Requires JDK 11.0.23+, 17.0.11+, 21.0.3+, or 22+) | `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED=true` | `-Ddd.profiling.ddprof.liveheap.enabled=true` | - -**Note**: The live-heap engine does not depend on the `/proc/sys/kernel/perf_event_paranoid` setting. - - -### Heap profiling - -Heap profiling uses the JVM's built-in `jdk.OldObjectSample` JFR event to track objects that have been live in the heap for a sustained period of time. This helps identify potential memory leaks by showing which objects are accumulating over time. Unlike live heap profiling (which uses the Datadog profiler engine), heap profiling relies on JFR and is available on all platforms where JFR is supported. - -
This feature requires at least Java 11.0.12, 15.0.4, 16.0.2, 17.0.3 or 18 and newer
- -To enable the heap profiler, start your application with one of the following: +To disable live heap profiling: | Environment Variable | System Property | |---------------------|-----------------| -| `DD_PROFILING_HEAP_ENABLED=true` | `-Ddd.profiling.heap.enabled=true` | - -Alternatively, you can enable the following events in your `jfp` [override template file](#creating-and-using-a-jfr-template-override-file): - -``` -jdk.OldObjectSample#enabled=true -``` +| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED=false` | `-Ddd.profiling.ddprof.liveheap.enabled=false` | ### Heap histogram metrics @@ -142,13 +121,15 @@ The [Trace to Profiling integration][3] identifies code hotspots in slow traces. The following settings allow fine-grained control over the profiler engines. These are typically not needed for standard use cases. For detailed information about each profiler type, see the corresponding sections above: [CPU profiling](#cpu-profiling), [Wallclock](#wallclock), [Allocation profiling](#allocation-profiling), and [Live heap profiling](#live-heap-profiling). +**Note**: `DD_PROFILING_HEAP_ENABLED` / `-Ddd.profiling.heap.enabled` is a legacy setting that controls the JFR-based heap profiling fallback in isolation. In v1.61.0+, use `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` to enable or disable live heap profiling as a whole. + | Environment variable | System property | Description | |---------------------|-----------------|-------------| | `DD_PROFILING_DDPROF_ENABLED` | `-Ddd.profiling.ddprof.enabled` | Enable the Datadog profiler engine (Linux only, default: true since v1.7.0) | | `DD_PROFILING_DDPROF_CPU_ENABLED` | `-Ddd.profiling.ddprof.cpu.enabled` | Enable CPU profiling with the Datadog engine | | `DD_PROFILING_DDPROF_WALL_ENABLED` | `-Ddd.profiling.ddprof.wall.enabled` | Enable wallclock profiling (default: true since v1.7.0) | | `DD_PROFILING_DDPROF_ALLOC_ENABLED` | `-Ddd.profiling.ddprof.alloc.enabled` | Enable allocation profiling with the Datadog engine | -| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Enable live heap profiling (requires JDK 11.0.23+, 17.0.11+, 21.0.3+, or 22+) | +| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Enable or disable live heap profiling (default: true since v1.61.0; requires JDK 11+ with automatic engine selection) | | `DD_PROFILING_ENABLED_EVENTS` | `-Ddd.profiling.enabled.events` | Enable specific JFR events (for example: `jdk.ObjectAllocationInNewTLAB,jdk.ObjectAllocationOutsideTLAB`) | ### JDK Mission Control (JMC) event reference @@ -160,7 +141,7 @@ If you are analyzing profiles with JDK Mission Control, the following table prov | CPU | `jdk.ExecutionSample` | `datadog.ExecutionSample` | | Wallclock | - | `datadog.MethodSample` | | Allocation | `jdk.ObjectAllocationInNewTLAB`, `jdk.ObjectAllocationOutsideTLAB` | `datadog.ObjectAllocationInNewTLAB`, `datadog.ObjectAllocationOutsideTLAB` | -| Live heap | - | `datadog.HeapLiveObject` | +| Live heap | `jdk.OldObjectSample` | `datadog.HeapLiveObject` | ## Advanced configuration From da4c5703a6adc4597597c74c4cab26febfa583d1 Mon Sep 17 00:00:00 2001 From: Jaroslav Bachorik Date: Wed, 8 Apr 2026 23:25:14 +0200 Subject: [PATCH 2/7] docs(profiler): fix live heap profiling config key docs - DD_PROFILING_HEAP_ENABLED is the master toggle (gates both ddprof and JFR fallback), not a legacy key - DD_PROFILING_DDPROF_LIVEHEAP_ENABLED is ddprof-specific override only - Split disable instructions into two cases: all vs ddprof-only - Add DD_PROFILING_HEAP_ENABLED to config reference table Rationale: source code in dd-trace-java confirms DD_PROFILING_HEAP_ENABLED gates both engines; previous docs had the relationship backwards --- .../en/profiler/profiler_troubleshooting/java.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/content/en/profiler/profiler_troubleshooting/java.md b/content/en/profiler/profiler_troubleshooting/java.md index b87725b6401..b04a96e1582 100644 --- a/content/en/profiler/profiler_troubleshooting/java.md +++ b/content/en/profiler/profiler_troubleshooting/java.md @@ -89,7 +89,13 @@ The live-heap profiler: - Not available on Windows - For JMC users, the Datadog live-heap event is `datadog.HeapLiveObject`; the JFR fallback emits `jdk.OldObjectSample` -To disable live heap profiling: +To disable live heap profiling entirely (both Datadog engine and JFR fallback): + +| Environment Variable | System Property | +|---------------------|-----------------| +| `DD_PROFILING_HEAP_ENABLED=false` | `-Ddd.profiling.heap.enabled=false` | + +To disable only the Datadog native engine implementation (the JFR fallback `jdk.OldObjectSample` remains active on older JDKs): | Environment Variable | System Property | |---------------------|-----------------| @@ -121,15 +127,16 @@ The [Trace to Profiling integration][3] identifies code hotspots in slow traces. The following settings allow fine-grained control over the profiler engines. These are typically not needed for standard use cases. For detailed information about each profiler type, see the corresponding sections above: [CPU profiling](#cpu-profiling), [Wallclock](#wallclock), [Allocation profiling](#allocation-profiling), and [Live heap profiling](#live-heap-profiling). -**Note**: `DD_PROFILING_HEAP_ENABLED` / `-Ddd.profiling.heap.enabled` is a legacy setting that controls the JFR-based heap profiling fallback in isolation. In v1.61.0+, use `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` to enable or disable live heap profiling as a whole. +**Note**: `DD_PROFILING_HEAP_ENABLED` / `-Ddd.profiling.heap.enabled` is the main toggle for all live heap profiling. When set to `false`, it disables both the Datadog native engine and the JFR `jdk.OldObjectSample` fallback. `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` is a Datadog-engine-specific override that only controls the native implementation; the JFR fallback is unaffected by it. | Environment variable | System property | Description | |---------------------|-----------------|-------------| +| `DD_PROFILING_HEAP_ENABLED` | `-Ddd.profiling.heap.enabled` | Master toggle for all live heap profiling, including both the Datadog native engine and the JFR `jdk.OldObjectSample` fallback (default: true on supported platforms since v1.61.0) | | `DD_PROFILING_DDPROF_ENABLED` | `-Ddd.profiling.ddprof.enabled` | Enable the Datadog profiler engine (Linux only, default: true since v1.7.0) | | `DD_PROFILING_DDPROF_CPU_ENABLED` | `-Ddd.profiling.ddprof.cpu.enabled` | Enable CPU profiling with the Datadog engine | | `DD_PROFILING_DDPROF_WALL_ENABLED` | `-Ddd.profiling.ddprof.wall.enabled` | Enable wallclock profiling (default: true since v1.7.0) | | `DD_PROFILING_DDPROF_ALLOC_ENABLED` | `-Ddd.profiling.ddprof.alloc.enabled` | Enable allocation profiling with the Datadog engine | -| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Enable or disable live heap profiling (default: true since v1.61.0; requires JDK 11+ with automatic engine selection) | +| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Override for the Datadog native engine live heap implementation only; does not affect the JFR fallback (default: enabled on JDK 11+ when safe, requires `DD_PROFILING_HEAP_ENABLED=true`) | | `DD_PROFILING_ENABLED_EVENTS` | `-Ddd.profiling.enabled.events` | Enable specific JFR events (for example: `jdk.ObjectAllocationInNewTLAB,jdk.ObjectAllocationOutsideTLAB`) | ### JDK Mission Control (JMC) event reference From 3e72de7b208670130060ad70831697579fef3085 Mon Sep 17 00:00:00 2001 From: Jaroslav Bachorik Date: Fri, 10 Apr 2026 18:37:51 +0200 Subject: [PATCH 3/7] Update content/en/profiler/profiler_troubleshooting/java.md Co-authored-by: Bryce Eadie --- content/en/profiler/profiler_troubleshooting/java.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/profiler/profiler_troubleshooting/java.md b/content/en/profiler/profiler_troubleshooting/java.md index b04a96e1582..8ba4e680448 100644 --- a/content/en/profiler/profiler_troubleshooting/java.md +++ b/content/en/profiler/profiler_troubleshooting/java.md @@ -131,7 +131,7 @@ The following settings allow fine-grained control over the profiler engines. The | Environment variable | System property | Description | |---------------------|-----------------|-------------| -| `DD_PROFILING_HEAP_ENABLED` | `-Ddd.profiling.heap.enabled` | Master toggle for all live heap profiling, including both the Datadog native engine and the JFR `jdk.OldObjectSample` fallback (default: true on supported platforms since v1.61.0) | +| `DD_PROFILING_HEAP_ENABLED` | `-Ddd.profiling.heap.enabled` | Main toggle for all live heap profiling, including both the Datadog native engine and the JFR `jdk.OldObjectSample` fallback (default: true on supported platforms since v1.61.0) | | `DD_PROFILING_DDPROF_ENABLED` | `-Ddd.profiling.ddprof.enabled` | Enable the Datadog profiler engine (Linux only, default: true since v1.7.0) | | `DD_PROFILING_DDPROF_CPU_ENABLED` | `-Ddd.profiling.ddprof.cpu.enabled` | Enable CPU profiling with the Datadog engine | | `DD_PROFILING_DDPROF_WALL_ENABLED` | `-Ddd.profiling.ddprof.wall.enabled` | Enable wallclock profiling (default: true since v1.7.0) | From 67e37c0f5dfbddbe9a3dd068bd6d6f1f427f2d40 Mon Sep 17 00:00:00 2001 From: Jaroslav Bachorik Date: Fri, 10 Apr 2026 18:39:12 +0200 Subject: [PATCH 4/7] Update content/en/profiler/profiler_troubleshooting/java.md Co-authored-by: Bryce Eadie --- content/en/profiler/profiler_troubleshooting/java.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/profiler/profiler_troubleshooting/java.md b/content/en/profiler/profiler_troubleshooting/java.md index 8ba4e680448..7c3f352faf0 100644 --- a/content/en/profiler/profiler_troubleshooting/java.md +++ b/content/en/profiler/profiler_troubleshooting/java.md @@ -136,7 +136,7 @@ The following settings allow fine-grained control over the profiler engines. The | `DD_PROFILING_DDPROF_CPU_ENABLED` | `-Ddd.profiling.ddprof.cpu.enabled` | Enable CPU profiling with the Datadog engine | | `DD_PROFILING_DDPROF_WALL_ENABLED` | `-Ddd.profiling.ddprof.wall.enabled` | Enable wallclock profiling (default: true since v1.7.0) | | `DD_PROFILING_DDPROF_ALLOC_ENABLED` | `-Ddd.profiling.ddprof.alloc.enabled` | Enable allocation profiling with the Datadog engine | -| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Override for the Datadog native engine live heap implementation only; does not affect the JFR fallback (default: enabled on JDK 11+ when safe, requires `DD_PROFILING_HEAP_ENABLED=true`) | +| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Override for the Datadog native engine live heap implementation only; does not affect the JFR fallback (default: enabled on supported JDK versions (11.0.23+, 17.0.11+, 21.0.3+, or 22+), requires DD_PROFILING_HEAP_ENABLED=true`) | | `DD_PROFILING_ENABLED_EVENTS` | `-Ddd.profiling.enabled.events` | Enable specific JFR events (for example: `jdk.ObjectAllocationInNewTLAB,jdk.ObjectAllocationOutsideTLAB`) | ### JDK Mission Control (JMC) event reference From 7a49b375e3a614a5fd4edb1f50531cb308baf8ae Mon Sep 17 00:00:00 2001 From: Jaroslav Bachorik Date: Fri, 10 Apr 2026 18:39:46 +0200 Subject: [PATCH 5/7] Update content/en/profiler/profiler_troubleshooting/java.md Co-authored-by: Bryce Eadie --- content/en/profiler/profiler_troubleshooting/java.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/content/en/profiler/profiler_troubleshooting/java.md b/content/en/profiler/profiler_troubleshooting/java.md index 7c3f352faf0..4e4c22572dc 100644 --- a/content/en/profiler/profiler_troubleshooting/java.md +++ b/content/en/profiler/profiler_troubleshooting/java.md @@ -127,7 +127,9 @@ The [Trace to Profiling integration][3] identifies code hotspots in slow traces. The following settings allow fine-grained control over the profiler engines. These are typically not needed for standard use cases. For detailed information about each profiler type, see the corresponding sections above: [CPU profiling](#cpu-profiling), [Wallclock](#wallclock), [Allocation profiling](#allocation-profiling), and [Live heap profiling](#live-heap-profiling). -**Note**: `DD_PROFILING_HEAP_ENABLED` / `-Ddd.profiling.heap.enabled` is the main toggle for all live heap profiling. When set to `false`, it disables both the Datadog native engine and the JFR `jdk.OldObjectSample` fallback. `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` is a Datadog-engine-specific override that only controls the native implementation; the JFR fallback is unaffected by it. +**Note**: + - `DD_PROFILING_HEAP_ENABLED` controls all live heap profiling. Setting it to `false` disables both the Datadog native engine and the JFR `jdk.OldObjectSample` fallback. + - `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` only controls the Datadog native engine. The JFR fallback is unaffected by this setting. | Environment variable | System property | Description | |---------------------|-----------------|-------------| From dfc557c18dc99248899b95f54057ee84e5796d34 Mon Sep 17 00:00:00 2001 From: Jaroslav Bachorik Date: Fri, 10 Apr 2026 18:39:54 +0200 Subject: [PATCH 6/7] Update content/en/profiler/profile_types.md Co-authored-by: Bryce Eadie --- content/en/profiler/profile_types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/profiler/profile_types.md b/content/en/profiler/profile_types.md index 274cde4ff94..e48f2d32d15 100644 --- a/content/en/profiler/profile_types.md +++ b/content/en/profiler/profile_types.md @@ -33,7 +33,7 @@ Allocated Memory _Requires: Java 11_ Live Heap (v1.61.0+) -: The objects and memory allocated by each method that have not yet been garbage collected. This is useful for investigating the overall memory usage of your service and identifying potential memory leaks. The profiler automatically uses the most accurate method available for your JVM version.
+: The objects and memory allocated by each method that have not yet been garbage collected. This is useful for investigating the overall memory usage of your service and identifying potential memory leaks. The profiler automatically uses the most accurate engine available for your JVM version.
_Requires: Java 11+_ Wall Time in Native Code From ded00ae9b64f49efbe63d6a7409e8dd52072bded Mon Sep 17 00:00:00 2001 From: Bryce Eadie Date: Fri, 10 Apr 2026 10:09:05 -0700 Subject: [PATCH 7/7] Fix formatting --- content/en/profiler/profiler_troubleshooting/java.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/profiler/profiler_troubleshooting/java.md b/content/en/profiler/profiler_troubleshooting/java.md index 4e4c22572dc..982ae28e175 100644 --- a/content/en/profiler/profiler_troubleshooting/java.md +++ b/content/en/profiler/profiler_troubleshooting/java.md @@ -138,7 +138,7 @@ The following settings allow fine-grained control over the profiler engines. The | `DD_PROFILING_DDPROF_CPU_ENABLED` | `-Ddd.profiling.ddprof.cpu.enabled` | Enable CPU profiling with the Datadog engine | | `DD_PROFILING_DDPROF_WALL_ENABLED` | `-Ddd.profiling.ddprof.wall.enabled` | Enable wallclock profiling (default: true since v1.7.0) | | `DD_PROFILING_DDPROF_ALLOC_ENABLED` | `-Ddd.profiling.ddprof.alloc.enabled` | Enable allocation profiling with the Datadog engine | -| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Override for the Datadog native engine live heap implementation only; does not affect the JFR fallback (default: enabled on supported JDK versions (11.0.23+, 17.0.11+, 21.0.3+, or 22+), requires DD_PROFILING_HEAP_ENABLED=true`) | +| `DD_PROFILING_DDPROF_LIVEHEAP_ENABLED` | `-Ddd.profiling.ddprof.liveheap.enabled` | Override for the Datadog native engine live heap implementation only; does not affect the JFR fallback (default: enabled on supported JDK versions (11.0.23+, 17.0.11+, 21.0.3+, or 22+), requires `DD_PROFILING_HEAP_ENABLED=true`) | | `DD_PROFILING_ENABLED_EVENTS` | `-Ddd.profiling.enabled.events` | Enable specific JFR events (for example: `jdk.ObjectAllocationInNewTLAB,jdk.ObjectAllocationOutsideTLAB`) | ### JDK Mission Control (JMC) event reference