Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 139 additions & 0 deletions documentation/specs/multithreading/thread-safe-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,145 @@ public bool Execute(...)
}
```

## Managing Static State Across Builds

### Problem: Static State Leaks Across Builds

MSBuild reuses worker nodes across builds by default. Any static field in a task that runs on a reused node retains its value from previous builds. Tasks that run exclusively on the main (scheduler) node were historically unaffected because the main process exited after each build. With MSBuild Server, the main node also persists, extending this problem to those tasks as well. In multithreaded builds, static fields introduce an additional risk: concurrent tasks sharing the same static field can cause race conditions.

Static fields in tasks can:

- **Leak data across builds** — a static cache populated during one build remains populated for subsequent builds on reused nodes, even if project state has changed.
- **Cause thread-safety issues** — in multithreaded builds, concurrent tasks sharing a static field can race unless the field is designed for concurrent access.

Thread-safety of static fields is the task author's responsibility, same as in any multithreaded application. For the data leak problem, MSBuild provides `IBuildEngine4.RegisterTaskObject` — an API that lets tasks store objects with explicit, engine-managed lifetimes instead of relying on static fields.

```csharp
void IBuildEngine4.RegisterTaskObject(object key, object obj, RegisteredTaskObjectLifetime lifetime, bool allowEarlyCollection);
object IBuildEngine4.GetRegisteredTaskObject(object key, RegisteredTaskObjectLifetime lifetime);
object IBuildEngine4.UnregisterTaskObject(object key, RegisteredTaskObjectLifetime lifetime);
```

The engine stores registered objects. All three methods are thread-safe and may be called concurrently from multiple tasks. If multiple tasks attempt to register an object with the same key concurrently, only the first registration takes effect — subsequent calls are ignored. When an object's lifetime expires, MSBuild calls `IDisposable.Dispose` on it if it implements `IDisposable`, then removes it.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this order right? Shouldn't it unregister (so nothing else can get it) then dispose it?

Copy link
Copy Markdown
Member Author

@AR-May AR-May Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this happens no task is running, so the order should not matter. From the code, the order is "dispose everything and then clear the dictionary".


`RegisteredTaskObjectLifetime` controls when objects are disposed:

| Lifetime | Disposed When | Use Case |
|----------|---------------|----------|
| `Build` | The build completes | Per-build caches and resources that must not leak across builds. |
| `AppDomain` | The MSBuild process exits | Objects that are safe to share across builds. |

With MSBuild Server, `Build` lifetime objects are disposed between each build request, giving task authors the same isolation they previously got from process-level separation.

### Example: Migrating a Static Cache

**Before — static cache that leaks across builds:**

```csharp
public class MyTask : Task
{
private static readonly Dictionary<string, string> s_cache = new();

public override bool Execute()
{
...
s_cache[key] = value;
...
}
}
```

This cache persists across builds on reused nodes. It is also not thread-safe for concurrent access.

**After — engine-managed lifetime:**

```csharp
public class MyTask : Task
{
private const string CacheKey = "MyNamespace.MyTask.Cache";

public override bool Execute()
{
var engine4 = (IBuildEngine4)BuildEngine;

var cache = (ConcurrentDictionary<string, string>)engine4.GetRegisteredTaskObject(
CacheKey, RegisteredTaskObjectLifetime.Build);
if (cache is null)
{
engine4.RegisterTaskObject(
CacheKey, new ConcurrentDictionary<string, string>(),
RegisteredTaskObjectLifetime.Build,
allowEarlyCollection: true);
// Re-read to get the authoritative instance in case another
// task registered first.
cache = (ConcurrentDictionary<string, string>)engine4.GetRegisteredTaskObject(
CacheKey, RegisteredTaskObjectLifetime.Build);
}

cache[key] = value;
...
}
}
```

The cache is now scoped to a single build and automatically discarded when the build completes.

### Cleanup-on-Dispose Pattern

When a static cache is used by utility classes or helper methods that do not have access to `IBuildEngine`, it cannot be replaced with a registered task object. Instead, the task may keep the static field and register a disposable wrapper that clears it when the build ends:

```csharp
internal static class MyHelper
{
// Static cache accessed by helper methods that have no IBuildEngine.
private static readonly ConcurrentDictionary<string, string> s_cache = new();
internal static void ClearCache() => s_cache.Clear();
}

public class MyTask : Task
{
private const string CleanerKey = "MyNamespace.MyTask.CacheCleaner";

public override bool Execute()
{
// Register a one-time cleanup wrapper so the static cache is
// cleared when the build ends and does not leak into future builds.
var engine4 = (IBuildEngine4)BuildEngine;
if (engine4.GetRegisteredTaskObject(CleanerKey, RegisteredTaskObjectLifetime.Build) is null)
{
// If another task instance races ahead, only one registration wins.
// This is safe: at least one cleanup wrapper will be registered.
engine4.RegisterTaskObject(
CleanerKey,
new CacheCleanup(MyHelper.ClearCache),
RegisteredTaskObjectLifetime.Build,
allowEarlyCollection: false);
}
...
}

/// <summary>
/// Invokes a cleanup delegate when disposed. Register with
/// RegisterTaskObject so MSBuild calls Dispose at end of build.
/// </summary>
private sealed class CacheCleanup(Action onDispose) : IDisposable
{
public void Dispose() => onDispose();
}
}
```

The same pattern can be applieed to third-party libraries that maintain their own static state — task may register a cleanup wrapper that calls the library's cache-clearing API.

`allowEarlyCollection` need to be set to `false` because early collection would trigger the cleanup mid-build. The static cache would then continue accumulating entries for the remainder of the build with no end-of-build cleanup.

### Guidelines for Task Authors

1. **Set `allowEarlyCollection: true`** when the cached data can be safely recreated. This lets MSBuild reclaim memory under pressure. Use `false` only for objects that must survive the entire build (e.g., cleanup wrappers, long-lived connections).
2. **Use a stable, unique key.** A `const string` with the fully-qualified task name avoids collisions (e.g., `"MyNamespace.MyTask.Cache"`).
3. **Handle null returns.** `GetRegisteredTaskObject` returns null when no object is registered under the key, or when a previously registered object was disposed through early collection.
4. **Registered objects must be thread-safe** in multithreaded builds, since multiple task instances may retrieve and use the same object concurrently.

## Appendix: Alternatives

This appendix collects alternative approaches considered during design.
Expand Down
12 changes: 12 additions & 0 deletions src/Framework/IBuildEngine4.cs
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,12 @@ public interface IBuildEngine4 : IBuildEngine3
/// manage limited process memory resources.
/// </para>
/// <para>
/// This method is thread-safe. If multiple threads concurrently attempt to register an object with the
/// same <paramref name="key"/> and <paramref name="lifetime"/>, only the first registration takes effect
/// and subsequent registrations are ignored. Callers should use <see cref="GetRegisteredTaskObject"/> after
/// registration to obtain the authoritative instance.
Comment on lines +54 to +57
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new doc asserts a “first registration wins; subsequent registrations are ignored” contract for RegisterTaskObject. While that matches MSBuild’s main implementation (uses ConcurrentDictionary.TryAdd), it is not universally true even within this repo (e.g., MockEngine.RegisterTaskObject overwrites existing values). Consider either (1) qualifying the statement as applying to MSBuild’s implementation specifically, or (2) updating other implementations to match this contract so the interface documentation remains accurate.

Suggested change
/// This method is thread-safe. If multiple threads concurrently attempt to register an object with the
/// same <paramref name="key"/> and <paramref name="lifetime"/>, only the first registration takes effect
/// and subsequent registrations are ignored. Callers should use <see cref="GetRegisteredTaskObject"/> after
/// registration to obtain the authoritative instance.
/// This method is thread-safe and may be called concurrently from multiple threads. Implementations may
/// choose different strategies when multiple threads attempt to register an object with the same
/// <paramref name="key"/> and <paramref name="lifetime"/>. In MSBuild's default build engine
/// implementation, only the first registration takes effect and subsequent registrations are ignored;
/// callers should use <see cref="GetRegisteredTaskObject"/> after registration to obtain the authoritative
/// instance.

Copilot uses AI. Check for mistakes.
/// </para>
/// <para>
/// The thread on which the object is disposed may be arbitrary - however it is guaranteed not to
/// be disposed while the task is executing, even if <paramref name="allowEarlyCollection"/> is set
/// to true.
Expand All @@ -72,6 +78,9 @@ public interface IBuildEngine4 : IBuildEngine3
/// The registered object, or null is there is no object registered under that key or the object
/// has been discarded through early collection.
/// </returns>
/// <remarks>
/// This method is thread-safe and may be called concurrently from multiple tasks.
/// </remarks>
object GetRegisteredTaskObject(object key, RegisteredTaskObjectLifetime lifetime);

/// <summary>
Expand All @@ -83,6 +92,9 @@ public interface IBuildEngine4 : IBuildEngine3
/// The registered object, or null is there is no object registered under that key or the object
/// has been discarded through early collection.
/// </returns>
/// <remarks>
/// This method is thread-safe and may be called concurrently from multiple tasks.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// This method is thread-safe and may be called concurrently from multiple tasks.
/// This method is thread-safe and may be called concurrently from multiple tasks. However, another task may be using the object after this method returns.

The implications of this kinda make my head hurt. Do we know of uses of this API? I might want to look at them.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have not found any uses neither in MSBuild repo, nor in sdk. It does not make much sense to unregister the object imo, as engine handles the clean-up. Cannot imagine a good use case. And it is really bad in mt mode. Especially if it is a IDisposable that the task may (and should) dispose after unregistering and it may happen during other task using the same object.

Should we attempt to collect telemetry on the usage?

/// </remarks>
object UnregisterTaskObject(object key, RegisteredTaskObjectLifetime lifetime);
}
}