From da4d268e723a6414d465d813693732c593288e7f Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Tue, 13 May 2025 06:18:50 -0400 Subject: [PATCH 1/8] Temporarily restore partial GH #4717 original CHANGES.txt text. Changes: * Partially restore restore original GH #4717 CHANGES.txt description * Change MSVS to MSVC in RELEASE.txt * Remove extraneous two blank lines introduced in common.py when variable moved --- CHANGES.txt | 15 +++++++-------- RELEASE.txt | 2 +- SCons/Tool/MSCommon/common.py | 2 -- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index b5df280824..0653e44d78 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,14 +16,13 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER - Whatever John Doe did. From Joseph Brill: - - MSVS: Fix a significant MSVC/MSVC tool initialization slowdown when - vcpkg has been installed and the PSModulePath is not initialized - or propagated from the user's shell environment. To resolve this - The default Windows Powershell 7 path is added before the default - Windows Powershell 5 path in the carefully constructed - environment in which the MSVC batch files are run. The shell - environment variables and values for VCPKG_DISABLE_METRICS and - VCPKG_ROOT are added to the limited MSVC environment when defined. + - MSVC: The default Windows powershell 7 path is added before the + default Windows powershell 5 path in the limited environment in + which the MSVC batch files are run. This appears to fix a runtime + issue in the windows runner environment when the embedded Visual + Studio vcpkg component initializes without a PSModulePath defined. + The shell environment variables and values for VCPKG_DISABLE_METRICS + and VCPKG_ROOT are added to the limited MSVC environment when defined. At present, the VCPKG_DISABLE_METRICS and VCPKG_ROOT variables and values are not propagated to the SCons environment after running the MSVC batch files. diff --git a/RELEASE.txt b/RELEASE.txt index c892409bbe..19dab93ddf 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -35,7 +35,7 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY one from PEP 308 introduced in Python 2.5 (2006). The idiom being replaced (using and/or) is regarded as error prone. -- MSVS: The default Windows powershell 7 path is added before the default +- MSVC: The default Windows powershell 7 path is added before the default Windows powershell 5 path in the limited environment in which the MSVC batch files are run. diff --git a/SCons/Tool/MSCommon/common.py b/SCons/Tool/MSCommon/common.py index effc5cf2d8..82ab06af20 100644 --- a/SCons/Tool/MSCommon/common.py +++ b/SCons/Tool/MSCommon/common.py @@ -402,8 +402,6 @@ def normalize_env(env, keys, force: bool=False): return normenv - - def get_output(vcbat, args=None, env=None, skip_sendtelemetry=False): """Parse the output of given bat file, with given args.""" From 0797b2dd7fa7aff1319af22b496345e4ed10e452 Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Tue, 13 May 2025 16:23:20 -0400 Subject: [PATCH 2/8] Update CHANGES.txt [skip ci] --- CHANGES.txt | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 0653e44d78..f5e64f693f 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,16 +16,22 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER - Whatever John Doe did. From Joseph Brill: - - MSVC: The default Windows powershell 7 path is added before the - default Windows powershell 5 path in the limited environment in - which the MSVC batch files are run. This appears to fix a runtime - issue in the windows runner environment when the embedded Visual - Studio vcpkg component initializes without a PSModulePath defined. - The shell environment variables and values for VCPKG_DISABLE_METRICS - and VCPKG_ROOT are added to the limited MSVC environment when defined. - At present, the VCPKG_DISABLE_METRICS and VCPKG_ROOT variables and - values are not propagated to the SCons environment after running - the MSVC batch files. + + - MSVC: A significant delay was experienced in the windows 2022 + runner and the windows 2025 runner when the MSVC batch files are + called during tool initialization due to the vcpkg component + installed with Visual Studio 2022. The vcpkg initialization delay + was significantly reduced in the windows 2022 and 2025 runner + environments by adding the default windows powershell 7 path + before the default powershell 5 path in the minimal environment in + which the MSVC batch files are run. Powershell 7 is known to be + installed in the windows runners. The Visual Studio vcpkg component + is not installed in the windows 2019 runner. The shell environment + variables and values for VCPKG_DISABLE_METRICS and VCPKG_ROOT are + added to the minimal environment in which the the MSVC batch files + are run. At present, the VCPKG_DISABLE_METRICS and VCPKG_ROOT + variables and values are not propagated to the SCons environment + after running the MSVC batch files. From Bill Prendergast: - Fixed SCons.Variables.PackageVariable to correctly test the default From 23eef475334358452ea947179eab3e88762306ff Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Tue, 13 May 2025 16:24:01 -0400 Subject: [PATCH 3/8] Update CHANGES.txt [skip ci] --- CHANGES.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGES.txt b/CHANGES.txt index f5e64f693f..dd5e5ce2e6 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,7 +16,6 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER - Whatever John Doe did. From Joseph Brill: - - MSVC: A significant delay was experienced in the windows 2022 runner and the windows 2025 runner when the MSVC batch files are called during tool initialization due to the vcpkg component From b39e674210d22bd2275a770640255d9e8bb0cf5d Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Mon, 26 May 2025 11:41:15 -0400 Subject: [PATCH 4/8] MSVC: Import POWERSHELL_TELEMETRY_OPTOUT, PSDisableModuleAnalysisCacheCleanup, and PSModuleAnalysisCachePath into the limited msvc environment when defined. Environment variables added to msvc environment import list: * POWERSHELL_TELEMETRY_OPTOUT: opt-out of telemetry. * PSDisableModuleAnalysisCacheCleanup: disable checking for modules that no longer exist when writing module analysis cache. * PSModuleAnalysisCachePath: path and filename used to cache data about modules and their cmdlets. The PSModuleAnalysisCachePath is defined in windows runners 2019, 2022, and 2025. The significant delays experienced in the 2022 and 2025 runners when running the msvc batch files due to vcpkg with powershell 5 and without PSModulePath defined appear to be eliminated when the powershell module analysis cache path is propagated to the msvc environment. Additional changes: * Move the PS7 path after the PS5 path. * Add the elapsed execution time of the msvc batch files to the debug log. --- RELEASE.txt | 3 ++- SCons/Tool/MSCommon/common.py | 23 ++++++++++++++++------- 2 files changed, 18 insertions(+), 8 deletions(-) diff --git a/RELEASE.txt b/RELEASE.txt index 19dab93ddf..34008fdc9a 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -44,7 +44,8 @@ FIXES - Fixed SCons.Variables.PackageVariable to correctly test the default setting against both enable & disable strings. (Fixes #4702) -- MSVS: Fix significant slowdown initializing MSVC tools when vcpkg has + +- MSVC: Fix significant slowdown initializing MSVC tools when vcpkg has been installed on the system. IMPROVEMENTS diff --git a/SCons/Tool/MSCommon/common.py b/SCons/Tool/MSCommon/common.py index 82ab06af20..a21518cc93 100644 --- a/SCons/Tool/MSCommon/common.py +++ b/SCons/Tool/MSCommon/common.py @@ -30,6 +30,7 @@ import os import re import sys +import time from contextlib import suppress from subprocess import DEVNULL, PIPE from pathlib import Path @@ -70,6 +71,9 @@ 'windir', # windows directory (SystemRoot not available in 95/98/ME) 'VCPKG_DISABLE_METRICS', 'VCPKG_ROOT', + 'POWERSHELL_TELEMETRY_OPTOUT', + 'PSDisableModuleAnalysisCacheCleanup', + 'PSModuleAnalysisCachePath', ] class MSVCCacheInvalidWarning(SCons.Warnings.WarningOnByDefault): @@ -379,6 +383,13 @@ def normalize_env(env, keys, force: bool=False): if sys32_wbem_dir not in normenv['PATH']: normenv['PATH'] = normenv['PATH'] + os.pathsep + sys32_wbem_dir + # Without Powershell in PATH, an internal call to a telemetry + # function (starting with a VS2019 update) can fail + # Note can also set VSCMD_SKIP_SENDTELEMETRY to avoid this. + sys32_ps_dir = os.path.join(sys32_dir, r'WindowsPowerShell\v1.0') + if sys32_ps_dir not in normenv['PATH']: + normenv['PATH'] = normenv['PATH'] + os.pathsep + sys32_ps_dir + # ProgramFiles for PowerShell 7 Path and PSModulePath progfiles_dir = os.environ.get("ProgramFiles") if not progfiles_dir: @@ -391,13 +402,6 @@ def normalize_env(env, keys, force: bool=False): if progfiles_ps_dir not in normenv["PATH"]: normenv["PATH"] = normenv["PATH"] + os.pathsep + progfiles_ps_dir - # Without Powershell in PATH, an internal call to a telemetry - # function (starting with a VS2019 update) can fail - # Note can also set VSCMD_SKIP_SENDTELEMETRY to avoid this. - sys32_ps_dir = os.path.join(sys32_dir, r'WindowsPowerShell\v1.0') - if sys32_ps_dir not in normenv['PATH']: - normenv['PATH'] = normenv['PATH'] + os.pathsep + sys32_ps_dir - debug("PATH: %s", normenv['PATH']) return normenv @@ -423,10 +427,15 @@ def get_output(vcbat, args=None, env=None, skip_sendtelemetry=False): debug("Calling '%s'", vcbat) cmd_str = '"%s" & set' % vcbat + beg_time = time.time() + cp = SCons.Action.scons_subproc_run( env, cmd_str, stdin=DEVNULL, stdout=PIPE, stderr=PIPE, ) + end_time = time.time() + debug("Elapsed %.2fs", end_time - beg_time) + # Extra debug logic, uncomment if necessary # debug('stdout:%s', cp.stdout) # debug('stderr:%s', cp.stderr) From 025a41d3bec96bdc544ada85e2326e8f047f56b4 Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Thu, 29 May 2025 12:12:24 -0400 Subject: [PATCH 5/8] MSVC: update the powershell executable order on the msvc environment PATH and add limited known shell PSModulePath paths. Changes: * Add the pwsh and powershell paths to the msvc environment PATH in the order discovered on the shell environment PATH. * Add the pwsh and powershell AllUsers and installation PSModule paths to the msvc environment PSModulePath in the order discovered on the shell environment PSModulePath. --- SCons/Tool/MSCommon/common.py | 117 +++++++++++++++++++++++++++++----- 1 file changed, 102 insertions(+), 15 deletions(-) diff --git a/SCons/Tool/MSCommon/common.py b/SCons/Tool/MSCommon/common.py index a21518cc93..7fcd43483b 100644 --- a/SCons/Tool/MSCommon/common.py +++ b/SCons/Tool/MSCommon/common.py @@ -343,6 +343,89 @@ def _force_vscmd_skip_sendtelemetry(env): return True +class _Powershell: + + _PSEXECUTABLES = ( + "pwsh.exe", + "powershell.exe", + ) + + _PSMODULEPATH_MAP = {os.path.normcase(os.path.abspath(p)): p for p in [ + # os.path.expandvars(r"%USERPROFILE%\Documents\PowerShell\Modules"), # current user + os.path.expandvars(r"%ProgramFiles%\PowerShell\Modules"), # all users + os.path.expandvars(r"%ProgramFiles%\PowerShell\7\Modules"), # installation location + # os.path.expandvars(r"%USERPROFILE%\Documents\WindowsPowerShell\Modules"), # current user + os.path.expandvars(r"%ProgramFiles%\WindowsPowerShell\Modules"), # all users + os.path.expandvars(r"%windir%\System32\WindowsPowerShell\v1.0\Modules"), # installation location + ]} + + _cache_norm_path = {} + + @classmethod + def _get_norm_path(cls, p): + norm_path = cls._cache_norm_path.get(p) + if norm_path is None: + norm_path = os.path.normcase(os.path.abspath(p)) + cls._cache_norm_path[p] = norm_path + cls._cache_norm_path[norm_path] = norm_path + return norm_path + + _cache_is_psmodulepath = {} + + @classmethod + def _is_psmodulepath(cls, p): + is_psmodulepath = cls._cache_is_psmodulepath.get(p) + if is_psmodulepath is None: + norm_path = cls._get_norm_path(p) + is_psmodulepath = bool(norm_path in cls._PSMODULEPATH_MAP) + cls._cache_is_psmodulepath[p] = is_psmodulepath + cls._cache_is_psmodulepath[norm_path] = is_psmodulepath + return is_psmodulepath + + _cache_psmodulepath = {} + + @classmethod + def get_psmodulepath(cls, pathspec): + psmodulepath = cls._cache_psmodulepath.get(pathspec) + if psmodulepath is None: + psmodulepath_paths = [] + for p in pathspec.split(os.pathsep): + p = p.strip() + if not p: + continue + if not cls._is_psmodulepath(p): + continue + psmodulepath_paths.append(p) + psmodulepath = os.pathsep.join(psmodulepath_paths) + cls._cache_psmodulepath[pathspec] = psmodulepath + return psmodulepath + + _cache_psexe_paths = {} + + @classmethod + def get_psexe_paths(cls, pathspec): + psexe_paths = cls._cache_psexe_paths.get(pathspec) + if psexe_paths is None: + psexe_set = set(cls._PSEXECUTABLES) + psexe_paths = [] + for p in pathspec.split(os.pathsep): + p = p.strip() + if not p: + continue + for psexe in psexe_set: + psexe_path = os.path.join(p, psexe) + if not os.path.exists(psexe_path): + continue + psexe_paths.append(p) + psexe_set.remove(psexe) + break + if psexe_set: + continue + break + cls._cache_psexe_paths[pathspec] = tuple(psexe_paths) + return psexe_paths + + def normalize_env(env, keys, force: bool=False): """Given a dictionary representing a shell environment, add the variables from os.environ needed for the processing of .bat files; the keys are @@ -386,23 +469,27 @@ def normalize_env(env, keys, force: bool=False): # Without Powershell in PATH, an internal call to a telemetry # function (starting with a VS2019 update) can fail # Note can also set VSCMD_SKIP_SENDTELEMETRY to avoid this. - sys32_ps_dir = os.path.join(sys32_dir, r'WindowsPowerShell\v1.0') - if sys32_ps_dir not in normenv['PATH']: - normenv['PATH'] = normenv['PATH'] + os.pathsep + sys32_ps_dir - - # ProgramFiles for PowerShell 7 Path and PSModulePath - progfiles_dir = os.environ.get("ProgramFiles") - if not progfiles_dir: - sysroot_drive, _ = os.path.splitdrive(sys32_dir) - sysroot_path = sysroot_drive + os.sep - progfiles_dir = os.path.join(sysroot_path, "Program Files") - - # Powershell 7 - progfiles_ps_dir = os.path.join(progfiles_dir, "PowerShell", "7") - if progfiles_ps_dir not in normenv["PATH"]: - normenv["PATH"] = normenv["PATH"] + os.pathsep + progfiles_ps_dir + + # Find the powershell executable paths. Add the known powershell.exe + # path to the end of the shell system path (just in case). + # The VS vcpkg component prefers pwsh.exe if it's on the path. + sys32_ps_dir = os.path.join(sys32_dir, 'WindowsPowerShell', 'v1.0') + psexe_searchlist = os.pathsep.join([os.environ.get("PATH", ""), sys32_ps_dir]) + psexe_pathlist = _Powershell.get_psexe_paths(psexe_searchlist) + + # Add powershell executable paths in the order discovered. + for psexe_path in psexe_pathlist: + if psexe_path not in normenv['PATH']: + normenv['PATH'] = normenv['PATH'] + os.pathsep + psexe_path debug("PATH: %s", normenv['PATH']) + + # Add psmodulepath paths in the order discovered. + psmodulepath = _Powershell.get_psmodulepath(os.environ.get("PSModulePath", "")) + if psmodulepath: + normenv["PSModulePath"] = psmodulepath + + debug("PSModulePath: %s", normenv.get('PSModulePath','')) return normenv From f25895b054dc40ab0558e5cd8dae9cf457d9f1aa Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Thu, 29 May 2025 13:03:23 -0400 Subject: [PATCH 6/8] MSVC: update CHANGES.txt and RELEASE.txt. --- CHANGES.txt | 39 +++++++++++++++++++++++++-------------- RELEASE.txt | 15 ++++++++++----- 2 files changed, 35 insertions(+), 19 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index dd5e5ce2e6..c5fc905e45 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -17,20 +17,31 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER From Joseph Brill: - MSVC: A significant delay was experienced in the windows 2022 - runner and the windows 2025 runner when the MSVC batch files are - called during tool initialization due to the vcpkg component - installed with Visual Studio 2022. The vcpkg initialization delay - was significantly reduced in the windows 2022 and 2025 runner - environments by adding the default windows powershell 7 path - before the default powershell 5 path in the minimal environment in - which the MSVC batch files are run. Powershell 7 is known to be - installed in the windows runners. The Visual Studio vcpkg component - is not installed in the windows 2019 runner. The shell environment - variables and values for VCPKG_DISABLE_METRICS and VCPKG_ROOT are - added to the minimal environment in which the the MSVC batch files - are run. At present, the VCPKG_DISABLE_METRICS and VCPKG_ROOT - variables and values are not propagated to the SCons environment - after running the MSVC batch files. + and 2025 runners due to the limited environment in which the MSVC + batch files are invoked. The vcpkg component installed in Visual + Studio invokes a powershell script when the MSVC batch files are + called. The limited MSVC environment 1) did not have the preferred + pwsh executable on the system path, 2) did not include the powershell + module analysis cache location (PSModuleAnalysisCachePath), and 3) did + not include the powershell module path (PSModulePath) which appears to + be responsible for the performance degradation in the windows 2022 and + 2025 runners. The VS vcpkg component is not installed in the windows + 2019 runner. Adding the pwsh and powershell executable paths in the + order discovered on the shell environment path, passing the powershell + module analysis cache location, and adding a limited subset of the + powershell module path appears to have eliminated the significant + delays. In the windows runners, any one of the these three changes + appears to eliminite the significant delays. It is hoped that the + combination of all three changes will guard against significant + delays in other environment configurations as well. + - MSVC: The following shell variables and values are passed to the + environment in which the MSVC batch files are invoked: + VCPKG_DISABLE_METRICS, VCPKG_ROOT, POWERSHELL_TELEMETRY_OPTOUT, + PSDisableModuleAnalysisCacheCleanup, and PSModuleAnalysisCachePath. + A limited subset of the shell PSModulePath environment variable value + is passed to the MSVC environment. None of these variables and values + are propagated to the SCons environment after running the MSVC batch + files. From Bill Prendergast: - Fixed SCons.Variables.PackageVariable to correctly test the default diff --git a/RELEASE.txt b/RELEASE.txt index 34008fdc9a..72ed3bc07b 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -35,9 +35,13 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY one from PEP 308 introduced in Python 2.5 (2006). The idiom being replaced (using and/or) is regarded as error prone. -- MSVC: The default Windows powershell 7 path is added before the default - Windows powershell 5 path in the limited environment in which the - MSVC batch files are run. +- MSVC: The following shell environment variables are now included in + the environment in which the MSVC batch files are invoked when defined: + VCPKG_DISABLE_METRICS, VCPKG_ROOT, POWERSHELL_TELEMETRY_OPTOUT, + PSDisableModuleAnalysisCacheCleanup, and PSModuleAnalysisCachePath. + A subset of the shell environment PSModulePath path elements is + included in the limited environment in which the MSVC batch files + are invoked. FIXES ----- @@ -45,8 +49,9 @@ FIXES - Fixed SCons.Variables.PackageVariable to correctly test the default setting against both enable & disable strings. (Fixes #4702) -- MSVC: Fix significant slowdown initializing MSVC tools when vcpkg has - been installed on the system. +- MSVC: Fixed a significant delay that was experienced in the windows + 2022 and 2025 runners due to the limited environment in which the MSVC + batch files are invoked. The windows 2019 runner was not affected. IMPROVEMENTS ------------ From 02133b126a833947128e409d28cc2be2dce652dc Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Sat, 31 May 2025 08:31:38 -0400 Subject: [PATCH 7/8] Fix MSVC environment system path construction issues. Changes: * Prior implementation tested for path membership in a string. This would not add a path that was a proper prefix of an existing path member. Use a list of path elements to test for path membership. * Prior implementation tested for case sensitive path membership. Windows paths are not case sensitive. Use normalized case of absolute path in tests for membership. --- SCons/Tool/MSCommon/common.py | 62 ++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 20 deletions(-) diff --git a/SCons/Tool/MSCommon/common.py b/SCons/Tool/MSCommon/common.py index 7fcd43483b..27a8460393 100644 --- a/SCons/Tool/MSCommon/common.py +++ b/SCons/Tool/MSCommon/common.py @@ -343,7 +343,7 @@ def _force_vscmd_skip_sendtelemetry(env): return True -class _Powershell: +class _PathManager: _PSEXECUTABLES = ( "pwsh.exe", @@ -382,12 +382,12 @@ def _is_psmodulepath(cls, p): cls._cache_is_psmodulepath[norm_path] = is_psmodulepath return is_psmodulepath - _cache_psmodulepath = {} + _cache_psmodulepath_paths = {} @classmethod - def get_psmodulepath(cls, pathspec): - psmodulepath = cls._cache_psmodulepath.get(pathspec) - if psmodulepath is None: + def get_psmodulepath_paths(cls, pathspec): + psmodulepath_paths = cls._cache_psmodulepath_paths.get(pathspec) + if psmodulepath_paths is None: psmodulepath_paths = [] for p in pathspec.split(os.pathsep): p = p.strip() @@ -396,9 +396,9 @@ def get_psmodulepath(cls, pathspec): if not cls._is_psmodulepath(p): continue psmodulepath_paths.append(p) - psmodulepath = os.pathsep.join(psmodulepath_paths) - cls._cache_psmodulepath[pathspec] = psmodulepath - return psmodulepath + psmodulepath_paths = tuple(psmodulepath_paths) + cls._cache_psmodulepath_paths[pathspec] = psmodulepath_paths + return psmodulepath_paths _cache_psexe_paths = {} @@ -422,9 +422,32 @@ def get_psexe_paths(cls, pathspec): if psexe_set: continue break - cls._cache_psexe_paths[pathspec] = tuple(psexe_paths) + psexe_paths = tuple(psexe_paths) + cls._cache_psexe_paths[pathspec] = psexe_paths return psexe_paths + _cache_minimal_pathspec = {} + + @classmethod + def get_minimal_pathspec(cls, pathlist): + pathlist_t = tuple(pathlist) + minimal_pathspec = cls._cache_minimal_pathspec.get(pathlist_t) + if minimal_pathspec is None: + minimal_paths = [] + seen = set() + for p in pathlist: + p = p.strip() + if not p: + continue + norm_path = cls._get_norm_path(p) + if norm_path in seen: + continue + seen.add(norm_path) + minimal_paths.append(p) + minimal_pathspec = os.pathsep.join(minimal_paths) + cls._cache_minimal_pathspec[pathlist_t] = minimal_pathspec + return minimal_pathspec + def normalize_env(env, keys, force: bool=False): """Given a dictionary representing a shell environment, add the variables @@ -450,21 +473,21 @@ def normalize_env(env, keys, force: bool=False): else: debug("keys: skipped[%s]", k) + syspath_pathlist = normenv.get("PATH", "").split(os.pathsep) + # add some things to PATH to prevent problems: # Shouldn't be necessary to add system32, since the default environment # should include it, but keep this here to be safe (needed for reg.exe) sys32_dir = os.path.join( os.environ.get("SystemRoot", os.environ.get("windir", r"C:\Windows")), "System32" ) - if sys32_dir not in normenv["PATH"]: - normenv["PATH"] = normenv["PATH"] + os.pathsep + sys32_dir + syspath_pathlist.append(sys32_dir) # Without Wbem in PATH, vcvarsall.bat has a "'wmic' is not recognized" # error starting with Visual Studio 2017, although the script still # seems to work anyway. sys32_wbem_dir = os.path.join(sys32_dir, 'Wbem') - if sys32_wbem_dir not in normenv['PATH']: - normenv['PATH'] = normenv['PATH'] + os.pathsep + sys32_wbem_dir + syspath_pathlist.append(sys32_wbem_dir) # Without Powershell in PATH, an internal call to a telemetry # function (starting with a VS2019 update) can fail @@ -475,19 +498,18 @@ def normalize_env(env, keys, force: bool=False): # The VS vcpkg component prefers pwsh.exe if it's on the path. sys32_ps_dir = os.path.join(sys32_dir, 'WindowsPowerShell', 'v1.0') psexe_searchlist = os.pathsep.join([os.environ.get("PATH", ""), sys32_ps_dir]) - psexe_pathlist = _Powershell.get_psexe_paths(psexe_searchlist) + psexe_pathlist = _PathManager.get_psexe_paths(psexe_searchlist) # Add powershell executable paths in the order discovered. - for psexe_path in psexe_pathlist: - if psexe_path not in normenv['PATH']: - normenv['PATH'] = normenv['PATH'] + os.pathsep + psexe_path + syspath_pathlist.extend(psexe_pathlist) + normenv['PATH'] = _PathManager.get_minimal_pathspec(syspath_pathlist) debug("PATH: %s", normenv['PATH']) # Add psmodulepath paths in the order discovered. - psmodulepath = _Powershell.get_psmodulepath(os.environ.get("PSModulePath", "")) - if psmodulepath: - normenv["PSModulePath"] = psmodulepath + psmodulepath_pathlist = _PathManager.get_psmodulepath_paths(os.environ.get("PSModulePath", "")) + if psmodulepath_pathlist: + normenv["PSModulePath"] = _PathManager.get_minimal_pathspec(psmodulepath_pathlist) debug("PSModulePath: %s", normenv.get('PSModulePath','')) return normenv From 3290ac7bc9793e0fcdedaa4173010b975a3312ab Mon Sep 17 00:00:00 2001 From: Joseph Brill <48932340+jcbrill@users.noreply.github.com> Date: Sun, 1 Jun 2025 18:45:05 -0400 Subject: [PATCH 8/8] Rework CHANGES.txt and RELEASE.txt per review comments. --- CHANGES.txt | 55 ++++++++++++++++++++++++++++------------------------- RELEASE.txt | 16 +++++++++------- 2 files changed, 38 insertions(+), 33 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index b0c5ea8c6d..e147b40325 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,32 +16,35 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER - Whatever John Doe did. From Joseph Brill: - - MSVC: A significant delay was experienced in the windows 2022 - and 2025 runners due to the limited environment in which the MSVC - batch files are invoked. The vcpkg component installed in Visual - Studio invokes a powershell script when the MSVC batch files are - called. The limited MSVC environment 1) did not have the preferred - pwsh executable on the system path, 2) did not include the powershell - module analysis cache location (PSModuleAnalysisCachePath), and 3) did - not include the powershell module path (PSModulePath) which appears to - be responsible for the performance degradation in the windows 2022 and - 2025 runners. The VS vcpkg component is not installed in the windows - 2019 runner. Adding the pwsh and powershell executable paths in the - order discovered on the shell environment path, passing the powershell - module analysis cache location, and adding a limited subset of the - powershell module path appears to have eliminated the significant - delays. In the windows runners, any one of the these three changes - appears to eliminite the significant delays. It is hoped that the - combination of all three changes will guard against significant - delays in other environment configurations as well. - - MSVC: The following shell variables and values are passed to the - environment in which the MSVC batch files are invoked: + - MSVC: A significant delay was experienced in the Github Actions windows + 2022 and 2025 runners due to the environment used by SCons to initialize + MSVC when the Visual Studio vcpkg component is installed. The Visual + Studio vcpkg component is not installed in the Github Actions windows + 2019 runner. + The Visual Studio vcpkg component invokes a powershell script when the + MSVC batch files are called. The significant delay in the Github + Actions windows 2022 and 2025 runners appears due to the environment + used by SCons to initialize MSVC not including the pwsh executable on + the system path, not including the powershell module analysis cache + location, and not including the powershell module path. + Adding the pwsh and powershell executable paths in the order discovered + on the shell environment path, passing the powershell module analysis + cache location, and adding a subset of the powershell module path to the + environment used by SCons to initialize MSVC appears to have eliminated + the significant delays in the Github Actions windows 2022 and 2025 + runners. + In the Github Actions windows 2022 and 2025 runners, any one of the + three additions appears to eliminate the significant delays. It is hoped + that the combination of all three additions will guard against + significant delays in other environment configurations as well. + - MSVC: The following shell environment variables are now included in the + environment used by SCons to initialize MSVC when defined: VCPKG_DISABLE_METRICS, VCPKG_ROOT, POWERSHELL_TELEMETRY_OPTOUT, - PSDisableModuleAnalysisCacheCleanup, and PSModuleAnalysisCachePath. - A limited subset of the shell PSModulePath environment variable value - is passed to the MSVC environment. None of these variables and values - are propagated to the SCons environment after running the MSVC batch - files. + PSDisableModuleAnalysisCacheCleanup, and PSModuleAnalysisCachePath. A + subset of the shell environment PSModulePath is included in the + environment used by SCons to initialize MSVC when defined. None of + these variables and values are propagated to the user's SCons + environment after running the MSVC batch files. From Edward Peek: - Fix the variant dir component being missing from generated source file @@ -97,7 +100,7 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER - Ninja tool generate_command() fixed to call subst() with correct arguments in ListAction case. Unit tests added for generate_command. Fixes #4580. - + From Edward Peek: - Fix the variant dir component being missing from generated source file paths with CompilationDatabase() builder (Fixes #4003). diff --git a/RELEASE.txt b/RELEASE.txt index 908ae303ca..b8fb3acd71 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -36,12 +36,13 @@ CHANGED/ENHANCED EXISTING FUNCTIONALITY replaced (using and/or) is regarded as error prone. - MSVC: The following shell environment variables are now included in - the environment in which the MSVC batch files are invoked when defined: + the environment used by SCons to initialize MSVC when defined: VCPKG_DISABLE_METRICS, VCPKG_ROOT, POWERSHELL_TELEMETRY_OPTOUT, PSDisableModuleAnalysisCacheCleanup, and PSModuleAnalysisCachePath. - A subset of the shell environment PSModulePath path elements is - included in the limited environment in which the MSVC batch files - are invoked. + A subset of the shell environment PSModulePath is included in the + environment used by SCons to initialize MSVC when defined. None of + these variables and values are propagated to the user's SCons + environment after running the MSVC batch files. FIXES ----- @@ -49,9 +50,10 @@ FIXES - Fixed SCons.Variables.PackageVariable to correctly test the default setting against both enable & disable strings. (Fixes #4702) -- MSVC: Fixed a significant delay that was experienced in the windows - 2022 and 2025 runners due to the limited environment in which the MSVC - batch files are invoked. The windows 2019 runner was not affected. +- MSVC: Fixed a significant delay experienced in the Github Actions + windows 2022 and 2025 runners due to the environment used by SCons + to initialize MSVC when the Visual Studio vcpkg component is + installed. The Github Actions windows 2019 runner was not affected. - Fix the variant dir component being missing from generated source file paths with CompilationDatabase() builder (Fixes #4003).