Skip to content

Commit aa32d5e

Browse files
elrrrrrrrclaude
elrrrrrrr
and
claude
committed
fix(EdgedriverBinary): generate per-platform URLs, drop dead listing API (#1026)
`EdgedriverBinary.fetch('/<version>/')` has been broken since around **2026-04-07** — the legacy Azure Blob container listing API now returns **HTTP 409 `PublicAccessNotPermitted`** after Microsoft disabled public access on the storage account: ``` https://msedgewebdriverstorage.blob.core.windows.net/edgewebdriver?prefix=<version>/&delimiter=/&comp=list → HTTP 409 PublicAccessNotPermitted ``` This made `test/common/adapter/binary/EdgedriverBinary.test.ts > fetch() > should work` fail on the `items.length >= 3` assertion, blocking `@eggjs/egg` monorepo's E2E pipeline and any project depending on EdgedriverBinary. Microsoft has migrated Edge WebDriver hosting to `https://msedgedriver.microsoft.com/`. Per-version files are still reachable at the same path pattern: ``` https://msedgedriver.microsoft.com/<version>/edgedriver_<platform>.zip ``` The new host **also** serves a `listing.json` dump (used by Microsoft's own catalog page at https://msedgedriver.microsoft.com/catalog/), but it has two drawbacks: 1. **It's a single static file containing every version since `112.0.1722.39`** — ~9000 entries, ~1.2MB. I verified empirically that **none of `?top` / `?limit` / `?max` / `?pageSize` / `?prefix` / `?latest` query parameters do anything** — the response is byte-identical regardless of query string. There is no server-side filtering. 2. Even with caching, that's 1.2MB extra per sync. So instead this PR mirrors the approach already used by [`FirefoxBinary`](https://github.com/cnpm/cnpmcore/blob/master/app/common/adapter/binary/FirefoxBinary.ts) and [`ChromeForTestingBinary`](https://github.com/cnpm/cnpmcore/blob/master/app/common/adapter/binary/ChromeForTestingBinary.ts): **don't call any listing API for sub-dirs at all.** Instead, enumerate the per-version download URLs from a static list of known platform filenames: ```ts const EDGEDRIVER_PLATFORM_FILES = [ 'edgedriver_arm64.zip', 'edgedriver_linux64.zip', 'edgedriver_mac64.zip', 'edgedriver_mac64_m1.zip', 'edgedriver_win32.zip', 'edgedriver_win64.zip', ] as const; ``` Each emitted `BinaryItem` carries `ignoreDownloadStatuses: [404]`, so any version that doesn't ship every platform (e.g. older builds without `mac64_m1`) is skipped cleanly by cnpmcore's sync pipeline rather than failing the task. The root-dir listing path (`#syncDirItems` → `https://edgeupdates.microsoft.com/api/products`) is **completely unchanged** — that JSON API still works. | | Use `listing.json` | Generate URLs (this PR) | |---|---|---| | Network per sub-dir fetch | ~1.2MB JSON dump | **0 bytes** | | Per-sync caching needed | yes (~1.2MB cached + reset logic) | no | | Failure modes | listing endpoint down → all sub-dir fetches fail | per-platform 404 handled cleanly | | Code surface | listing parse + cache + error path | one `.map()` over a const array | | Singleton state safety | needs careful reset in `initFetch` | none, stateless | | Platform discovery | from server response | static const (loses any net-new platform until updated) | The trade-off is that if Microsoft adds a new Edge WebDriver platform (e.g. `linux_arm64`), this adapter won't pick it up until the const list is updated. That's a maintenance cost but a small one — the platform list has been stable at six entries throughout the entire history of the new CDN. - **Root-dir listing (`fetch('/')`)**: still uses `https://edgeupdates.microsoft.com/api/products`. Microsoft only exposes the very latest release of each channel via this API, so the result today is ~5 versions total (one per channel: Stable / Beta / Dev / Canary). Historical version coverage was — and still is — governed by what cnpmcore had previously synced, not by what edgeupdates returns now. - **Version ordering in the root listing**: items come out in the order Microsoft's API returns them, which is grouped by channel (Stable → Beta → Dev → Canary), newest within each channel. This is *not* a strict latest-first sort, but it matches the pre-PR behavior. If a strict ordering is desired, that's a separate change. Microsoft's per-version download URL works for **versions from `112.0.1722.39` onwards**. Versions older than 112 are no longer downloadable from any public Microsoft endpoint: | Version | Old Blob URL | New URL | |---|---|---| | 80.0.361.111 | 409 PublicAccessNotPermitted | 404 | | 100.0.1185.27 | 409 PublicAccessNotPermitted | 404 | | 110.0.1587.69 | 409 PublicAccessNotPermitted | 404 | | **112.0.1722.39** | 409 PublicAccessNotPermitted | **200 ✓** | | 120.0.2210.133 | 409 PublicAccessNotPermitted | 200 ✓ | | 148.0.3966.0 | 409 PublicAccessNotPermitted | 200 ✓ | This is a Microsoft-side cutoff, not something cnpmcore can work around. **Existing mirrored copies of pre-112 Edge WebDriver binaries in cnpmcore's own storage are unaffected** — this PR only changes *discovery* of new files from upstream. Also adds `await binary.initFetch()` in the test's `beforeEach`. `EdgedriverBinary` is a `@SingletonProto`, so `app.getEggObject(EdgedriverBinary)` returns the same instance across tests, which means `dirItems` populated by one test would leak into the next. This was a pre-existing latent issue masked by the old test only having one `it` block, but it surfaces immediately when adding more cases. There is no formal Microsoft announcement for this specific storage lockdown, but the migration off `azureedge.net` and the gradual lockdown of the legacy XML listing have been documented across multiple community projects. Timeline: - **2024-05** — [MicrosoftEdge/EdgeWebDriver#146](MicrosoftEdge/EdgeWebDriver#146): Azure Blob listing stopped updating for versions ≥ 125.0. - **2025-01** — [MicrosoftEdge/EdgeWebDriver#183](MicrosoftEdge/EdgeWebDriver#183): `azureedge.net` CDN sunset (Edgio bankruptcy); Microsoft initially said it would continue under a different CDN provider. - **2025-07** — [seleniumbase/SeleniumBase#3888](seleniumbase/SeleniumBase#3888), [MicrosoftEdge/EdgeWebDriver#201](MicrosoftEdge/EdgeWebDriver#201), [#203](MicrosoftEdge/EdgeWebDriver#203): `msedgedriver.azureedge.net` DNS stopped resolving; new CDN is `msedgedriver.microsoft.com`. SeleniumBase fixed this in 4.40.6. - **2026-04-07** — (this PR): The Azure Blob container backing the old XML listing also had public access disabled; only `msedgedriver.microsoft.com` works now. - [x] `EdgedriverBinary.test.ts > should list recent stable versions from edgeupdates.microsoft.com` — green - [x] `EdgedriverBinary.test.ts > should generate all known platform driver URLs for a version` — green (asserts the exact 6 generated items with `ignoreDownloadStatuses: [404]` on each) - [x] **All 6 cnpmcore CI matrices** (`mysql node@22/24 × jsonBuilder true/false`, `postgresql node@22/24`) green - [x] `test-deployment`, `typecheck` (lint + fmtcheck + tsc), `codecov/patch`, `codecov/project` green - [ ] Manual smoke (suggested for reviewer): a fresh sync of a recent Edge WebDriver version (e.g. 148.0.3966.0) enumerates all 6 platform binaries and downloads all of them successfully, without 404s - [ ] Manual smoke (suggested for reviewer): a sync of an older version that doesn't ship every platform (e.g. 112.0.1722.39 has no `mac64_m1`) — the missing platform should be skipped via `ignoreDownloadStatuses: [404]` and the rest should complete 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> * **Bug Fixes** * Improved handling of unavailable EdgeDriver versions for specific platforms—missing combinations are now skipped instead of causing failures. * **Improvements** * Enhanced EdgeDriver binary delivery infrastructure for greater reliability and consistency. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent bcdc08a commit aa32d5e

File tree

2 files changed

+109
-154
lines changed

2 files changed

+109
-154
lines changed

‎app/common/adapter/binary/EdgedriverBinary.ts‎

Lines changed: 41 additions & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,36 @@
1-
import path from 'node:path';
21
import { SingletonProto } from '@eggjs/tegg';
32
import {
43
AbstractBinary, FetchResult, BinaryItem, BinaryAdapter,
54
} from './AbstractBinary';
65
import { BinaryType } from '../../enum/Binary';
76

7+
// Microsoft moved Edge WebDriver binaries to https://msedgedriver.microsoft.com/
8+
// in July 2025 after `msedgedriver.azureedge.net` was retired, and around
9+
// 2026-04-07 also disabled public access on the legacy Azure Blob container
10+
// that used to host the XML file listing. There is still no paginated/filtered
11+
// listing API — the only "listing" endpoint on the new host is a ~1.2MB static
12+
// JSON dump (`/listing.json`, ~9000 entries covering every version since
13+
// 112.0.1722.39).
14+
//
15+
// To avoid hammering that 1.2MB dump for every version subdirectory during a
16+
// sync, we mirror the approach used by `FirefoxBinary` / `ChromeForTestingBinary`
17+
// and generate the per-version download URLs from a static list of known
18+
// platform filenames. cnpmcore's sync pipeline honors the per-item
19+
// `ignoreDownloadStatuses` field, so any version that doesn't ship a given
20+
// platform (e.g. older builds without `edgedriver_mac64_m1.zip`) gets a clean
21+
// 404 and is skipped rather than failing the sync.
22+
const EDGEDRIVER_DOWNLOAD_BASE = 'https://msedgedriver.microsoft.com/';
23+
// Platform filenames observed in Microsoft's current `listing.json` dump.
24+
// Every version since 112.0.1722.39 ships some subset of these six files.
25+
const EDGEDRIVER_PLATFORM_FILES = [
26+
'edgedriver_arm64.zip',
27+
'edgedriver_linux64.zip',
28+
'edgedriver_mac64.zip',
29+
'edgedriver_mac64_m1.zip',
30+
'edgedriver_win32.zip',
31+
'edgedriver_win64.zip',
32+
] as const;
33+
834
@SingletonProto()
935
@BinaryAdapter(BinaryType.Edgedriver)
1036
export class EdgedriverBinary extends AbstractBinary {
@@ -32,91 +58,6 @@ export class EdgedriverBinary extends AbstractBinary {
3258
return;
3359
}
3460
this.logger.info('[EdgedriverBinary] remote data length: %s', data.length);
35-
// [
36-
// {
37-
// "Product": "Stable",
38-
// "Releases": [
39-
// {
40-
// "ReleaseId": 73376,
41-
// "Platform": "iOS",
42-
// "Architecture": "arm64",
43-
// "CVEs": [],
44-
// "ProductVersion": "124.0.2478.89",
45-
// "Artifacts": [],
46-
// "PublishedTime": "2024-05-07T02:57:00",
47-
// "ExpectedExpiryDate": "2025-05-07T02:57:00"
48-
// },
49-
// {
50-
// "ReleaseId": 73629,
51-
// "Platform": "Windows",
52-
// "Architecture": "x86",
53-
// "CVEs": [
54-
// "CVE-2024-4559",
55-
// "CVE-2024-4671"
56-
// ],
57-
// "ProductVersion": "124.0.2478.97",
58-
// "Artifacts": [
59-
// {
60-
// "ArtifactName": "msi",
61-
// "Location": "https://msedge.sf.dl.delivery.mp.microsoft.com/filestreamingservice/files/aa1c9fe3-bb9c-4a80-9ff7-5c109701fbfe/MicrosoftEdgeEnterpriseX86.msi",
62-
// "Hash": "4CEF7B907D3E2371E953C41190E32C3560CEE7D3F16D7550CA156DC976EBCB80",
63-
// "HashAlgorithm": "SHA256",
64-
// "SizeInBytes": 162029568
65-
// }
66-
// ],
67-
// "PublishedTime": "2024-05-11T06:47:00",
68-
// "ExpectedExpiryDate": "2025-05-10T16:59:00"
69-
// },
70-
// {
71-
// "ReleaseId": 73630,
72-
// "Platform": "Linux",
73-
// "Architecture": "x64",
74-
// "CVEs": [
75-
// "CVE-2024-4559"
76-
// ],
77-
// "ProductVersion": "124.0.2478.97",
78-
// "Artifacts": [
79-
// {
80-
// "ArtifactName": "rpm",
81-
// "Location": "https://packages.microsoft.com/yumrepos/edge/microsoft-edge-stable-124.0.2478.97-1.x86_64.rpm",
82-
// "Hash": "32D9C333544DDD9C56FED54844E89EF00F3E5620942C07B9B68D214016687895",
83-
// "HashAlgorithm": "SHA256",
84-
// "SizeInBytes": 169877932
85-
// },
86-
// {
87-
// "ArtifactName": "deb",
88-
// "Location": "https://packages.microsoft.com/repos/edge/pool/main/m/microsoft-edge-stable/microsoft-edge-stable_124.0.2478.97-1_amd64.deb",
89-
// "Hash": "85D0AD1D63847B3DD54F0F214D18A2B54462BB43291536E773AD1B8B29BBF799",
90-
// "HashAlgorithm": "SHA256",
91-
// "SizeInBytes": 167546042
92-
// }
93-
// ],
94-
// "PublishedTime": "2024-05-10T17:01:00",
95-
// "ExpectedExpiryDate": "2025-05-10T17:01:00"
96-
// },
97-
// {
98-
// "Product": "EdgeUpdate",
99-
// "Releases": [
100-
// {
101-
// "ReleaseId": 73493,
102-
// "Platform": "Windows",
103-
// "Architecture": "x86",
104-
// "CVEs": [],
105-
// "ProductVersion": "1.3.187.37",
106-
// "Artifacts": [
107-
// {
108-
// "ArtifactName": "exe",
109-
// "Location": "https://msedge.sf.dl.delivery.mp.microsoft.com/filestreamingservice/files/a2fa84fe-796b-4f80-b1cd-f4d1f5731aa8/MicrosoftEdgeUpdateSetup_X86_1.3.187.37.exe",
110-
// "Hash": "503088D22461FEE5D7B6B011609D73FFD5869D3ACE1DBB0F00F8F3B9D122C514",
111-
// "HashAlgorithm": "SHA256",
112-
// "SizeInBytes": 1622072
113-
// }
114-
// ],
115-
// "PublishedTime": "2024-05-08T05:44:00",
116-
// "ExpectedExpiryDate": "2025-05-08T05:44:00"
117-
// }
118-
// ]
119-
// }
12061
const products = data as {
12162
Product: string;
12263
Releases: {
@@ -163,51 +104,20 @@ export class EdgedriverBinary extends AbstractBinary {
163104
return { items: this.dirItems![dir], nextParams: null };
164105
}
165106

166-
// fetch sub dir
167-
// /foo/ => foo/
107+
// fetch sub dir: generate the known platform filenames for this version.
108+
// We intentionally don't call any listing API — see the file-level
109+
// comment for the rationale. Any platform that doesn't exist for a
110+
// specific version is skipped cleanly via `ignoreDownloadStatuses`.
111+
// /126.0.2578.0/ => 126.0.2578.0/
168112
const subDir = dir.substring(1);
169-
// https://msedgewebdriverstorage.blob.core.windows.net/edgewebdriver?prefix=124.0.2478.97/&delimiter=/&maxresults=100&restype=container&comp=list
170-
const url = `https://msedgewebdriverstorage.blob.core.windows.net/edgewebdriver?prefix=${encodeURIComponent(subDir)}&delimiter=/&maxresults=100&restype=container&comp=list`;
171-
const xml = await this.requestXml(url);
172-
return { items: this.#parseItems(xml), nextParams: null };
173-
}
174-
175-
#parseItems(xml: string): BinaryItem[] {
176-
const items: BinaryItem[] = [];
177-
// <Blob><Name>124.0.2478.97/edgedriver_arm64.zip</Name><Url>https://msedgewebdriverstorage.blob.core.windows.net/edgewebdriver/124.0.2478.97/edgedriver_arm64.zip</Url><Properties><Last-Modified>Fri, 10 May 2024 18:35:44 GMT</Last-Modified><Etag>0x8DC712000713C13</Etag><Content-Length>9191362</Content-Length><Content-Type>application/octet-stream</Content-Type><Content-Encoding /><Content-Language /><Content-MD5>1tjPTf5JU6KKB06Qf1JOGw==</Content-MD5><Cache-Control /><BlobType>BlockBlob</BlobType><LeaseStatus>unlocked</LeaseStatus></Properties></Blob>
178-
const fileRe = /<Blob><Name>([^<]+?)<\/Name><Url>([^<]+?)<\/Url><Properties><Last\-Modified>([^<]+?)<\/Last\-Modified><Etag>(?:[^<]+?)<\/Etag><Content\-Length>(\d+)<\/Content\-Length>/g;
179-
const matchItems = xml.matchAll(fileRe);
180-
for (const m of matchItems) {
181-
const fullname = m[1].trim();
182-
// <Blob>
183-
// <Name>124.0.2478.97/edgedriver_arm64.zip</Name>
184-
// <Url>https://msedgewebdriverstorage.blob.core.windows.net/edgewebdriver/124.0.2478.97/edgedriver_arm64.zip</Url>
185-
// <Properties>
186-
// <Last-Modified>Fri, 10 May 2024 18:35:44 GMT</Last-Modified>
187-
// <Etag>0x8DC712000713C13</Etag>
188-
// <Content-Length>9191362</Content-Length>
189-
// <Content-Type>application/octet-stream</Content-Type>
190-
// <Content-Encoding/>
191-
// <Content-Language/>
192-
// <Content-MD5>1tjPTf5JU6KKB06Qf1JOGw==</Content-MD5>
193-
// <Cache-Control/>
194-
// <BlobType>BlockBlob</BlobType>
195-
// <LeaseStatus>unlocked</LeaseStatus>
196-
// </Properties>
197-
// </Blob>
198-
// ignore size = 0 dir
199-
const name = path.basename(fullname);
200-
const url = m[2].trim();
201-
const date = m[3].trim();
202-
const size = parseInt(m[4].trim());
203-
items.push({
204-
name,
205-
isDir: false,
206-
url,
207-
size,
208-
date,
209-
});
210-
}
211-
return items;
113+
const items: BinaryItem[] = EDGEDRIVER_PLATFORM_FILES.map((name) => ({
114+
name,
115+
isDir: false,
116+
url: `${EDGEDRIVER_DOWNLOAD_BASE}${subDir}${name}`,
117+
size: '-',
118+
date: '-',
119+
ignoreDownloadStatuses: [404],
120+
}));
121+
return { items, nextParams: null };
212122
}
213123
}

‎test/common/adapter/binary/EdgedriverBinary.test.ts‎

Lines changed: 68 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,19 @@ describe('test/common/adapter/binary/EdgedriverBinary.test.ts', () => {
77
let binary: EdgedriverBinary;
88
beforeEach(async () => {
99
binary = await app.getEggObject(EdgedriverBinary);
10+
// EdgedriverBinary is a @SingletonProto — reset its per-sync cache so
11+
// each test sees a fresh state (the first `fetch('/')` call populates
12+
// `dirItems` which would otherwise persist across tests).
13+
await binary.initFetch();
1014
});
1115

1216
describe('fetch()', () => {
13-
it('should work', async () => {
17+
it('should list recent stable versions from edgeupdates.microsoft.com', async () => {
1418
app.mockHttpclient('https://edgeupdates.microsoft.com/api/products', 'GET', {
1519
data: await TestUtil.readFixturesFile('edgeupdates.json'),
1620
persist: false,
1721
});
18-
let result = await binary.fetch('/');
22+
const result = await binary.fetch('/');
1923
assert.deepEqual(result, {
2024
items: [
2125
{
@@ -49,28 +53,69 @@ describe('test/common/adapter/binary/EdgedriverBinary.test.ts', () => {
4953
],
5054
nextParams: null,
5155
});
56+
});
5257

53-
const latestVersion = result!.items![result!.items.length - 1].name;
54-
assert(latestVersion);
55-
assert.equal(latestVersion, '126.0.2578.0/');
56-
result = await binary.fetch(`/${latestVersion}`);
57-
const items = result!.items;
58-
assert(items.length >= 3);
59-
for (const item of items) {
60-
// {
61-
// name: 'edgedriver_win64.zip',
62-
// isDir: false,
63-
// url: 'https://msedgewebdriverstorage.blob.core.windows.net/edgewebdriver/126.0.2578.0/edgedriver_win64.zip',
64-
// size: 9564395,
65-
// date: 'Fri, 10 May 2024 17:04:10 GMT'
66-
// }
67-
assert.equal(item.isDir, false);
68-
assert.match(item.name, /^edgedriver_\w+.zip$/);
69-
assert.match(item.url, /^https:\/\//);
70-
assert(typeof item.size === 'number');
71-
assert(item.size > 0);
72-
assert(item.date);
73-
}
58+
it('should generate all known platform driver URLs for a version', async () => {
59+
app.mockHttpclient('https://edgeupdates.microsoft.com/api/products', 'GET', {
60+
data: await TestUtil.readFixturesFile('edgeupdates.json'),
61+
persist: false,
62+
});
63+
const result = await binary.fetch('/126.0.2578.0/');
64+
assert.ok(result);
65+
assert.equal(result.nextParams, null);
66+
// Expect all six known platform filenames, pointing at the new CDN,
67+
// with `ignoreDownloadStatuses: [404]` so older versions that don't
68+
// ship every platform get skipped cleanly instead of failing the sync.
69+
assert.deepEqual(result.items, [
70+
{
71+
name: 'edgedriver_arm64.zip',
72+
isDir: false,
73+
url: 'https://msedgedriver.microsoft.com/126.0.2578.0/edgedriver_arm64.zip',
74+
size: '-',
75+
date: '-',
76+
ignoreDownloadStatuses: [404],
77+
},
78+
{
79+
name: 'edgedriver_linux64.zip',
80+
isDir: false,
81+
url: 'https://msedgedriver.microsoft.com/126.0.2578.0/edgedriver_linux64.zip',
82+
size: '-',
83+
date: '-',
84+
ignoreDownloadStatuses: [404],
85+
},
86+
{
87+
name: 'edgedriver_mac64.zip',
88+
isDir: false,
89+
url: 'https://msedgedriver.microsoft.com/126.0.2578.0/edgedriver_mac64.zip',
90+
size: '-',
91+
date: '-',
92+
ignoreDownloadStatuses: [404],
93+
},
94+
{
95+
name: 'edgedriver_mac64_m1.zip',
96+
isDir: false,
97+
url: 'https://msedgedriver.microsoft.com/126.0.2578.0/edgedriver_mac64_m1.zip',
98+
size: '-',
99+
date: '-',
100+
ignoreDownloadStatuses: [404],
101+
},
102+
{
103+
name: 'edgedriver_win32.zip',
104+
isDir: false,
105+
url: 'https://msedgedriver.microsoft.com/126.0.2578.0/edgedriver_win32.zip',
106+
size: '-',
107+
date: '-',
108+
ignoreDownloadStatuses: [404],
109+
},
110+
{
111+
name: 'edgedriver_win64.zip',
112+
isDir: false,
113+
url: 'https://msedgedriver.microsoft.com/126.0.2578.0/edgedriver_win64.zip',
114+
size: '-',
115+
date: '-',
116+
ignoreDownloadStatuses: [404],
117+
},
118+
]);
74119
});
75120
});
76121
});

0 commit comments

Comments
 (0)