Fix broken external links flagged by build-and-test-links CI - #486
Fix broken external links flagged by build-and-test-links CI#486yarikoptic wants to merge 2 commits into
Conversation
The scheduled link check has been failing for a while with 10 broken external links across `content/about/collaborators.md` and `content/about/webinars.md`. Each link was investigated individually and either pointed at the current URL, replaced with a `web.archive.org` snapshot when no live successor exists, or added to `.linkcheckerrc` when the site is live in a real browser but rejects scripted user agents / is unreachable from GitHub runners. Per-link resolution: - `psychoinformatics.de/lab-members.html` (Michael Hanke, 404) -> `psychoinformatics.de/persons/` (site restructured). - `psychoinformatics.de/research.html` (404) -> `psychoinformatics.de/projects/` (same restructure). - `med.upenn.edu/bbl/faculty-tsatterthwaithe.html` (Satterthwaite, 404) -> `pennlinc.io/team/ted-satterthwaite` (he moved from BBL to founding director of PennLINC). - `med.unc.edu/childrensresearch/directory/michael-oshea-md-mph/` (404) -> `web.archive.org` snapshot `20250323133550` (page removed from UNC; no successor faculty page found). Both occurrences updated (collaborators.md and webinars.md). - `linknovate.com/grant/eager-community-...-317869/` (403) -> `web.archive.org` snapshot `20250805170804` (Linknovate now blocks scripted user agents). - `psychiatry.uams.edu/research/birc/?_ga=...` (404) -> `psychiatry.uams.edu/research/brain-imaging-research-center/` (slug renamed; stale GA tracking param dropped). - `lemon.martinos.org/pi/` (Eugenio Iglesias, ConnectTimeout) -> `martinos.org/investigator/juan-eugenio-iglesias/` (the `lemon` subdomain is gone; canonical Martinos investigator page). - `contecenter.uci.edu/overall-center/` (403 to bots, live in browser) -> added to `.linkcheckerrc` ignore list. - `matter.childmind.org/` (Arno Klein - 403 to bots, live in browser) -> added to `.linkcheckerrc` ignore list. - `connects.mgh.harvard.edu/` (LINC - read timeout from GH runners) -> added to `.linkcheckerrc` ignore list. Verified locally by reproducing the CI workflow: `hugo` build -> strip `<meta name="robots">` -> serve via `python3 -m http.server 1313` -> `linkchecker --check-extern -f .linkcheckerrc http://localhost:1313 --no-status --no-warnings`. Result: `0 errors found` (down from 10), 859 links across 1024 URLs. Co-Authored-By: Claude Code 2.1.220 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
✅ Deploy Preview for repronim ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
Pull request overview
Updates ReproNim site content and link-check configuration to resolve external-link failures reported by the scheduled “build-and-test-links” CI, primarily by replacing moved/removed URLs with current locations or stable archive snapshots and by excluding a small set of sites that block automated link checkers.
Changes:
- Updated broken external URLs in webinar and collaborator pages (including replacements with
web.archive.orgsnapshots where appropriate). - Added
.linkcheckerrcignore patterns for a few domains that are reachable in a normal browser but unreliable/blocked from CI runners.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| content/about/webinars.md | Replaces outdated external URLs (UAMS BIRC, Martinos investigator, UNC faculty page via archive) to restore link-check pass. |
| content/about/collaborators.md | Updates/replaces multiple collaborator/service-project external links (including archive snapshots) that were 404/403. |
| .linkcheckerrc | Adds targeted ignore entries for external sites that block scripted user agents or time out from GitHub Actions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
3f87d3b to
c926c48
Compare
|
link checker is still failing, ill look into this |
|
I think it is a futile struggle... not sure how bids-specification is more robust. We might want at least to separate local checks from external |
|
It is a futile struggle. We can separate internal/external links but since we link to all our presenters websites/bios (the biggest source of problems) I think we should continue to check them. IMO its too much drift to warrant keeping it correct, but if we batch the fixes its not so bad. |
|
@yarikoptic @asmacdo Just based on intuition, and the sources of some broken links (several at least look like they are CP/SPs, maybe some other collaborators or fellows), I agree, we're always going to have some broken links coming around. People change institutions, or they build their own websites, and then change the platforms, or 'other' - so there just is variability that doesn't adhere to uniform rules I think, so just doing these checks in batches makes practical sense to me too I haven't investigated this set of links yet, a couple I recognize as having been problematic before and we found patches for them for awhile... |
|
we could trim to set of most stable... "red CI" run is pretty much useless since it gets forgotten. |
|
proposal: PR checks internal links only. external links still get checked on a cron, 1/month at the start of the month, and if it fails auto-open issue so we triage it during our start-of-themonth webmeeting. |
|
@juliebates and I discussed: she will find alt links for the ones that need to change, and list the links that aren't actually broken. When thats done, I'll have an agent add to linkchecker skip, and sub in the new links. I'll do that on this PR, so hopefully we can get green CI! Most recent link checker run:Start checking at 2026-08-18 14:15:00+000 URL URL URL URL URL URL URL URL URL URL URL URL URL Statistics: That's it. 875 links in 1043 URLs checked. 0 warnings found (178 ignored or duplicates not printed). 13 errors found. |
|
I dont think we should use wayback machine to keep links running, especially for bios. Prefer finding an updated link. |
|
FWIW: can be found/fixed in follow up. Might no longer (or intermittently) not exist overall (people move places etc). |
|
re: wayback machine backup, I think I have found currently active new links for all of the broken ones that were redirected that way. |
Work primarily by claude to fix up the failing CI. Invocation was
and then just instructed to use redirected page for Ted.
if CI passes, I will just merge since diff looks sane to me and result is definetely better than current state. if you see something "not right" -- follow up with a fix in a PR.
claude's summary:
The scheduled link check has been failing for a while with 10 broken external links across
content/about/collaborators.mdandcontent/about/webinars.md. Each link was investigated individually and either pointed at the current URL, replaced with aweb.archive.orgsnapshot when no live successor exists, or added to.linkcheckerrcwhen the site is live in a real browser but rejects scripted user agents / is unreachable from GitHub runners.Per-link resolution:
psychoinformatics.de/lab-members.html(Michael Hanke, 404) ->psychoinformatics.de/persons/(site restructured).psychoinformatics.de/research.html(404) ->psychoinformatics.de/projects/(same restructure).med.upenn.edu/bbl/faculty-tsatterthwaithe.html(Satterthwaite, 404) ->pennlinc.io/team/ted-satterthwaite(he moved from BBL to founding director of PennLINC).med.unc.edu/childrensresearch/directory/michael-oshea-md-mph/(404) ->web.archive.orgsnapshot20250323133550(page removed from UNC; no successor faculty page found). Both occurrences updated (collaborators.md and webinars.md).linknovate.com/grant/eager-community-...-317869/(403) ->web.archive.orgsnapshot20250805170804(Linknovate now blocks scripted user agents).psychiatry.uams.edu/research/birc/?_ga=...(404) ->psychiatry.uams.edu/research/brain-imaging-research-center/(slug renamed; stale GA tracking param dropped).lemon.martinos.org/pi/(Eugenio Iglesias, ConnectTimeout) ->martinos.org/investigator/juan-eugenio-iglesias/(thelemonsubdomain is gone; canonical Martinos investigator page).contecenter.uci.edu/overall-center/(403 to bots, live in browser) -> added to.linkcheckerrcignore list.matter.childmind.org/(Arno Klein - 403 to bots, live in browser) -> added to.linkcheckerrcignore list.connects.mgh.harvard.edu/(LINC - read timeout from GH runners) -> added to.linkcheckerrcignore list.Verified locally by reproducing the CI workflow:
hugobuild ->strip
<meta name="robots">-> serve viapython3 -m http.server 1313->linkchecker --check-extern -f .linkcheckerrc http://localhost:1313 --no-status --no-warnings. Result:0 errors found(down from 10), 859 links across 1024 URLs.