Skip to content

Fix broken external links flagged by build-and-test-links CI - #486

Open
yarikoptic wants to merge 2 commits into
mainfrom
bf-link-check-fixes
Open

Fix broken external links flagged by build-and-test-links CI#486
yarikoptic wants to merge 2 commits into
mainfrom
bf-link-check-fixes

Conversation

@yarikoptic

Copy link
Copy Markdown
Member

Work primarily by claude to fix up the failing CI. Invocation was

yolo "build-and-test-links CI failing for awhile. Review failing run eg https://github.com/ReproNim/repronim.org/actions/runs/30669522293/job/91284018685 and mitigate -- may be some links needs to be updated or pointed to archive.org snapshots from the past ... prepare a fix with a report on why/how for each failing link. Ensure that local test run of link checks passes"

and then just instructed to use redirected page for Ted.

if CI passes, I will just merge since diff looks sane to me and result is definetely better than current state. if you see something "not right" -- follow up with a fix in a PR.

claude's summary:

The scheduled link check has been failing for a while with 10 broken external links across content/about/collaborators.md and content/about/webinars.md. Each link was investigated individually and either pointed at the current URL, replaced with a web.archive.org snapshot when no live successor exists, or added to .linkcheckerrc when the site is live in a real browser but rejects scripted user agents / is unreachable from GitHub runners.

Per-link resolution:

  • psychoinformatics.de/lab-members.html (Michael Hanke, 404) -> psychoinformatics.de/persons/ (site restructured).
  • psychoinformatics.de/research.html (404) -> psychoinformatics.de/projects/ (same restructure).
  • med.upenn.edu/bbl/faculty-tsatterthwaithe.html (Satterthwaite, 404) -> pennlinc.io/team/ted-satterthwaite (he moved from BBL to founding director of PennLINC).
  • med.unc.edu/childrensresearch/directory/michael-oshea-md-mph/ (404) -> web.archive.org snapshot 20250323133550 (page removed from UNC; no successor faculty page found). Both occurrences updated (collaborators.md and webinars.md).
  • linknovate.com/grant/eager-community-...-317869/ (403) -> web.archive.org snapshot 20250805170804 (Linknovate now blocks scripted user agents).
  • psychiatry.uams.edu/research/birc/?_ga=... (404) -> psychiatry.uams.edu/research/brain-imaging-research-center/ (slug renamed; stale GA tracking param dropped).
  • lemon.martinos.org/pi/ (Eugenio Iglesias, ConnectTimeout) -> martinos.org/investigator/juan-eugenio-iglesias/ (the lemon subdomain is gone; canonical Martinos investigator page).
  • contecenter.uci.edu/overall-center/ (403 to bots, live in browser) -> added to .linkcheckerrc ignore list.
  • matter.childmind.org/ (Arno Klein - 403 to bots, live in browser) -> added to .linkcheckerrc ignore list.
  • connects.mgh.harvard.edu/ (LINC - read timeout from GH runners) -> added to .linkcheckerrc ignore list.

Verified locally by reproducing the CI workflow: hugo build ->
strip <meta name="robots"> -> serve via python3 -m http.server 1313 -> linkchecker --check-extern -f .linkcheckerrc http://localhost:1313 --no-status --no-warnings. Result: 0 errors found (down from 10), 859 links across 1024 URLs.

The scheduled link check has been failing for a while with 10 broken
external links across `content/about/collaborators.md` and
`content/about/webinars.md`. Each link was investigated individually
and either pointed at the current URL, replaced with a
`web.archive.org` snapshot when no live successor exists, or added to
`.linkcheckerrc` when the site is live in a real browser but rejects
scripted user agents / is unreachable from GitHub runners.

Per-link resolution:

- `psychoinformatics.de/lab-members.html` (Michael Hanke, 404)
  -> `psychoinformatics.de/persons/` (site restructured).
- `psychoinformatics.de/research.html` (404)
  -> `psychoinformatics.de/projects/` (same restructure).
- `med.upenn.edu/bbl/faculty-tsatterthwaithe.html` (Satterthwaite, 404)
  -> `pennlinc.io/team/ted-satterthwaite` (he moved from BBL to
  founding director of PennLINC).
- `med.unc.edu/childrensresearch/directory/michael-oshea-md-mph/` (404)
  -> `web.archive.org` snapshot `20250323133550` (page removed from
  UNC; no successor faculty page found). Both occurrences updated
  (collaborators.md and webinars.md).
- `linknovate.com/grant/eager-community-...-317869/` (403)
  -> `web.archive.org` snapshot `20250805170804` (Linknovate now
  blocks scripted user agents).
- `psychiatry.uams.edu/research/birc/?_ga=...` (404)
  -> `psychiatry.uams.edu/research/brain-imaging-research-center/`
  (slug renamed; stale GA tracking param dropped).
- `lemon.martinos.org/pi/` (Eugenio Iglesias, ConnectTimeout)
  -> `martinos.org/investigator/juan-eugenio-iglesias/` (the `lemon`
  subdomain is gone; canonical Martinos investigator page).
- `contecenter.uci.edu/overall-center/` (403 to bots, live in browser)
  -> added to `.linkcheckerrc` ignore list.
- `matter.childmind.org/` (Arno Klein - 403 to bots, live in browser)
  -> added to `.linkcheckerrc` ignore list.
- `connects.mgh.harvard.edu/` (LINC - read timeout from GH runners)
  -> added to `.linkcheckerrc` ignore list.

Verified locally by reproducing the CI workflow: `hugo` build ->
strip `<meta name="robots">` -> serve via `python3 -m http.server
1313` -> `linkchecker --check-extern -f .linkcheckerrc
http://localhost:1313 --no-status --no-warnings`. Result:
`0 errors found` (down from 10), 859 links across 1024 URLs.

Co-Authored-By: Claude Code 2.1.220 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@netlify

netlify Bot commented Aug 1, 2026

Copy link
Copy Markdown

Deploy Preview for repronim ready!

Name Link
🔨 Latest commit c926c48
🔍 Latest deploy log https://app.netlify.com/projects/repronim/deploys/6a6f65d3928d840008b6fdd7
😎 Deploy Preview https://deploy-preview-486--repronim.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates ReproNim site content and link-check configuration to resolve external-link failures reported by the scheduled “build-and-test-links” CI, primarily by replacing moved/removed URLs with current locations or stable archive snapshots and by excluding a small set of sites that block automated link checkers.

Changes:

  • Updated broken external URLs in webinar and collaborator pages (including replacements with web.archive.org snapshots where appropriate).
  • Added .linkcheckerrc ignore patterns for a few domains that are reachable in a normal browser but unreliable/blocked from CI runners.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
content/about/webinars.md Replaces outdated external URLs (UAMS BIRC, Martinos investigator, UNC faculty page via archive) to restore link-check pass.
content/about/collaborators.md Updates/replaces multiple collaborator/service-project external links (including archive snapshots) that were 404/403.
.linkcheckerrc Adds targeted ignore entries for external sites that block scripted user agents or time out from GitHub Actions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@asmacdo

asmacdo commented Aug 4, 2026

Copy link
Copy Markdown
Member

link checker is still failing, ill look into this

@yarikoptic

Copy link
Copy Markdown
Member Author

I think it is a futile struggle... not sure how bids-specification is more robust. We might want at least to separate local checks from external

@asmacdo

asmacdo commented Aug 5, 2026

Copy link
Copy Markdown
Member

It is a futile struggle. We can separate internal/external links but since we link to all our presenters websites/bios (the biggest source of problems) I think we should continue to check them. IMO its too much drift to warrant keeping it correct, but if we batch the fixes its not so bad.

@juliebates

Copy link
Copy Markdown
Collaborator

@yarikoptic @asmacdo Just based on intuition, and the sources of some broken links (several at least look like they are CP/SPs, maybe some other collaborators or fellows), I agree, we're always going to have some broken links coming around. People change institutions, or they build their own websites, and then change the platforms, or 'other' - so there just is variability that doesn't adhere to uniform rules I think, so just doing these checks in batches makes practical sense to me too

I haven't investigated this set of links yet, a couple I recognize as having been problematic before and we found patches for them for awhile...

@yarikoptic

Copy link
Copy Markdown
Member Author

we could trim to set of most stable... "red CI" run is pretty much useless since it gets forgotten.

@asmacdo

asmacdo commented Aug 6, 2026

Copy link
Copy Markdown
Member

proposal: PR checks internal links only. external links still get checked on a cron, 1/month at the start of the month, and if it fails auto-open issue so we triage it during our start-of-themonth webmeeting.

@asmacdo

asmacdo commented Aug 18, 2026

Copy link
Copy Markdown
Member

@juliebates and I discussed: she will find alt links for the ones that need to change, and list the links that aren't actually broken. When thats done, I'll have an agent add to linkchecker skip, and sub in the new links. I'll do that on this PR, so hopefully we can get green CI!

Most recent link checker run:

Start checking at 2026-08-18 14:15:00+000

URL https://nipoppy.readthedocs.io/en/stable/how_to_guides/user_guide/index.html' Name https://nipoppy.readthedocs.io/en/stable/how_to_guides/user_guide/index.html'
Parent URL http://localhost:1313/resources/tools/nipoppy/, line 1718, col 25
Real URL https://nipoppy.readthedocs.io/en/stable/how_to_guides/user_guide/index.html
Check time 0.207 seconds
Result Error: 404 Not Found

URL https://neurostars.org/tag/reprostim' Name https://neurostars.org/tag/reprostim'
Parent URL http://localhost:1313/resources/tools/reprostim/, line 1721, col 5
Real URL https://neurostars.org/tag/reprostim
Check time 0.706 seconds
Result Error: 404 Not Found

URL https://zenodo.org/' Name Zenodo'
Parent URL http://localhost:1313/about/principles/publish_everything/, line 1427, col 174
Real URL https://zenodo.org/
Check time 30.690 seconds
Size 92B
Result Error: 504 Gateway Time-out

URL https://www.psychoinformatics.de/lab-members.html' Name Michael Hanke'
Parent URL http://localhost:1313/about/collaborators/, line 1582, col 5
Real URL https://www.psychoinformatics.de/lab-members.html
Check time 1.031 seconds
Size 0B
Result Error: 404 Not Found

URL https://www.med.upenn.edu/bbl/faculty-tsatterthwaithe.html' Name Theodore Satterthwaite'
Parent URL http://localhost:1313/about/collaborators/, line 1588, col 5
Real URL https://www.med.upenn.edu/bbl/faculty-tsatterthwaithe.html
Check time 0.981 seconds
Result Error: 404 Not Found

URL https://www.med.unc.edu/childrensresearch/directory/michael-oshea-md-mph/' Name Michael O’Shea'
Parent URL http://localhost:1313/about/collaborators/, line 1618, col 5
Real URL https://www.med.unc.edu/childrensresearch/directory/michael-oshea-md-mph/
Check time 0.577 seconds
Size 25.06KB
Result Error: 404 Not Found

URL https://www.linknovate.com/grant/eager-community-building-and-workflows-for-data-sharing-with-publicly-accessible-and-consumable-metadata-317869/' Name EAGER: Community Building and Workflows for Data Sharing with Publicly Accessible and Consumable Metadata'
Parent URL http://localhost:1313/about/collaborators/, line 1634, col 10
Real URL https://www.linknovate.com/grant/eager-community-building-and-workflows-for-data-sharing-with-publicly-accessible-and-consumable-metadata-317869/
Check time 0.422 seconds
Result Error: 403 Forbidden

URL https://contecenter.uci.edu/overall-center/' Name Fragmented Early-life Experiences, Aberrant Circuit Maturation, Emotional Vulnerabilities'
Parent URL http://localhost:1313/about/collaborators/, line 1640, col 10
Real URL https://contecenter.uci.edu/overall-center/
Check time 0.368 seconds
Result Error: 403 Forbidden

URL https://www.psychoinformatics.de/research.html' Name Tracing the template: Investigating the representation of perceptual relevance'
Parent URL http://localhost:1313/about/collaborators/, line 1700, col 10
Real URL https://www.psychoinformatics.de/research.html
Check time 0.681 seconds
Size 0B
Result Error: 404 Not Found

URL https://matter.childmind.org/' Name Arno Klein'
Parent URL http://localhost:1313/about/webinars/, line 1859, col 124
Real URL https://matter.childmind.org/
Check time 0.143 seconds
Result Error: 403 Forbidden

URL https://chan.usc.edu/people/faculty/Sook-Lei_Liew' Name Sook-Lei Liew'
Parent URL http://localhost:1313/about/webinars/, line 1867, col 154
Real URL https://chan.usc.edu/people/faculty/Sook-Lei_Liew
Check time 116.952 seconds
Result Error: ReadTimeout: HTTPSConnectionPool(host='chan.usc.edu', port=443): Read timed out. (read timeout=60)

URL https://psychiatry.uams.edu/research/birc/?_ga=1.22816571.1823374017.1366814353' Name Brain Imaging Research Center at the University of Arkansas for Medical Sciences'
Parent URL http://localhost:1313/about/webinars/, line 1881, col 405
Real URL https://psychiatry.uams.edu/research/birc/?_ga=1.22816571.1823374017.1366814353
Check time 0.915 seconds
Result Error: 404 Not Found

URL https://lemon.martinos.org/pi/' Name Eugenio Iglesias'
Parent URL http://localhost:1313/about/webinars/, line 1895, col 148
Real URL https://lemon.martinos.org/pi/
Check time 60.414 seconds
Result Error: ConnectTimeout: HTTPSConnectionPool(host='lemon.martinos.org', port=443): Max retries exceeded with url: /robots.txt (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fb7e60cc050>, 'Connection to lemon.martinos...

Statistics:
Downloaded: 10.5MB.
Content types: 88 image, 597 text, 0 video, 0 audio, 49 application, 1 mail and 140 other.
URL lengths: min=14, max=402, avg=56.

That's it. 875 links in 1043 URLs checked. 0 warnings found (178 ignored or duplicates not printed). 13 errors found.
Stopped checking at 2026-08-18 14:24:48+000 (9 minutes, 47 seconds)
Error: Process completed with exit code 1.

@asmacdo

asmacdo commented Aug 18, 2026

Copy link
Copy Markdown
Member

I dont think we should use wayback machine to keep links running, especially for bios. Prefer finding an updated link.

@yarikoptic

Copy link
Copy Markdown
Member Author

FWIW: can be found/fixed in follow up. Might no longer (or intermittently) not exist overall (people move places etc).

@juliebates

Copy link
Copy Markdown
Collaborator

re: wayback machine backup, I think I have found currently active new links for all of the broken ones that were redirected that way.
People do move around and/or update their webpages ;-)
Just need to double check my list for all the replacement links, I have most if not all gathered

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants