test: e2e test website accountPayment loggedIn, includes magic.link testMode support in test-e2e job only by gobengo · Pull Request #1754 · web3-storage/web3.storage

gobengo · 2022-08-16T17:05:35Z

Motivation

e2e test the logged in flow for /account/payment ui
we hadn't previously found a way to e2e test auth
playwright.dev's html reporter is awesome and lets you see console, network, screenshots of the whole e2e test! preview
- more about this playwright.dev feature on debbie.codes
- this can really speed up the amount of time understanding e2e test failures!

What

add playwright-report (html) to ci artifacts. this is useful to debug failed e2e test runs (with screenshots, console, network recorded!)
add ability to opt-in to using magic.link testMode (not on by default, on in ci test-e2e)
test-e2e run has api running and db with loaded schema
test logged-in case of accountPayment

github-actions · 2022-08-16T17:10:39Z

Website preview 🔗✨

build log

…e api

github-actions · 2022-08-16T17:53:57Z

`package-lock.json` changes

Click to toggle table visibility

Name	Previous	Current
`@cspotcode/source-map-support`	-	0.8.1
`@jridgewell/trace-mapping`	0.3.4	0.3.9
`@tsconfig/node10`	-	1.0.9
`@tsconfig/node12`	-	1.0.11
`@tsconfig/node14`	-	1.0.3
`@tsconfig/node16`	-	1.0.3
`@typescript-eslint/parser`	-	5.33.1
`@typescript-eslint/scope-manager`	-	5.33.1
`@typescript-eslint/types`	-	5.33.1
`@typescript-eslint/typescript-estree`	-	5.33.1
`@typescript-eslint/visitor-keys`	-	5.33.1
`acorn-walk`	-	8.2.0
`arg`	-	4.1.3
`create-require`	-	1.1.1
`make-error`	-	1.3.6
`ts-node`	-	10.9.1
`tsutils`	-	3.21.0
`v8-compile-cache-lib`	-	3.0.1
`yn`	-	3.1.1

…om .env and uses gh action secrets

…error=true

This reverts commit a504193.

…ar keys, not rewrite with values from env

…tmode

…sed in ci website workflow)

…_MAGIC... env vars

gobengo · 2022-08-23T00:39:24Z

 };

 /**
- * @param {Object} [props]


This was required for next build to build locally for me with tsc, and makes sense to clarify that the argument destructuring wont work if props === undefined

gobengo · 2022-08-23T00:40:52Z

@@ -1,5 +1,6 @@
 const API = /** @type {string} **/ (process.env.NEXT_PUBLIC_API);
 const MAGIC_TOKEN = /** @type {string} **/ (process.env.NEXT_PUBLIC_MAGIC);
+const MAGIC_TESTMODE_ENABLED = Boolean(process.env.NEXT_PUBLIC_MAGIC_TESTMODE_ENABLED);


I think that we're relying on miniflare/wrangler to replace this NEXT_PUBLIC_MAGIC_TESTMODE_ENABLED (and NEXT_PUBLIC_MAGIC) is one reason you have to explicitly tell miniflare about your env vars either through the env file or through the --binding KEY=VALUE flags I added when the website ci workflow invokes npm -w '@web3-storage/api' start. I think maybe miniflare devs would be hesitant to automatically pass everything through from miniflare env without an allowlist because it could lead to a developers secrets getting into the build without explicit configuration that thats what they'd want.

gobengo · 2022-08-23T01:22:48Z

the main new thing I'd point out, which I think made the code clearer, was to reify the fact that that there are two distinct kinds of of 'magic.link bypasses'

one we use only in unit tests, which allows a special allowlisted bearer token to work, but only when env.DANGEROUSLY_BYPASS_MAGIC_AUTH is truthy
one we use only in e2e tests, which enables detection of any incoming Bearer tokens that are JWTs that look like a magic.link test mode token (it looks in jwt sub and aud), and doesnt bother to validate those tokens + ensures those tokens can be resolved to appropriate metadata for the rest of our 'user login' stuff to work, but only when env.NEXT_PUBLIC_MAGIC_TESTMODE_ENABLED (which also gets used on the frontend)
- this bypass, when enabled, is also used in a controller used by the loginOrRegister route that handles magic.link tokens (because now those tokens can be testMode tokens but only when enabled via env var aka only in test-e2e ci)

yusefnapora

I don't see anything that looks dangerous here (at least, nothing that's not lampshaded with "dangerouslyDoTheThing" naming 👍 ), and I like the explicit naming for the different types of bypass.

I'm not super crazy about the env file rewriter script... but I'm guessing that we need it in order to allow miniflare to use the bindings set in CI? That seems like a reasonable justification, IMO.

It would also be rad to add a note to the dev docs / README about the bypass modes - @gobengo maybe you could grab that as a follow up when you're back from dweb camp? I don't think we need to block on it, but it seems worth a mention. I left a couple other non-blocking suggestions if you have time to grab.

yusefnapora · 2022-08-23T22:06:48Z

@@ -1,3 +1,5 @@
+import { magicTestModeIsEnabledFromEnv, maybeJsonParseable } from './utils/env.js'
+
 export const createMagicTestTokenBypass = (


If we can remove the export here and just export the concrete value magicLinkBypassForUnitTestingWithTestToken, that seems a bit cleaner. But if you need to export this for testing, that's okay by me...

Co-authored-by: Yusef Napora <yusef@napora.org>

gobengo · 2022-08-24T00:52:02Z

status: I believe checks are green except for GITHUB_TOKEN rate limits

olizilla · 2022-08-24T08:32:02Z

+  assert.equal(fn({
+    NEXT_PUBLIC_MAGIC_TESTMODE_ENABLED: 'not-json-parseable'
+  }), true)


@gobengo having anything truth that isn't 'false' or '0' mean true seems like a foot gun. Please can we have exactly 1 valid way to enable test mode:

NEXT_PUBLIC_MAGIC_TESTMODE_ENABLED === 'true'

gobengo added 2 commits August 16, 2022 10:06

feat(@web3-storage/website): stub /account/payment page in website

c19d541

good progress on accountPayment.e2e loggedIn

d7be490

gobengo force-pushed the e2e-authn branch from eb7d942 to d7be490 Compare August 16, 2022 17:07

gobengo changed the title ~~e2e test accountPayment loggedIn~~ test(website): e2e test accountPayment loggedIn Aug 16, 2022

website playwright runs 'npm start' in monorepo root so e2e tests hav…

7deeb34

…e api

gobengo force-pushed the e2e-authn branch from b3418cb to 7deeb34 Compare August 16, 2022 17:28

website e2e gh action writes full .env file from .env.tpl

511f013

gobengo added 14 commits August 16, 2022 11:16

website workflow e2e scrubs NEXT_PUBLIC_MAGIC and MAGIC_SECRET_KEY fr…

a0bd33d

…om .env and uses gh action secrets

try fix test

37d9bdf

wip

2ac4962

website test-e2e continue-on-error=true

6593cae

test-e2e fails after artifacts

0c26d9c

gh action playwright timeout 2min

193bc7a

playwright uses PLAYWRIGHT_TIMEOUT if present

75d60bd

wip

8966f6b

wip

6eb2ca5

website workflow generates .env before building website

93ad61d

website build in test-e2e uses .env from monorepo root

030dff2

wip

b742708

wip

6033b10

wip

ff5f586

gobengo force-pushed the e2e-authn branch from 4f89a02 to ff5f586 Compare August 16, 2022 21:40

gobengo added 6 commits August 16, 2022 15:01

wip

05096c3

website workflow fix npm usage when loading schema in db

7560eaf

api/src/env passes testMode to Magic constructor based on env

6ab0504

clean up

6ae8ed2

fix tsc error in website playwright globalSetup

9443450

website workflow preview sticky-pull-request-comment has continue-on-…

d50949d

…error=true

gobengo added 8 commits August 22, 2022 17:07

Revert "ci: website remove 'add magic config to .env'"

cc56d72

This reverts commit a504193.

ci: website adjust 'write .env' step to use sed to remove magic env v…

d72314a

…ar keys, not rewrite with values from env

createMagicLoginController now uses magicLinkBypassForE2ETestingInTes…

6f609b8

…tmode

website envfile script excludes vars in csv ENVFILE_WITHOUT (replace …

7ab57d9

…sed in ci website workflow)

lint

95db3b2

ci: website workflow remove unused lines

244e809

ci: website test-e2e build website step provides required NEXT_PUBLIC…

f4e7cc6

…_MAGIC... env vars

ci: remove website workflow continue-on-error that is not wanted

2fa6205

gobengo mentioned this pull request Aug 23, 2022

ci: build website doesnt use dotenv/config #1782

Merged

gobengo added 3 commits August 22, 2022 17:27

ci: build website doesnt use dotenv/config

6fbd50d

fix jsdoc in website loading component

51ef3c5

ci: website test-e2e no longer builds website before test

21d6f0d

gobengo commented Aug 23, 2022

View reviewed changes

gobengo requested review from olizilla and yusefnapora August 23, 2022 00:54

gobengo changed the title ~~test: add magic.link testMode in ci, use it to e2e test website accountPayment loggedIn~~ test: e2e test website accountPayment loggedIn, includes magic.link testMode support in test-e2e job only Aug 23, 2022

yusefnapora approved these changes Aug 23, 2022

View reviewed changes

gobengo and others added 2 commits August 23, 2022 15:34

add comment to envfile script explaining ENVFILE_WITHOUT

4389ba0

Co-authored-by: Yusef Napora <yusef@napora.org>

dont export createMagicTestTokenBypass

88a488d

gobengo mentioned this pull request Aug 23, 2022

document how auth works in unit tests, e2e tests #1793

Open

gobengo added 2 commits August 23, 2022 15:39

Merge branch 'main' into e2e-authn

ad880b2

fix bad import in packages/api/test/scripts/helpers.js

5750ce4

gobengo merged commit 6d6f92c into main Aug 24, 2022

gobengo deleted the e2e-authn branch August 24, 2022 01:38

This was referenced Aug 24, 2022

chore(main): release website 2.17.0 #1771

Merged

chore(main): release api 7.4.0 #1792

Merged

olizilla reviewed Aug 24, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: e2e test website accountPayment loggedIn, includes magic.link testMode support in test-e2e job only#1754

test: e2e test website accountPayment loggedIn, includes magic.link testMode support in test-e2e job only#1754
gobengo merged 73 commits intomainfrom
e2e-authn

gobengo commented Aug 16, 2022 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 16, 2022 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 16, 2022 •

edited

Loading

Uh oh!

gobengo Aug 23, 2022

Uh oh!

gobengo Aug 23, 2022 •

edited

Loading

Uh oh!

gobengo commented Aug 23, 2022 •

edited

Loading

Uh oh!

yusefnapora left a comment

Uh oh!

yusefnapora Aug 23, 2022

Uh oh!

gobengo Aug 23, 2022

Uh oh!

Uh oh!

gobengo commented Aug 24, 2022

Uh oh!

olizilla Aug 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -1,3 +1,5 @@
		import { magicTestModeIsEnabledFromEnv, maybeJsonParseable } from './utils/env.js'

		export const createMagicTestTokenBypass = (

Conversation

gobengo commented Aug 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Website preview 🔗✨

Uh oh!

github-actions Bot commented Aug 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

package-lock.json changes

Uh oh!

gobengo Aug 23, 2022

Choose a reason for hiding this comment

Uh oh!

gobengo Aug 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gobengo commented Aug 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yusefnapora left a comment

Choose a reason for hiding this comment

Uh oh!

yusefnapora Aug 23, 2022

Choose a reason for hiding this comment

Uh oh!

gobengo Aug 23, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gobengo commented Aug 24, 2022

Uh oh!

olizilla Aug 24, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gobengo commented Aug 16, 2022 •

edited

Loading

github-actions Bot commented Aug 16, 2022 •

edited

Loading

github-actions Bot commented Aug 16, 2022 •

edited

Loading

`package-lock.json` changes

gobengo Aug 23, 2022 •

edited

Loading

gobengo commented Aug 23, 2022 •

edited

Loading