Skip to content

Auto-create imported merchant rule categories that don't exist on the policy#96408

Closed
MelvinBot wants to merge 2 commits into
mainfrom
claude-autoAddImportedMerchantRuleCategory
Closed

Auto-create imported merchant rule categories that don't exist on the policy#96408
MelvinBot wants to merge 2 commits into
mainfrom
claude-autoAddImportedMerchantRuleCategory

Conversation

@MelvinBot

@MelvinBot MelvinBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Explanation of Change

When importing merchant rules from a spreadsheet, an Updated category cell was passed straight through as free text with no check against the workspace's category list, and no category was ever created for it. A rule only applies its category to a matching expense if that category actually exists on the policy, so importing a rule whose category didn't exist left the rule showing a category that never got applied to expenses (deploy blocker from #95105).

This change makes the import auto-create any imported category that doesn't already exist on the workspace, so the imported category is added to the category list instead of being silently dropped — matching the reporter's expected behavior.

  • Added a small createPolicyCategories(policyID, categoryNames) helper in Policy/Category.ts that reuses the existing buildOptimisticPolicyCategories optimistic data and the CREATE_WORKSPACE_CATEGORIES command to create several enabled categories at once.
  • importMerchantRulesSpreadsheet now receives the policy's existing categories, computes the distinct imported categories that aren't already on the policy, and creates them before the rules import.
  • ImportedMerchantRulesPage reads POLICY_CATEGORIES for the policy and passes them to the action.

This is an App-only fix that relies on the already-deployed CreateWorkspaceCategories backend command, so no new backend work is required for the category to exist when a matching expense is later created.

AI tests run by MelvinBot (human co-author must still complete Tests/QA)
  • npm run typecheck-tsgo — passed
  • npm run lint-changed — passed
  • npm run react-compiler-compliance-check check src/pages/workspace/rules/MerchantRules/ImportedMerchantRulesPage.tsx — passed

Not run in this environment: the full npm test suite and npm run storybook --smoke-test (there is no existing unit test for this action). Please run these before marking ready.

Fixed Issues

$ #96389
PROPOSAL: #96389 (comment)

Tests

  1. Create a CSV file with the following content ("Team Lunch" and "Airfare Deluxe" must NOT already exist as categories on the workspace; "Travel" is a default category that already exists):
Merchant,Category
Chipotle,Team Lunch
Delta,Airfare Deluxe
Marriott,Travel

merchant-rules-categories.csv

  1. As a workspace admin on a Control workspace with Rules and Categories enabled, go to Workspaces > [workspace] > Rules.
  2. Click More > Import merchant rules and upload the CSV.
  3. Map the Merchant column to Merchant is and the Category column to Updated category, then click Import.
  4. Verify the confirmation modal says 3 merchant rules have been added.
  5. In the Merchant section of the Rules page, verify the three rule rows display their categories (Team Lunch, Airfare Deluxe, Travel).
  6. Go to Workspaces > [workspace] > Categories and verify Team Lunch and Airfare Deluxe now exist and are enabled.
  7. Verify Travel was not duplicated in the category list.
  • Verify that no errors appear in the JS console

Offline tests

  1. Go offline and run the import from the Tests section.
  2. The category creation is optimistic: the new categories appear in Categories immediately in a pending (greyed) state.
  3. The rules import itself waits for a server response, so the confirmation modal only appears after going back online. Reconnect and verify the modal shows, the rules are created, and the pending state on the new categories clears.

QA Steps

Same as tests

  • Verify that no errors appear in the JS console

PR Author Checklist

  • I linked the correct issue in the ### Fixed Issues section above
  • I wrote clear testing steps that cover the changes made in this PR
    • I added steps for local testing in the Tests section
    • I added steps for the expected offline behavior in the Offline steps section
    • I added steps for Staging and/or Production testing in the QA steps section
    • I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
    • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
    • I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
  • I included screenshots or videos for tests on all platforms
  • I ran the tests on all platforms & verified they passed on:
    • Android: Native
    • Android: mWeb Chrome
    • iOS: Native
    • iOS: mWeb Safari
    • MacOS: Chrome / Safari
  • I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
  • I followed proper code patterns (see Reviewing the code)
    • I verified that comments were added to code that is not self explanatory
    • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
    • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
  • If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
  • I followed the guidelines as stated in the Review Guidelines
  • I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
  • If a new CSS style is added I verified that:
    • A similar style doesn't already exist
    • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))
  • If new assets were added or existing ones were modified, I verified that:
    • The assets are optimized and compressed (for SVG files, run npm run compress-svg)
    • The assets load correctly across all supported platforms.
  • If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
  • If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
  • If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
  • If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
  • If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
    • I verified that all the inputs inside a form are aligned with each other.
    • I added Design label and/or tagged @Expensify/design so the design team can review the changes.
  • I added unit tests for any new feature or bug fix in this PR to help automatically prevent regressions in this user flow.
  • If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

MacOS: Chrome / Safari pr96408-1-import-mapping pr96408-2-rule-rows pr96408-3-categories-created

MelvinBot and others added 2 commits July 17, 2026 15:03
… policy

Co-authored-by: Nicolás Bonet <NicolasBonet@users.noreply.github.com>
Co-authored-by: Nicolás Bonet <NicolasBonet@users.noreply.github.com>
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Looks like you've decreased code coverage for some files. Please write tests to increase, or at least maintain, the existing level of code coverage. See our documentation here for how to interpret this table.

Files with missing lines Coverage Δ
.../rules/MerchantRules/ImportedMerchantRulesPage.tsx 0.00% <0.00%> (ø)
src/libs/actions/Policy/Rules.ts 31.06% <0.00%> (-0.98%) ⬇️
src/libs/actions/Policy/Category.ts 49.71% <0.00%> (-1.00%) ⬇️
... and 8 files with indirect coverage changes

@MelvinBot

Copy link
Copy Markdown
Contributor Author

I checked all the checks on this PR and there's no code fix to make — every substantive check is passing.

The only failing check is PR Author Checklist / checklist, which fails because the checklist boxes in the PR description haven't been ticked (0 out of 38 items completed). That's a human task, not a code issue — NicolasBonet, please fill out the PR Author Checklist and the Tests/QA sections to clear it.

Full check breakdown
  • Passing (32): typecheck, all 8 test jobs, ESLint check, Bun tests, Storybook tests, React Compiler Compliance, Oxfmt check, spellcheck, Check for unused styles, Compare knip issues against main, codecov/patch, codecov/project, all build/dryRun jobs, verifySignedCommits, security/snyk, CLA, validate.
  • Failing (1): checklist — PR Author Checklist not filled out (log). Human-only; nothing to fix in code.
  • Still running (2): baseline-perf-tests, branch-perf-tests — these were in progress at analysis time. This PR only changes the merchant-rule spreadsheet import action, which isn't exercised by the render benchmarks, so a regression from this diff isn't expected.

Since there's no failing check caused by the code, I haven't pushed anything. If the perf tests end up failing, re-ping me and I'll investigate.

@MelvinBot

Copy link
Copy Markdown
Contributor Author

Re-checked now that the perf tests finished. There's still no code fix to make — both remaining failures are unrelated to this PR's code.

1. baseline-perf-tests — infrastructure failure, not code. The job failed on the artifact upload step, not on any test:

##[error]Failed to FinalizeArtifact: Received non-retryable error:
Failed request: (403) Forbidden ... HTTP status code 403 "Forbidden"

This is a GitHub Actions artifact-storage hiccup. On top of that, baseline-perf-tests measures the base branch (main), not this PR's diff, so it can't be caused by these changes. branch-perf-tests (which runs the PR branch) passed. The fix is simply to re-run the failed job — re-running the Reassure Performance Tests workflow should clear it.

2. PR Author Checklist / checklist — human task. Fails only because the checklist boxes in the PR description are unticked (0 of 38). NicolasBonet, please fill out the PR Author Checklist and the Tests/QA sections to clear it.

Every substantive check passes — typecheck, all 8 test jobs, ESLint, Storybook, React Compiler Compliance, Bun tests, branch-perf-tests, builds, etc. — so I haven't pushed anything.

Notes
  • I couldn't verify behavior in a browser in this run (web prep was unavailable), but no code changed and no browser verification is needed to reach this conclusion.
  • If re-running the perf workflow doesn't clear baseline-perf-tests, re-ping me and I'll dig further.

@MelvinBot

Copy link
Copy Markdown
Contributor Author

I looked at both failing checks — neither is caused by this PR's code, so there's nothing to fix in the diff.

1. baseline-perf-tests — transient CI infrastructure failure (just re-run it). The perf tests themselves all passed:

Test Suites: 16 passed, 16 total
Tests:       74 passed, 74 total
✅  Written Baseline performance measurements to .reassure/baseline.perf

The job only failed afterward on the artifact upload step:

##[error]Failed to FinalizeArtifact: Received non-retryable error: Failed request: (403) Forbidden

That's a GitHub artifact-service hiccup, not a test/code issue. On top of that, baseline-perf-tests runs against main — it doesn't even include this PR's changes. The branch-side check, branch-perf-tests (which does run this PR's code), passed. Please re-run the failed job (Re-run failed jobs on this run); it should go green. I wasn't able to trigger the re-run myself from here.

2. PR Author Checklist / checklist — human task. Fails only because the checklist boxes in the PR description are unticked (0 of 38). NicolasBonet, please complete the PR Author Checklist and fill out the Tests/QA sections to clear it.

Everything else (all 8 test jobs, typecheck, ESLint, Storybook, React Compiler, Bun tests, builds, etc.) is passing.

@MelvinBot

Copy link
Copy Markdown
Contributor Author

Still no code fix to push — both failing checks are infra/human, not caused by this PR.

1. baseline-perf-tests — CI infra failure (safe to re-run). The performance tests themselves all passed (Test Suites: 16 passed, Tests: 74 passed, baseline measurements written successfully). The job failed only in the artifact upload step afterward:

##[error]Failed to FinalizeArtifact: Received non-retryable error: Failed request: (403) Forbidden: Error from intermediary with HTTP status code 403 "Forbidden"

This is a GitHub Actions/blob-storage upload flake against the main baseline — it has nothing to do with this diff (baseline tests run on main, not your branch, and your branch-perf-tests passed). Re-run the failed baseline-perf-tests job and it should go green.

2. PR Author Checklist / checklist — human task. Fails because the checklist in the PR description is unticked (0 of 38). NicolasBonet, please fill out the PR Author Checklist and the Tests/QA sections.

Every other check passes (typecheck, all 8 test jobs, ESLint, Storybook, React Compiler, Bun tests, builds, branch-perf-tests, etc.), so there's nothing to fix in code and I haven't pushed anything.

@NicolasBonet
NicolasBonet marked this pull request as ready for review July 17, 2026 16:42
@NicolasBonet
NicolasBonet requested a review from a team as a code owner July 17, 2026 16:42
@melvin-bot
melvin-bot Bot requested review from daledah and removed request for a team July 17, 2026 16:42
@melvin-bot

melvin-bot Bot commented Jul 17, 2026

Copy link
Copy Markdown

@daledah Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

@melvin-bot
melvin-bot Bot requested review from trjExpensify and removed request for a team July 17, 2026 16:42
@truph01

truph01 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@NicolasBonet Could you assign me to review this?

@NicolasBonet

Copy link
Copy Markdown
Contributor

@NicolasBonet Could you assign me to review this?

Is this something you can review within the next hour? Would really like to unblock the deployment soon.

@truph01

truph01 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@NicolasBonet Could you assign me to review this?

Is this something you can review within the next hour? Would really like to unblock the deployment soon.

Yes

@NicolasBonet
NicolasBonet requested a review from truph01 July 17, 2026 16:53

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9a9bda7c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

.filter((category): category is string => !!category && !policyCategories?.[category]),
),
];
createPolicyCategories(policyID, missingCategories);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Serialize category creation before importing rules

When the spreadsheet references a category that is not already on the policy, this enqueues CREATE_WORKSPACE_CATEGORIES and then immediately continues to API.makeRequestWithSideEffects for the merchant-rule import. API.write only puts the create request on the sequential write queue, while side-effect requests run right away, so the import can reach the backend before the category exists server-side; in that case the imported rule can still lose or reject the new category despite the optimistic Onyx update. Please make the import wait for the category write to finish, or have the server-side import create the missing categories atomically.

Useful? React with 👍 / 👎.

@MelvinBot

Copy link
Copy Markdown
Contributor Author

The perf tests have now finished, and there's still no code fix to make — both remaining failures are non-code issues.

1. baseline-perf-tests — CI infrastructure flake, not this PR. The performance suite itself passed completely (Test Suites: 16 passed, 16 total / Tests: 74 passed, 74 total). The job only failed afterward at the artifact-upload step:

##[error]Failed to FinalizeArtifact: Received non-retryable error: Failed request: (403) Forbidden

That's the actions/upload-artifact action hitting a transient 403 from the artifact backend — infrastructure, not code. Also note baseline-perf-tests measures the base branch (main), not this PR's changes. Re-running that job should clear it.

2. checklist (PR Author Checklist) — human task. Still failing because the checklist boxes and Tests/QA sections in the PR description aren't filled out (0 of 38 items). NicolasBonet, please complete those to clear it.

Everything else is green (typecheck, all test jobs, ESLint, Storybook, branch-perf-tests, builds, etc.). Since neither failure is caused by the code, I haven't pushed anything. Re-run baseline-perf-tests and fill out the checklist, and this PR should be all green.

@truph01

truph01 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@NicolasBonet, do you think the current approach violates our 1:1:1 rules?

Also, we're calling createPolicyCategories(policyID, missingCategories) before API.makeRequestWithSideEffects(SIDE_EFFECT_REQUEST_COMMANDS.IMPORT_MERCHANT_RULES_SPREADSHEET, parameters).

What happens if the second request fails? We won't be able to roll back the categories created by the first request, which could leave the data in an inconsistent state.

@truph01

truph01 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@NicolasBonet I think everything should be handled on the BE side via ImportMerchantRulesSpreadsheet API

@NicolasBonet

Copy link
Copy Markdown
Contributor

Closing this PR — two reasons:

  1. The Codex comment is correct: CreateWorkspaceCategories goes on the client's sequential write queue, while the rules import runs immediately as a side-effect request, so the import can reach the backend before the categories exist server-side. A client-side fix can't reliably order these two calls.

  2. While moving the fix to the backend (https://github.com/Expensify/Web-Expensify/pull/54643, now also closed), we realized auto-creating workspace categories from a merchant rules import is new product behavior with no precedent — everywhere else (personal expense rules, apply-time checks) a rule may reference a nonexistent category and it's simply skipped when applying. Auto-creating also has real side effects: a typo in the category column silently adds an enabled category visible to the whole workspace, and ad-hoc categories won't exist in connected accounting integrations.

We'll settle the desired behavior in #96389 first (e.g. skip/warn at import time, surface "these categories will be created" in the import preview, or auto-create if that's what product actually wants) before reintroducing a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants