Skip to content

fix(model-resolution): add free-model fallback for free-only provider setups (fixes #1883)#3821

Open
mrosnerr wants to merge 2 commits into
code-yeongyu:devfrom
mrosnerr:fix/free-model-fallback-v3
Open

fix(model-resolution): add free-model fallback for free-only provider setups (fixes #1883)#3821
mrosnerr wants to merge 2 commits into
code-yeongyu:devfrom
mrosnerr:fix/free-model-fallback-v3

Conversation

@mrosnerr
Copy link
Copy Markdown
Contributor

@mrosnerr mrosnerr commented May 6, 2026

Fixes #1883

Summary

Subagents resolve to paid models when only free providers are configured. This adds a free-model fallback that appends free opencode models to the fallback chain when the user only has free models available.

Paid users are unaffected — detection is based on availableModels, not connectedProviders, so Zen subscribers don't get downgraded.

Free model list is sourced from a generated JSON snapshot refreshed by the existing weekly refresh-model-capabilities workflow. Model rotation is a JSON edit, not a code change.

Changes

File Change
free-model-fallback.ts Free-only detection + append strategy
model-selection.ts Wire in free-only logic before fallback chain resolution
model-selection-input.ts Extract existing helpers (pure refactor)
free-opencode-models.generated.json Generated snapshot from models.dev
script/build-free-models.ts Standalone build script for the snapshot
script/free-model-extraction.ts Extraction logic (testable, separate from runner)
refresh-model-capabilities.yml Also runs build:free-models on weekly cron
*test.ts (×3) Unit, integration, and extraction tests

Testing

  • bun run build:free-models — generates snapshot, verified output matches live models.dev/api.json
  • bun run typecheck — clean
  • bun run build — clean
  • bun run test:model-capabilities — 58 pass (includes extraction edge cases + snapshot guardrails)
  • All 41 model-selection tests pass, 0 regressions

Context

Builds on #3529 (closed by @MoerAI, who created the initial module structure).

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 11 files

Confidence score: 3/5

  • Free-model detection in script/free-model-extraction.ts only checks cost.input, which can misclassify models with billed output or other costs as free and lead to incorrect listings
  • Workflow change in .github/workflows/refresh-model-capabilities.yml adds Opencode snapshot generation without the required cloning/reading step, which may break the intended automation flow
  • Given the two medium-severity issues, there is some risk of functional inaccuracies or workflow failures, so the merge is not risk-free
  • Pay close attention to script/free-model-extraction.ts - incomplete cost checks, and .github/workflows/refresh-model-capabilities.yml - missing Opencode source read step.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="script/free-model-extraction.ts">

<violation number="1" location="script/free-model-extraction.ts:18">
P2: Free-model detection is incomplete because it only checks `cost.input` and can include models with non-zero output or other billed costs.</violation>
</file>

<file name=".github/workflows/refresh-model-capabilities.yml">

<violation number="1" location=".github/workflows/refresh-model-capabilities.yml:32">
P2: Custom agent: **Opencode Compatibility**

Opencode snapshot generation is added without cloning/reading anomalyco/opencode as required by the rule.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread script/free-model-extraction.ts Outdated
Comment thread .github/workflows/refresh-model-capabilities.yml
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Requires human review: Auto-approval blocked by 1 unresolved issue from previous reviews.

@code-yeongyu
Copy link
Copy Markdown
Owner

[sisyphus-bot]

PR sweep first-pass triage on dev.

Needs rebase + review. Please rebase onto current dev first; then a review can land cleanly.

Assigning code-yeongyu so the maintainer can prioritize a focused review of this PR. If the linked issue above has been triaged, the verdict there is the authoritative direction for this change.

@code-yeongyu code-yeongyu self-assigned this May 16, 2026
@code-yeongyu
Copy link
Copy Markdown
Owner

[sisyphus-bot] Hi mrosnerr. 🙏 Thanks for the free-model fallback work for #1883.

Coming back to this from the 5/16 triage queue. Issue #1883 was closed on 5/13 with this PR (#3821) linked as the auto-closer reference, but the PR itself hasn't landed yet, so the underlying behavior is still on the prior path. The PR also currently reports CONFLICTING against current dev (11 files in src/ and src/cli/ have all moved since 5/06).

If you can spare the time, the cleanest next step is a rebase onto current dev (git fetch upstream && git rebase upstream/dev), resolving any conflicts in the model-resolution + provider-detection files. Once it's clean I'll run a focused review against the 463-line diff against the latest provider-availability shape; some of the related files (detect-current-config.ts, provider-availability.ts) were touched by other in-flight PRs this week so the rebase will need to dovetail with those.

If you'd rather close this and re-open a smaller, more focused PR layered on top of the current free-provider detection code, that's also fine. Either way, thanks for caring enough to fix the silent paid-model fallthrough; it's a real footgun for users who deliberately configured only free providers.

mrosnerr and others added 2 commits May 21, 2026 07:31
… setups (fixes code-yeongyu#1883)

Extract model-selection input helpers to dedicated module. Add
free-model-fallback module with append strategy: free models are added
as a tail to the existing fallback chain rather than replacing it.

Derive free-only from availableModels on warm cache (correct signal
that distinguishes free-tier from paid Zen subscribers). Fall back to
connectedProviders on cold cache only. Skip non-free entries in
cold-cache loop when free-only.

Free model list is sourced from a generated JSON snapshot
(free-opencode-models.generated.json) refreshed by the existing weekly
refresh-model-capabilities workflow. Model rotation is now a JSON edit
instead of a code change.

Paid users are completely unaffected (warm-cache path identical to dev).
Free users get free models appended as a safety net after the system
chain naturally exhausts.

Co-authored-by: MoerAI <skyinbss@gmail.com>
@mrosnerr mrosnerr force-pushed the fix/free-model-fallback-v3 branch from 460be58 to ddefdd9 Compare May 21, 2026 11:32
@mrosnerr
Copy link
Copy Markdown
Contributor Author

@code-yeongyu rebased 👍

@code-yeongyu code-yeongyu force-pushed the dev branch 2 times, most recently from eb25d29 to 2bfad49 Compare May 23, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Should use only free models when no providers configured

3 participants