Skip to content

Add GGUF compatibility inspection and resilient model download management #523

Description

@sauravpanda

Suggested priority: P2 — make model selection and loading predictable.

Users should be able to learn whether a model is supported and likely to fit before downloading its full weights. Existing GGUF metadata parsing, memory planning, chunked loading, and OPFS caching provide foundations, but need a unified preflight and download-management API.

Scope and acceptance criteria

  • Inspect GGUF headers via bounded range reads where supported; handle servers without Range support and reject malformed metadata.
  • Report architecture, quantization/mixed formats, context limits, tokenizer availability, and backend compatibility before weight allocation.
  • Estimate peak loading/inference memory, including staging copies, actual KV dtype, and GPU buffer limits; expose assumptions rather than promising exact available RAM.
  • Recommend a context/quantization configuration when the requested model exceeds the configured budget.
  • Support cancel/resume for downloads when server validators and ranges permit, and restart safely when model content changes.
  • Verify expected size/checksum when a manifest provides them; publish cache entries atomically so incomplete files are not treated as usable models.
  • Integrate storage quota estimates, understandable errors, and cache deletion/eviction controls.
  • Test interrupted downloads, quota failures, stale/corrupt cache entries, unsupported models, and a successful offline reload.

Evidence: memory planner, GGUF parser, OPFS helpers.

Related: #449 already tracks the model-gallery UI. This issue supplies the compatibility/download-management layer that gallery can consume; avoid duplicating its card/filter/load-button work. Builds on closed #394 and #397.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    browserBrowser/WASM specificenhancementNew feature or improvement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions