Skip to content

feat(mlx): add Apple Silicon video backend - #11444

Open
localai-org-maint-bot wants to merge 2 commits into
masterfrom
cron/issue-8085-mlx-video-20260810
Open

feat(mlx): add Apple Silicon video backend#11444
localai-org-maint-bot wants to merge 2 commits into
masterfrom
cron/issue-8085-mlx-video-20260810

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Description

Adds an Apple-Silicon-only mlx-video backend for LTX-2 and converted Wan2.1/Wan2.2 checkpoints through LocalAI's existing POST /video API.

  • maps LocalAI video requests to the pinned upstream MLX-Video CLI without shell interpolation
  • supports text-to-video and image-to-video, plus LTX-2 end-frame conditioning
  • wires Darwin backend builds, backend-gallery capability selection, and backend capability discovery
  • documents installation and model configuration, with a runnable example

Closes #8085

Notes for reviewers

The pinned upstream currently exposes generation as CLI-oriented functions that load their pipeline within a generation invocation. This backend keeps that lifecycle rather than relying on upstream internals that do not provide a stable reusable pipeline API.

Wan checkpoints must already be converted to MLX and available as a local directory. The upstream LTX-2 path can resolve supported Hugging Face repositories.

Test plan

  • python3 backend/python/mlx-video/test.py (4 tests pass)
  • python3 -m py_compile backend/python/mlx-video/backend.py backend/python/mlx-video/mlx_video_utils.py backend/python/mlx-video/test.py
  • bash -n backend/python/mlx-video/install.sh backend/python/mlx-video/run.sh backend/python/mlx-video/test.sh
  • YAML parse for backend/index.yaml, .github/backend-matrix.yml, and the example config
  • make test-ci-scripts (41 tests pass)
  • git diff --check
  • go test ./core/config/... reached 499/500 specs; the only failure is the existing network-dependent config validation test because this environment returns HTTP 403 for raw.githubusercontent.com

A native backend package/model generation smoke test requires an Apple Silicon runner and is left to Darwin CI.

Add a Darwin-only MLX-Video backend for LTX-2 and converted Wan checkpoints, expose it through the existing video API, and wire packaging, discovery, tests, docs, and an example.

Assisted-by: Codex:gpt-5
The pinned mlx-video package requires Python 3.11 or newer, while an empty PYTHON_VERSION selected the backend helper default of 3.10. Pin the available 3.11.13 portable runtime for the Darwin package build.

Assisted-by: Codex:gpt-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Support for mlx-video Backend for Video Generation on macOS

1 participant