SCOPE: Motion plan replay plugin by sucrammal · Pull Request #782 · viamrobotics/visualization

Marcus (sucrammal) · 2026-06-16T20:11:36Z

Changes

Adds useMotionPlanReplayer.svelte.ts context managing plan list state (add/remove/select/step) and driving playback through reconcileSnapshotEntities
Adds parse-plan.ts (zod-validated extraction of frame_system/trajectory from concatenated plan JSON chunks), build-frame-descriptors.ts (resolves static vs. revolute-joint frames, handles quaternion/euler/OV orientations, model-frame reparenting into per-step transforms), and plan-to-snapshots.ts (renders trajectory steps into Snapshot[])
Adds MotionPlanReplayerUI.svelte (floating panel – upload, list, select, remove) and MotionPlanReplayerScrubber.svelte (play/pause/step/seek transport), wired into +layout.svelte and exported from plugins/index.ts
Adds PartOfPlan relation (relations.ts) tying every entity spawned for a plan, including nested model assets, to a single plan-root entity, so switching, clearing, or removing a plan cascades cleanup via koota's autoDestroy instead of manual entity bookkeeping
Tests: Adds specs for parse-plan, build-frame-descriptors, plan-to-snapshots, and the PartOfPlan relation

Links:

Testing:

Open the visualizer, click the "Motion Plan Replayer" dashboard button (or drag-and-drop a plan JSON file) to open the panel
Feel free to use some sample plans here.
Upload a valid motion plan JSON (frame_system + trajectory) → it appears in the list and the scrubber appears at the bottom
Press play → frames animate through the trajectory; step forward/back and drag the scrubber → poses update correctly at each step
With one plan active, select a second plan → the first plan's entities (including any model geometry) are fully removed before the second renders
Remove the active plan, or click Exit on the scrubber → no leftover entities remain in the scene
Drop/upload a malformed or non-plan JSON file → a toast error appears and nothing is spawned
Upload a file with a name that's already loaded → the "already loaded" warning toast appears

POC demo:

motion-plan-replay-POC.mov

Salad1:

saladMotionPlanReplay.mov

Vino:

vinoMotionPlanReplay.mov

Cappucina

cappucinaMotionPlanReplay.mov

For now, no helpers are being placed in lib/snapshot.ts. Consider moving some of these helpers later.

changeset-bot · 2026-06-16T20:11:41Z

⚠️ No Changeset found

Latest commit: e13263d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-06-16T20:15:38Z

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-10 19:52 UTC

1. Joints (frame_type: rotational) should not be apart of the main tree 2. Geometry: translations vs. translations. One is the offset from origin frame for the attachment point of the next link, the other is the offset from origin frame to geometry center of the link.

Cameras, grippers, and obstacles parented to model frames (e.g. "left-arm") were permanently orphaned because model frames are never spawned as ECS entities. They now redirect to the arm's end-effector via model.primary_output_frame, falling back to model.links[last].id: Viam's convention that the last link is always the end-effector.

Updated motion replay entities get their preserved opacity restored, so slider changes stick across scrubs.

Semantic error messages for duplicate or invalid motion plan JSON.

Fill in gaps (large jumps in angle change) with sub-steps, where every joint moves at a velocity proportional to its total delta.

- planToSnapshots unexported - GeometryDescriptor collapsed: parseGeometry() now returns Geometry | null directly - computeJointedLinkPose moved - LocalPose → Pose getting rid of unnecessary type.

Matthew MacFarquhar (mattmacf98)

Great POC! looks very nice!

my main things for when we do the full impl are

we should try to be more koota native and create a plan entity and relationships for the plan <--> plan entities
I think we can use zod to parse the plans a little easier
we should strip back some of the (very cool) functionality you added in P0 (i.e. no linear interp between steps, no preserve opacity and visibility (yet), no multiple plans loaded at a time)

Matthew MacFarquhar (mattmacf98) · 2026-06-22T16:26:07Z

 }
+
+/* Push toasts above the motion plan scrubber bar (fixed bottom-4, ~44px tall). */
+body.has-scrubber [aria-label='Toasts'] {


todo: for final impl we probably should look into how to do this with component placements instead of the app css

Matthew MacFarquhar (mattmacf98) · 2026-06-22T16:33:59Z

+
+	const clearActivePlan = () => {
+		for (const entry of entityMap.values()) {
+			if (world.has(entry.entity)) hierarchy.destroyEntityTree(world, entry.entity)


for the final setup, lets look into making a new PlanEntity, which we can then give relationships to all the world entities that are part of it (see the selection plugin https://github.com/viamrobotics/visualization/blob/main/src/lib/plugins/Selection/traits.ts for inspo)

this way we can make this flow more koota native and the relationship has an auto destroy hook e.x. https://github.com/viamrobotics/visualization/blob/main/src/lib/plugins/Selection/relations.ts#L9 so we can make it so all world entities auto clean up when a plan entity is removed

Tried mirroring the pattern from Selection plugin here: 5631b12. This makes the destroy plan path logically clearer! Is this up to your spec?

Matthew MacFarquhar (mattmacf98) · 2026-06-22T17:04:33Z

+	provideMotionPlanReplayer(untrack(() => plans))
+</script>
+
+{#if plans === undefined}


it would be cool if drag and drop just updated the plans array and then we could use the ui for both cases

Designed a more explicit hook here (f3f70cb) which calls the same addPlan hook from app vs. standalone.

As part of POC, file drop imported plugin types for parsing motion plans. Moved out of FileDrop and created upload functionality contained to the plugin

This reverts commit 6ee653c.

This reverts commit b941f27.

When a frame (e.g. gripper) is parented to a model frame (e.g. left-arm), it should first try to parent itself to the model frame's primary_output_frame, which is the primary tip/end-effector frame. If that metadata is not found, default to the last frame ofthe model frame

- Added src/lib/plugins/MotionPlanReplayer/relations.ts (PartOfPlan relation, autoDestroy: 'source') - Updated useMotionPlanReplayer.svelte.ts to spawn a planEntity, tag spawned entities (recursively through ChildOf) with PartOfPlan, replace the manual clearActivePlan teardown loop with a single planEntity.destroy(), and drive the coloring/opacity pass off world.query(planRelations.PartOfPlan(planEntity))

Jason (mrloureed) · 2026-07-01T20:02:57Z

Marcus (@sucrammal) Some UX / UI feedback for you:

axis helpers are not showing up like they do generally
color mapping would be nice if we know the resource type (arm is one color, grippers, etc.) but if we don't have the info it's okay to not do this
motion player items look like radio buttons so it's not obvious to me that you can click them to deselect; consider using checkboxes or eye icons instead; maybe the latter for consistency
when one uploads a motion file it should auto select
move the exit replay button to the right rather than on the left
make the movie player widget's background the same as selected button dark gray

Jason (mrloureed)

added comment to the ticket for design feedback

- Motion plan items use eyes for toggle visibility - Scrubber matches color of selected buttons - Auto-select newly uploaded plans - Move exit button to the right.

Marcus (sucrammal) · 2026-07-06T21:04:44Z

Jason (@mrloureed) Re: your UI/UX feedback. Alongside persistent color, opacity, and visibility toggle edits, I'm going to push off implementing axes helpers and anything color at the moment, as per Matthew MacFarquhar (@mattmacf98):

we should strip back some of the (very cool) functionality you added in P0 (i.e. no linear interp between steps, no preserve opacity and visibility (yet), no multiple plans loaded at a time)

But, implemented everything else!

Screen.Recording.2026-07-06.at.4.43.34.PM.mov

Marcus (sucrammal) added 5 commits June 16, 2026 15:03

Scaffold plan JSON → Snapshot[] data pipeline

5920b5c

Move tests

3a81882

Allow file dropping of motion plan JSON

ea40eae

Add in motion plan plugin UI elements + reorg

d8782fa

For now, no helpers are being placed in lib/snapshot.ts. Consider moving some of these helpers later.

use in +layout

caa9ccd

Marcus (sucrammal) added 11 commits June 17, 2026 10:05

Fix file drop; scrubber appears when plan selected

4334197

Motion plans running, first go

cb7336f

Merge branch 'main' into APP-9316-motion-plan-replay

2c26251

Motion replay entity opacity

b941f27

Updated motion replay entities get their preserved opacity restored, so slider changes stick across scrubs.

Better error state

6ad0e89

Semantic error messages for duplicate or invalid motion plan JSON.

Cleanup logs and code

db5ab73

Interpolation per Dan's advice

6ee653c

Fill in gaps (large jumps in angle change) with sub-steps, where every joint moves at a velocity proportional to its total delta.

Refactor and code cleanup.

1304ee1

- planToSnapshots unexported - GeometryDescriptor collapsed: parseGeometry() now returns Geometry | null directly - computeJointedLinkPose moved - LocalPose → Pose getting rid of unnecessary type.

Update plan-to-snapshots.spec.ts

9445896

Marcus (sucrammal) requested a review from Matthew MacFarquhar (mattmacf98) June 18, 2026 21:07

Matthew MacFarquhar (mattmacf98) reviewed Jun 22, 2026

View reviewed changes

Marcus (sucrammal) added 10 commits June 22, 2026 14:25

Fix file drop

7930ad6

As part of POC, file drop imported plugin types for parsing motion plans. Moved out of FileDrop and created upload functionality contained to the plugin

Use prime / lucide for style consistency

90625d7

Revert "Interpolation per Dan's advice"

36ffdf1

This reverts commit 6ee653c.

Revert "Motion replay entity opacity"

21d810f

This reverts commit b941f27.

Consolidate scrubber interaction with timer

111fafe

Parse plans with zod, simplify caller.

d3be9d2

Design clear hook for app-embedded plan upload

f3f70cb

Refactor: Joints as ECS entities

58cd153

Readd support for frames parented to model frames

e87fe24

Euler angle support; Fix geometry translation

c08a58c

Marcus (sucrammal) added 3 commits July 1, 2026 11:36

Merge branch 'main' into APP-9316-motion-plan-replay

8fdb6b9

Marcus (sucrammal) requested a review from Matthew MacFarquhar (mattmacf98) July 1, 2026 16:29

Revert useFileDrop

cc87cee

Marcus (sucrammal) requested a review from Jason (mrloureed) July 1, 2026 18:40

Jason (mrloureed) requested changes Jul 2, 2026

View reviewed changes

Easy UI/UX hits from Jason

fa999e5

- Motion plan items use eyes for toggle visibility - Scrubber matches color of selected buttons - Auto-select newly uploaded plans - Move exit button to the right.

Merge branch 'main' into APP-9316-motion-plan-replay

31f4afe

Marcus (sucrammal) requested a review from Jason (mrloureed) July 6, 2026 21:08

Lint

e13263d

Marcus (sucrammal) marked this pull request as ready for review July 7, 2026 15:30

Marcus (sucrammal) closed this Jul 10, 2026

Uh oh!

Conversation

Marcus (sucrammal) commented Jun 16, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Links:

Testing:

Uh oh!

changeset-bot Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Matthew MacFarquhar (mattmacf98) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Matthew MacFarquhar (mattmacf98) Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Matthew MacFarquhar (mattmacf98) Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Marcus (sucrammal) Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Matthew MacFarquhar (mattmacf98) Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Marcus (sucrammal) Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Jason (mrloureed) commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jason (mrloureed) left a comment

Choose a reason for hiding this comment

Uh oh!

Marcus (sucrammal) commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Marcus (sucrammal) commented Jun 16, 2026 •

edited by atlassian Bot

Loading

changeset-bot Bot commented Jun 16, 2026 •

edited

Loading

github-actions Bot commented Jun 16, 2026 •

edited

Loading

Jason (mrloureed) commented Jul 1, 2026 •

edited

Loading

Marcus (sucrammal) commented Jul 6, 2026 •

edited

Loading