Dimos Web SDK

[leshy]

Dimos uses pubsub architecture in the backend; we want an equally nice, easy to use dimos web library.

pubsub to web

a service that exposes dimos pubsub to the browser, with roughly the same API dimos pubsub already has:

import { Dimos } from "@dimos/web"

const dimos = await Dimos.connect("url")

console.log(dimos.topics)

dimos.topics.image.subscribe(console.log)
dimos.topics.lidar.subscribe(console.log)

looks easy, but we're subscribing over unreliable connections to pipelines pushing large amounts of data. requirements:

• per-stream QoS — an image topic eating all the bandwidth should drop frames, not slow down odometry
• optional serverside data decimation (voxelize a pointcloud before sending, for example)
• low latency control path (teleop)

open questions:

• QoS model — mqtt QoS and zenoh QoS are good references. should the client set QoS dynamically per subscription, like setQos in TS API Spec #2502?
• websocket vs webrtc data channels? one channel or one per topic?
• video: we can't stream our raw Image topic — encode h264 serverside?

module/subsystem web

a module should be able to define custom messages to send clientside, serve custom files (including html), and register interactions with the web (a clientside button calling a module rpc, teleop). #2502 sketches this with @web_module / @web_init.

gui

easy to use visualizers of pipeline data — efficient pointcloud visualizer etc.

prior art

#2502 is a concrete proposal already covering much of this (dynamic QoS, per-connection whitelists, module frontends, RPC). treat this issue as the umbrella spec and #2502 as a candidate design.

[leshy]

similar (but nicer formated) design from Paul #2710