need to support ds.publish(deploy=True) for non-remote modes

[lukemarsden]

for script and interactive modes, we need a way to trigger model builds, deployments & monitoring from the ds.publish() call. in remote mode, ds.publish(deploy=True) does the following:

=== Dotscience remote publish ===

Found project mnist-keras-tf.

*  Uploading output/model files....... done
   -> Dotscience run: https://[redacted]-dotscience.com/project/b0ca2640-01dd-42cb-b505-3e7f281b9cfb/runs/summary/f3e055f8-187e-4933-a748-bb551586f752

*  Building docker image................ done
   -> Docker image: quay.io/dotscience-playground/models:f4d65f0c-8768-4cc3-995b-b14d038f97ae-0a296917-fa0a-4c6a-ba36-68002f38bd60

*  Deploying to Kubernetes... done
   -> Endpoint: https://b6185b30.app.cloud.dotscience.net/v1/models/model:predict

*  Creating Grafana dashboard... done
   -> Dashboard: https://monitoring.[redacted]-dotscience.com/d/064e26cf-1b3a-42f7-be2a-e7474ff84cfe/monitoring-model-mnistkerastf

Waiting for model endpoint to become active.... done
=== Dotscience publish complete ===

Need something equivalent for script and interactive modes. Possibly this could go hand-in-hand with switching those modes to use an explicit, synchronous API for runs rather than waiting for runs to be picked up.

Design question: should the build, publish, monitor APIs be exposed on the agent, so dotscience python lib talks to local container on the runner? Or should we plumb in creds so the python lib can talk directly to the gateway?

Alaric's opinion: "No" to both. Just set a flag in the model metadata that the gateway notices and decides to trigger the build and deploy when it registers the model in the commit watcher. That way there's no need to block the python library until the model is uploaded, no issues when connectivity to the gateway is flaky, etc.

[rusenask]

Maybe a mix of both would be the best:

1. Synchronous API on the agent that the library can use. It would also mean that it could show nice errors in the cells when something fails.
2. Once sync is complete, do an HTTP request to the gateway, the same one as the library already uses for remote mode so we remain consistent.

I am quite against "magical" metadata tags that would actually start deployments. Deployments come and go, you can delete them from k8s, through UI or through CLI, we shouldn't contaminate metadata with deployment stuff.

It will also be better this way in the future for users that only use dotscience for deployment & metrics and not using tracking & provenance.

[itamarst]

It's not clear to me how a synchronous API would allow showing errors, because the deploy would still be asynchronous? Or am I misunderstanding the proposal? It sounds like you're saying:

1. ds in user process calls agent saying "please deploy this."
2. agent says "OK"
3. eventually agent finishes syncing
4. agent tells gateway to deploy

Yes?

[itamarst]

@lukemarsden what is the use case for this?

I can see in remote mode you want to be able to publish in automated fashion... but if someone is using the Dotscience UI anyway, why not just have them use the UI?

Like... ds.publish(deploy=True) could print out a link that people can then follow to do the deployment, in Jupyter at lease.

[prisamuel]

@itamarst The current user journey for deploying models for non-remote mode is a bit fragmented. The user registers the models in Jupyter and then navigates across to the models tab, searches for the right model and then clicks through to build and then finds the built model in the deploy tab and clicks through to deploy. The idea behind doing a ds.publish(deploy=True) within Jupyter was to make the initial build & deployment seamless and easy.

There seems to be a pending design discussion around the approach to doing this. Perhaps we should discourse on this. @rusenask and @lukemarsden API's on the agent are not consistent with our current approach, the gateway does all the auth and orchestrating for us and it would be sensible to continue sending HTTP requests to the gateway for both the build and the deploy?

[alaric-dotmesh]

I think there's also a bigger issue of whether we want to control "deploy" from ds.publish at all.

If we do I still think that setting a metadata flag for "this model needs to be deployed" (and never having ds talk to the gateway to deploy stuff, even in remote mode) would be the cleanest thing: it would work the same in remote and runner mode, it has fewer network failure modes as all communication is done through the dotmesh push, etc.

But why is the code that produces the model making the decision as to whether it should be deployed or not? As @itamarst says I wonder what the use case is. Alternatives to consider are:

1. A project-level setting of "deploy all models as soon as they arrive" - if the deploy=True is hardcoded in the source, this is almost the same thing, except that it's controlled from a configuration flag in the web UI rather than embedded in source.
2. Why would we want to build a model and NOT deploy it? (guessing) Is it for efficiency, as only a fraction of models are ever actually used? In which case, might we want to change the deployment structure a little so that it's done lazily - for every model we deploy a lightweight placeholder that, the first time it's actually invoked, fetches the actual model files and then works thereafter? Or something?

Yes, I think we should take this to discourse! And we need to start with better understanding the actual use case.

[lukemarsden]

The product requirement for this is to be able to say we make deploying to Kubernetes insanely easy, with just one command. Yes, that command is a shortcut to clicking a bunch of buttons (or CLI commands or API calls), but we need to make that one command work in hosted Jupyter mode so we can demo insanely easy deploy + monitor AND track data and model provenance -- need the whole value prop in a single, simple example. (Clicking around the UI after doing the command is a useful way to explain what it's done, btw.)

I really don't mind how it's implemented, please just pick something and get on with it. :)

[rusenask]

I think that it must be easy to start, stop, update deployments. Having explicit functions for that helps :)

Regarding build & deploy pairing, builds are fast but let's not forget about the use case were we just build but users use their own platform to deploy models (they might not use Kubernetes).

[lukemarsden]

Agreed, the ds.publish(deploy=True) is just a shortcut, really for demo & getting started purposes. Serious users will want to use the fully fledged tools for managing deployments (our wonderful CLI and GUI).

[alaric-dotmesh]

If we have a project-level setting (default to on!) to build&deploy models by default, that will fulfill the requirement for demos and getting started, and people can choose to turn it off and use manual control if they want to do advanced stuff. I don't see controlled deployment from ds.publish as being essential to either use case!