-
Notifications
You must be signed in to change notification settings - Fork 191
docs: documentation for autoglouon usage #619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ZabinskiMichal
wants to merge
3
commits into
kserve:main
Choose a base branch
from
ZabinskiMichal:autogluon_documentation_support
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
256 changes: 256 additions & 0 deletions
256
docs/model-serving/predictive-inference/frameworks/autogluon/autogluon.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,256 @@ | ||
| --- | ||
| title: AutoGluon | ||
| description: Deploy AutoGluon TabularPredictor and TimeSeriesPredictor models with KServe | ||
| --- | ||
|
|
||
| # Deploying AutoGluon Models with KServe | ||
|
|
||
| This guide explains how to deploy AutoGluon models with KServe using the `autogluon` model format and the `kserve-autogluonserver` runtime. | ||
|
|
||
| The runtime supports: | ||
|
|
||
| - `autogluon.tabular.TabularPredictor` | ||
| - `autogluon.timeseries.TimeSeriesPredictor` | ||
|
|
||
| ## Supported Predictor Types and Protocols | ||
|
|
||
| | Predictor Type | Supported Inference Protocol | | ||
| | --- | --- | | ||
| | TabularPredictor | REST v1, REST v2 | | ||
| | TimeSeriesPredictor | REST v1 JSON only | | ||
|
|
||
| Time series v2 tensor payloads are not supported in this release. | ||
|
|
||
| ## Auto-Detection and `modelFormat.name` | ||
|
|
||
| AutoGluon predictor type is auto-detected from the model artifact in `storageUri`: | ||
|
|
||
| - The runtime first tries `TimeSeriesPredictor.load(...)`. | ||
| - If that fails, it tries `TabularPredictor.load(...)`. | ||
|
|
||
| Use `modelFormat.name: autogluon` for both tabular and time series models. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before you begin, make sure you have: | ||
|
|
||
| - A Kubernetes cluster with [KServe installed](../../../../getting-started/quickstart-guide.md). | ||
| - Access to a storage backend reachable by your cluster (for example, GCS, S3, or Azure Blob). | ||
| - A model saved with either `TabularPredictor.save(path)` or `TimeSeriesPredictor.save(path)`. | ||
|
|
||
| :::warning Model Artifacts Must Be a Directory | ||
| AutoGluon models must be stored as a predictor directory generated by `TabularPredictor.save(path)` or `TimeSeriesPredictor.save(path)`, not as a single file artifact. | ||
| ::: | ||
|
|
||
| ## Deploy the Model with REST Endpoint | ||
|
|
||
| Create an `InferenceService` with explicit runtime selection. | ||
|
|
||
| ### Tabular Example | ||
|
|
||
| ```yaml | ||
| apiVersion: "serving.kserve.io/v1beta1" | ||
| kind: "InferenceService" | ||
| metadata: | ||
| name: "autogluon-titanic" | ||
| spec: | ||
| predictor: | ||
| model: | ||
| modelFormat: | ||
| name: autogluon | ||
| protocolVersion: v2 | ||
|
Jooho marked this conversation as resolved.
|
||
| runtime: kserve-autogluonserver | ||
| storageUri: "gs://your-bucket/autogluon-model/" | ||
| resources: | ||
| requests: | ||
| cpu: "100m" | ||
| memory: "1Gi" | ||
| limits: | ||
| cpu: "1" | ||
| memory: "2Gi" | ||
| ``` | ||
|
|
||
| ### Time Series Example | ||
|
|
||
| ```yaml | ||
| apiVersion: "serving.kserve.io/v1beta1" | ||
| kind: "InferenceService" | ||
| metadata: | ||
| name: "autogluon-ts-forecast" | ||
| spec: | ||
| predictor: | ||
| model: | ||
| modelFormat: | ||
| name: autogluon | ||
| runtime: kserve-autogluonserver | ||
| storageUri: "gs://your-bucket/path/to/timeseries-predictor-save/" | ||
| resources: | ||
| requests: | ||
| cpu: "100m" | ||
| memory: "2Gi" | ||
| limits: | ||
| cpu: "2" | ||
| memory: "4Gi" | ||
| ``` | ||
|
|
||
| Apply your manifest: | ||
|
|
||
| ```bash | ||
| kubectl apply -f autogluon.yaml | ||
| ``` | ||
|
|
||
| :::tip Runtime Availability | ||
| The `kserve-autogluonserver` runtime may not be installed by default in every release bundle. Verify that the `ClusterServingRuntime` exists in your cluster before deploying the `InferenceService`. | ||
| ::: | ||
|
|
||
| ## Run Inference | ||
|
|
||
| First, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set `INGRESS_HOST` and `INGRESS_PORT`. | ||
|
|
||
| ### Tabular REST v1 Example | ||
|
|
||
| Use this for tabular models. | ||
|
|
||
| Sample payload: | ||
|
|
||
| ```json | ||
| { | ||
| "instances": [ | ||
| { | ||
| "PassengerId": 1, | ||
| "Pclass": 3, | ||
| "Sex": "male" | ||
| }, | ||
| { | ||
| "PassengerId": 2, | ||
| "Pclass": 1, | ||
| "Sex": "female" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| Before sending the request, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set the `INGRESS_HOST` and `INGRESS_PORT` environment variables. | ||
|
|
||
| ```bash | ||
| SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-titanic -o jsonpath='{.status.url}' | cut -d "/" -f 3) | ||
| curl -v \ | ||
| -H "Host: ${SERVICE_HOSTNAME}" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d @./autogluon-input-v1.json \ | ||
| http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/autogluon-titanic:predict | ||
|
Jooho marked this conversation as resolved.
|
||
| ``` | ||
|
|
||
| ### Tabular REST v2 Example | ||
|
|
||
| For v2 requests, provide one input tensor per feature. Each tensor `name` must match the feature name expected by the model, and all features must have a consistent batch length. | ||
|
|
||
| ```json | ||
| { | ||
| "inputs": [ | ||
| { "name": "PassengerId", "shape": [2], "datatype": "INT64", "data": [1, 2] }, | ||
| { "name": "Pclass", "shape": [2], "datatype": "INT64", "data": [3, 1] }, | ||
| { "name": "Sex", "shape": [2], "datatype": "BYTES", "data": ["male", "female"] } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| Before sending the request, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set the `INGRESS_HOST` and `INGRESS_PORT` environment variables. | ||
|
|
||
| ```bash | ||
| SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-titanic -o jsonpath='{.status.url}' | cut -d "/" -f 3) | ||
| curl -v \ | ||
| -H "Host: ${SERVICE_HOSTNAME}" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d @./autogluon-input-v2.json \ | ||
| http://${INGRESS_HOST}:${INGRESS_PORT}/v2/models/autogluon-titanic/infer | ||
| ``` | ||
|
|
||
| Expected response: | ||
|
|
||
| ```json | ||
| { | ||
| "model_name": "autogluon-titanic", | ||
| "outputs": [ | ||
| { "name": "predictions", "datatype": "INT64", "shape": [2], "data": [1, 0] } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ### Time Series REST v1 Example | ||
|
|
||
| Use this for `TimeSeriesPredictor` models. Time series requests use JSON payloads with top-level `instances` and optional `known_covariates`. | ||
|
|
||
| Sample payload: | ||
|
|
||
| ```json | ||
| { | ||
| "instances": [ | ||
| { "item_id": "A", "timestamp": "2024-01-01T00:00:00", "target": 12.3 }, | ||
| { "item_id": "A", "timestamp": "2024-01-02T00:00:00", "target": 11.1 } | ||
| ], | ||
| "known_covariates": [ | ||
| { "item_id": "A", "timestamp": "2024-01-03T00:00:00", "promo": 1 }, | ||
| { "item_id": "A", "timestamp": "2024-01-04T00:00:00", "promo": 0 } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| Before sending the request, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set the `INGRESS_HOST` and `INGRESS_PORT` environment variables. | ||
|
|
||
| ```bash | ||
| SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-ts-forecast -o jsonpath='{.status.url}' | cut -d "/" -f 3) | ||
| curl -v \ | ||
| -H "Host: ${SERVICE_HOSTNAME}" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d @./autogluon-timeseries-input-v1.json \ | ||
| http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/autogluon-ts-forecast:predict | ||
| ``` | ||
|
|
||
| Expected response: | ||
|
|
||
| ```json | ||
| { | ||
| "predictions": [ | ||
| { | ||
| "item_id": "A", | ||
| "timestamp": "2024-01-03T00:00:00", | ||
| "mean": 10.87, | ||
| "0.1": 9.95, | ||
| "0.9": 11.66 | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ## Prediction Probabilities | ||
|
|
||
| The AutoGluon runtime supports returning probabilities via the `PREDICT_PROBA=true` environment setting in the runtime container configuration. | ||
|
|
||
| :::note | ||
| When probability output is enabled, output schema differs from class prediction output. | ||
| ::: | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| - `PREDICT_PROBA` (tabular): set to `true` to return class probabilities via `predict_proba()` instead of predicted labels via `predict()`. | ||
| - `AUTOGLUON_TS_ID_COLUMN` (time series): overrides the item identifier column used in JSON payloads. | ||
| - `AUTOGLUON_TS_TIMESTAMP_COLUMN` (time series): overrides the timestamp column used in JSON payloads. | ||
|
|
||
| For time series, the target column name always comes from `TimeSeriesPredictor.target` in the loaded model and is not configurable via environment variable. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| - Ensure `storageUri` points to a predictor directory created by `TabularPredictor.save(path)` or `TimeSeriesPredictor.save(path)`. | ||
| - For tabular v2 requests, verify each feature is provided as a separate tensor with matching batch length. | ||
| - For time series requests, ensure column names in `instances` and `known_covariates` match model expectations (including id, timestamp, and target). | ||
| - For time series models, use REST v1 JSON (`/v1/models/{name}:predict`) instead of v2 tensor payloads. | ||
| - If no runtime is selected automatically, set `runtime: kserve-autogluonserver` explicitly. | ||
|
|
||
| ## References | ||
|
|
||
| - [AutoGluon server README](https://github.com/kserve/kserve/tree/master/python/autogluonserver) | ||
| - [KServe runtime definitions](https://github.com/kserve/kserve/tree/master/config/runtimes) | ||
| - [KServe examples and tests](https://github.com/kserve/kserve/tree/master/test/e2e/predictor) | ||
| - [AutoGluon runtime tests](https://github.com/kserve/kserve/blob/master/test/e2e/predictor/test_autogluon.py) | ||
| - [AutoGluon time series tests](https://github.com/kserve/kserve/blob/master/test/e2e/predictor/test_autogluon_timeseries.py) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a public example model to play with?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, we have one pubic model in out bucket, but since now moving that to kserv's bucket, lets give example from your bucket when this fill be added
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we will be moving to a public bucket, once we get it we will make these avilalable there as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yuzisun have you uploaded the model?