diff --git a/docs/model-serving/predictive-inference/frameworks/autogluon/autogluon.md b/docs/model-serving/predictive-inference/frameworks/autogluon/autogluon.md new file mode 100644 index 000000000..8fd2d5fd6 --- /dev/null +++ b/docs/model-serving/predictive-inference/frameworks/autogluon/autogluon.md @@ -0,0 +1,256 @@ +--- +title: AutoGluon +description: Deploy AutoGluon TabularPredictor and TimeSeriesPredictor models with KServe +--- + +# Deploying AutoGluon Models with KServe + +This guide explains how to deploy AutoGluon models with KServe using the `autogluon` model format and the `kserve-autogluonserver` runtime. + +The runtime supports: + +- `autogluon.tabular.TabularPredictor` +- `autogluon.timeseries.TimeSeriesPredictor` + +## Supported Predictor Types and Protocols + +| Predictor Type | Supported Inference Protocol | +| --- | --- | +| TabularPredictor | REST v1, REST v2 | +| TimeSeriesPredictor | REST v1 JSON only | + +Time series v2 tensor payloads are not supported in this release. + +## Auto-Detection and `modelFormat.name` + +AutoGluon predictor type is auto-detected from the model artifact in `storageUri`: + +- The runtime first tries `TimeSeriesPredictor.load(...)`. +- If that fails, it tries `TabularPredictor.load(...)`. + +Use `modelFormat.name: autogluon` for both tabular and time series models. + +## Prerequisites + +Before you begin, make sure you have: + +- A Kubernetes cluster with [KServe installed](../../../../getting-started/quickstart-guide.md). +- Access to a storage backend reachable by your cluster (for example, GCS, S3, or Azure Blob). +- A model saved with either `TabularPredictor.save(path)` or `TimeSeriesPredictor.save(path)`. + +:::warning Model Artifacts Must Be a Directory +AutoGluon models must be stored as a predictor directory generated by `TabularPredictor.save(path)` or `TimeSeriesPredictor.save(path)`, not as a single file artifact. +::: + +## Deploy the Model with REST Endpoint + +Create an `InferenceService` with explicit runtime selection. + +### Tabular Example + +```yaml +apiVersion: "serving.kserve.io/v1beta1" +kind: "InferenceService" +metadata: + name: "autogluon-titanic" +spec: + predictor: + model: + modelFormat: + name: autogluon + protocolVersion: v2 + runtime: kserve-autogluonserver + storageUri: "gs://your-bucket/autogluon-model/" + resources: + requests: + cpu: "100m" + memory: "1Gi" + limits: + cpu: "1" + memory: "2Gi" +``` + +### Time Series Example + +```yaml +apiVersion: "serving.kserve.io/v1beta1" +kind: "InferenceService" +metadata: + name: "autogluon-ts-forecast" +spec: + predictor: + model: + modelFormat: + name: autogluon + runtime: kserve-autogluonserver + storageUri: "gs://your-bucket/path/to/timeseries-predictor-save/" + resources: + requests: + cpu: "100m" + memory: "2Gi" + limits: + cpu: "2" + memory: "4Gi" +``` + +Apply your manifest: + +```bash +kubectl apply -f autogluon.yaml +``` + +:::tip Runtime Availability +The `kserve-autogluonserver` runtime may not be installed by default in every release bundle. Verify that the `ClusterServingRuntime` exists in your cluster before deploying the `InferenceService`. +::: + +## Run Inference + +First, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set `INGRESS_HOST` and `INGRESS_PORT`. + +### Tabular REST v1 Example + +Use this for tabular models. + +Sample payload: + +```json +{ + "instances": [ + { + "PassengerId": 1, + "Pclass": 3, + "Sex": "male" + }, + { + "PassengerId": 2, + "Pclass": 1, + "Sex": "female" + } + ] +} +``` + +Before sending the request, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set the `INGRESS_HOST` and `INGRESS_PORT` environment variables. + +```bash +SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-titanic -o jsonpath='{.status.url}' | cut -d "/" -f 3) +curl -v \ + -H "Host: ${SERVICE_HOSTNAME}" \ + -H "Content-Type: application/json" \ + -d @./autogluon-input-v1.json \ + http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/autogluon-titanic:predict +``` + +### Tabular REST v2 Example + +For v2 requests, provide one input tensor per feature. Each tensor `name` must match the feature name expected by the model, and all features must have a consistent batch length. + +```json +{ + "inputs": [ + { "name": "PassengerId", "shape": [2], "datatype": "INT64", "data": [1, 2] }, + { "name": "Pclass", "shape": [2], "datatype": "INT64", "data": [3, 1] }, + { "name": "Sex", "shape": [2], "datatype": "BYTES", "data": ["male", "female"] } + ] +} +``` + +Before sending the request, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set the `INGRESS_HOST` and `INGRESS_PORT` environment variables. + +```bash +SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-titanic -o jsonpath='{.status.url}' | cut -d "/" -f 3) +curl -v \ + -H "Host: ${SERVICE_HOSTNAME}" \ + -H "Content-Type: application/json" \ + -d @./autogluon-input-v2.json \ + http://${INGRESS_HOST}:${INGRESS_PORT}/v2/models/autogluon-titanic/infer +``` + +Expected response: + +```json +{ + "model_name": "autogluon-titanic", + "outputs": [ + { "name": "predictions", "datatype": "INT64", "shape": [2], "data": [1, 0] } + ] +} +``` + +### Time Series REST v1 Example + +Use this for `TimeSeriesPredictor` models. Time series requests use JSON payloads with top-level `instances` and optional `known_covariates`. + +Sample payload: + +```json +{ + "instances": [ + { "item_id": "A", "timestamp": "2024-01-01T00:00:00", "target": 12.3 }, + { "item_id": "A", "timestamp": "2024-01-02T00:00:00", "target": 11.1 } + ], + "known_covariates": [ + { "item_id": "A", "timestamp": "2024-01-03T00:00:00", "promo": 1 }, + { "item_id": "A", "timestamp": "2024-01-04T00:00:00", "promo": 0 } + ] +} +``` + +Before sending the request, [determine the ingress IP and ports](../../../../getting-started/predictive-first-isvc.md#4-determine-the-ingress-ip-and-ports), then set the `INGRESS_HOST` and `INGRESS_PORT` environment variables. + +```bash +SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-ts-forecast -o jsonpath='{.status.url}' | cut -d "/" -f 3) +curl -v \ + -H "Host: ${SERVICE_HOSTNAME}" \ + -H "Content-Type: application/json" \ + -d @./autogluon-timeseries-input-v1.json \ + http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/autogluon-ts-forecast:predict +``` + +Expected response: + +```json +{ + "predictions": [ + { + "item_id": "A", + "timestamp": "2024-01-03T00:00:00", + "mean": 10.87, + "0.1": 9.95, + "0.9": 11.66 + } + ] +} +``` + +## Prediction Probabilities + +The AutoGluon runtime supports returning probabilities via the `PREDICT_PROBA=true` environment setting in the runtime container configuration. + +:::note +When probability output is enabled, output schema differs from class prediction output. +::: + +## Environment Variables + +- `PREDICT_PROBA` (tabular): set to `true` to return class probabilities via `predict_proba()` instead of predicted labels via `predict()`. +- `AUTOGLUON_TS_ID_COLUMN` (time series): overrides the item identifier column used in JSON payloads. +- `AUTOGLUON_TS_TIMESTAMP_COLUMN` (time series): overrides the timestamp column used in JSON payloads. + +For time series, the target column name always comes from `TimeSeriesPredictor.target` in the loaded model and is not configurable via environment variable. + +## Troubleshooting + +- Ensure `storageUri` points to a predictor directory created by `TabularPredictor.save(path)` or `TimeSeriesPredictor.save(path)`. +- For tabular v2 requests, verify each feature is provided as a separate tensor with matching batch length. +- For time series requests, ensure column names in `instances` and `known_covariates` match model expectations (including id, timestamp, and target). +- For time series models, use REST v1 JSON (`/v1/models/{name}:predict`) instead of v2 tensor payloads. +- If no runtime is selected automatically, set `runtime: kserve-autogluonserver` explicitly. + +## References + +- [AutoGluon server README](https://github.com/kserve/kserve/tree/master/python/autogluonserver) +- [KServe runtime definitions](https://github.com/kserve/kserve/tree/master/config/runtimes) +- [KServe examples and tests](https://github.com/kserve/kserve/tree/master/test/e2e/predictor) +- [AutoGluon runtime tests](https://github.com/kserve/kserve/blob/master/test/e2e/predictor/test_autogluon.py) +- [AutoGluon time series tests](https://github.com/kserve/kserve/blob/master/test/e2e/predictor/test_autogluon_timeseries.py) diff --git a/docs/model-serving/predictive-inference/frameworks/overview.md b/docs/model-serving/predictive-inference/frameworks/overview.md index a228e80b0..7d19d7203 100644 --- a/docs/model-serving/predictive-inference/frameworks/overview.md +++ b/docs/model-serving/predictive-inference/frameworks/overview.md @@ -20,6 +20,7 @@ KServe supports multiple model serving runtimes including: - **[Hugging Face Server](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver)** - Specialized for transformer models with Open Inference and OpenAI Protocol support with [vLLM](https://github.com/vllm-project/vllm). - **[LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lightgbmserver)** - Specialized for LightGBM models. - **[XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgboostserver)** - Specialized for XGBoost models. +- **[AutoGluon ModelServer](https://github.com/kserve/kserve/tree/master/python/autogluonserver)** - Specialized for AutoGluon TabularPredictor models. - **[PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver)** - Specialized for PMML models. - **[SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver)** - Specialized for SKLearn models. - **[PaddlePaddle ModelServer](https://github.com/kserve/kserve/tree/master/python/paddlepaddle)** - Specialized for PaddlePaddle models. @@ -75,6 +76,7 @@ The following tables show model serving runtimes supported by KServe, split into | Framework | Exported Model Format | HTTP | gRPC | Default Runtime Version | Supported Framework (Major) Version(s) | Examples | |--------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|-------------|------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------| | [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | Custom implementation | v1, v2 | v2 | User-defined | User-defined | [GitHub Examples](https://github.com/kserve/kserve/tree/master/docs/samples/v1beta1/custom) | +| [AutoGluon ModelServer](https://github.com/kserve/kserve/tree/master/python/autogluonserver) | Directory saved with [TabularPredictor.save(path)](https://auto.gluon.ai/stable/api/autogluon.tabular.TabularPredictor.save.html) or [TimeSeriesPredictor.save(path)](https://auto.gluon.ai/stable/api/autogluon.timeseries.TimeSeriesPredictor.save.html) | v1, v2* | -- | Varies by release (verify runtime image tag) | Model format: 1; Runtime dependencies include [AutoGluon Tabular](https://github.com/autogluon/autogluon) and [AutoGluon TimeSeries](https://github.com/autogluon/autogluon) 1.5.x | [README and E2E](https://github.com/kserve/kserve/tree/master/python/autogluonserver) | | [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model (.bst)](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1, v2 | v2 | v (KServe) | 4 | [GitHub Examples](https://github.com/kserve/kserve/tree/master/docs/samples/v1beta1/lightgbm) | | [MLFlow ModelServer](https://mlserver.readthedocs.io/en/latest/runtimes/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.5.0 (MLServer) | 2 | [GitHub Examples](https://github.com/kserve/kserve/tree/master/docs/samples/v1beta1/mlflow) | | [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML (.pmml)](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1, v2 | v2 | v (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)), 3 (Spark MLlib) | [GitHub Examples](https://github.com/kserve/kserve/tree/master/docs/samples/v1beta1/pmml) | @@ -88,6 +90,7 @@ The following tables show model serving runtimes supported by KServe, split into ### Protocol Notes - **\*tensorflow**: TensorFlow implements its own prediction protocol in addition to KServe's standard protocols. See the [TensorFlow Serving Prediction API](https://github.com/tensorflow/serving/blob/master/tensorflow_serving/apis/prediction_service.proto) documentation. +- **\*autogluon**: Time series inference currently uses REST v1 JSON requests (`/v1/models/{name}:predict`). Tabular inference supports REST v1 and REST v2. diff --git a/sidebars.ts b/sidebars.ts index 272e83fd1..c16afb1c4 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -151,6 +151,7 @@ const sidebars: SidebarsConfig = { "model-serving/predictive-inference/frameworks/triton/torchscript/torchscript", "model-serving/predictive-inference/frameworks/sklearn/sklearn", "model-serving/predictive-inference/frameworks/xgboost/xgboost", + "model-serving/predictive-inference/frameworks/autogluon/autogluon", "model-serving/predictive-inference/frameworks/pmml/pmml", "model-serving/predictive-inference/frameworks/spark-mllib/spark-mllib", "model-serving/predictive-inference/frameworks/lightgbm/lightgbm",