Skip to content

Latest commit

 

History

History
51 lines (38 loc) · 2.62 KB

deployments.md

File metadata and controls

51 lines (38 loc) · 2.62 KB

Deployments

This document describes how to run each KernelPort deployment (model + backend + optional worker).

Deployment list

Name Manifest Backend Worker image How to run
identity-onnx models/sample-manifest.yaml onnx — MODEL_MANIFEST_PATH=... model_fetch then serve --backend onnx --model-path /models/<id>/model.onnx
helion-demo — helion kernelport-helion:gpu docker compose up (default)
luxtts models/luxtts-manifest.yaml helion kernelport-luxtts:gpu docker compose -f docker-compose.yml -f docker-compose.luxtts.yml up or Lambda deploy

The authoritative mapping is in deployments/deployments.yaml.

Running the LuxTTS deployment

  1. Set HUGGINGFACE_HUB_TOKEN (or HF_TOKEN) so the LuxTTS worker can pull the model from Hugging Face.
  2. Build and run with the LuxTTS override:
    docker compose -f docker-compose.yml -f docker-compose.luxtts.yml up --build
  3. Inference endpoint: gRPC at localhost:50051, method kernelport.v1.InferenceService/Infer. See LuxTTS tensor contract for input/output tensors.

Lambda Cloud deploy

See Lambda validation for the automated provision → inference validation → benchmark → cleanup workflow, required secrets, artifacts, and manual teardown instructions. The default workload is Helion softmax; LuxTTS is a separate selectable contract test.

LuxTTS tensor contract

Inputs (all optional except text and prompt_audio; omitted use LuxTTS defaults):

Name Dtype Shape Description
text U8 [N] UTF-8 bytes of the text to synthesize.
prompt_audio U8 [M] Raw reference audio file bytes (WAV or MP3).
rms F32 [1] Loudness (default 0.01).
t_shift F32 [1] Sampling param (default 0.9).
num_steps I32 [1] Sampling steps (default 4).
speed F32 [1] Playback speed (default 1.0).
return_smooth I32 [1] 0 = false, 1 = true (default 0).
ref_duration I32 [1] Reference duration in seconds (default 5).

Outputs:

Name Dtype Shape Description
audio F32 [S] Mono waveform at 48 kHz.
sample_rate I32 [1] 48000.