diff --git a/examples/gemma-27b-axolotl/training/README.md b/examples/gemma-27b-axolotl/training/README.md index 30fef07..19d973c 100644 --- a/examples/gemma-27b-axolotl/training/README.md +++ b/examples/gemma-27b-axolotl/training/README.md @@ -1,14 +1,28 @@ -## Run instructions +# Gemma 27B Fine-Tuning with Axolotl -### Update config.yaml -- If pushing your checkpoints to Huggingface, make sure to update the `hub_model_id` in `config.yaml`, else comment out the lines with `hub_model_id` and `hub_strategy`. -- If using Weights&Biases to track your run, update `use_wandb`, `wandb_project` and `wandb_entity` accordingly. -- Hyper parameters, dataset processing and other important fields are defined in this yaml. Please go over all fields to make sure they work for your training use-case. +This example fine-tunes Google's Gemma 27B model using the Axolotl framework on Baseten. -### Launch run +**Resources:** 2 nodes, 8x H100 GPUs each (16 GPUs total) -``` +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples gemma-27b-axolotl +cd gemma-27b-axolotl truss train push config.py ``` -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. \ No newline at end of file +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/glm-4-6-msswift/training/README.md b/examples/glm-4-6-msswift/training/README.md new file mode 100644 index 0000000..bf978c2 --- /dev/null +++ b/examples/glm-4-6-msswift/training/README.md @@ -0,0 +1,28 @@ +# GLM-4-6 Fine-Tuning with MS-Swift + +This example fine-tunes the GLM-4-6 model using the MS-Swift framework with MegatronLM on Baseten. + +**Resources:** 2 nodes, 8x H200 GPUs each (16 GPUs total) + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples glm-4-6-msswift +cd glm-4-6-msswift +truss train push config.py +``` + +> **Note:** This example requires H200 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/glm-4.7-flash-msswift/training/README.md b/examples/glm-4.7-flash-msswift/training/README.md new file mode 100644 index 0000000..4eaa4b9 --- /dev/null +++ b/examples/glm-4.7-flash-msswift/training/README.md @@ -0,0 +1,28 @@ +# GLM-4.7 Flash Fine-Tuning with MS-Swift + +This example fine-tunes the GLM-4.7 Flash model using the MS-Swift framework with MegatronLM on Baseten. + +**Resources:** 1 node, 8x H100 GPUs + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples glm-4.7-flash-msswift +cd glm-4.7-flash-msswift +truss train push config.py +``` + +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/glm-4.7-msswift/training/README.md b/examples/glm-4.7-msswift/training/README.md new file mode 100644 index 0000000..7e23588 --- /dev/null +++ b/examples/glm-4.7-msswift/training/README.md @@ -0,0 +1,28 @@ +# GLM-4.7 Fine-Tuning with MS-Swift + +This example fine-tunes the GLM-4.7 model using the MS-Swift framework with MegatronLM on Baseten. + +**Resources:** 2 nodes, 8x H200 GPUs each (16 GPUs total) + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples glm-4.7-msswift +cd glm-4.7-msswift +truss train push config.py +``` + +> **Note:** This example requires H200 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/llama-8b-lora-unsloth/training/README.md b/examples/llama-8b-lora-unsloth/training/README.md index c8b531e..9797d27 100644 --- a/examples/llama-8b-lora-unsloth/training/README.md +++ b/examples/llama-8b-lora-unsloth/training/README.md @@ -1,31 +1,26 @@ -# Llama 3.1 8B LoRA Training with Unsloth +# Llama 3.1 8B LoRA Fine-Tuning with Unsloth -This example demonstrates how to fine-tune [Llama 3.1 8B](https://huggingface.co/Meta-Llama-3.1-8B-Instruct) using LoRA (Low-Rank Adaptation) with [Unsloth](https://github.com/unslothai/unsloth), which provides significant speedups for training. +This example fine-tunes Meta's Llama 3.1 8B Instruct model using LoRA with the Unsloth framework on Baseten. -## Running the example +**Resources:** 1 node, 1x H100 GPU -### Install `truss` -Use the appropriate command for your package manager -```bash -# pip -pip install -U truss -# uv -uv add truss && uv sync --upgrade-package truss -``` - -### Create the workspace for your training project +## Prerequisites -```bash -# for the unsloth example -truss train init --examples llama-8b-lora-unsloth && cd llama-8b-lora-unsloth -``` +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` -### Kick off the job +## Getting Started -Make sure you've plugged in proper secrets (e.g. wandb api key, huggingface token) via Baseten Secrets and Environment Variables, and kick off your job +Initialize the example, navigate into the directory, and push the training job: ```bash +truss train init --examples llama-8b-lora-unsloth +cd llama-8b-lora-unsloth truss train push config.py ``` - -For more details, take a look at the [docs](https://docs.baseten.co/training/overview) diff --git a/examples/mnist-pytorch/training/README.md b/examples/mnist-pytorch/training/README.md new file mode 100644 index 0000000..895f28d --- /dev/null +++ b/examples/mnist-pytorch/training/README.md @@ -0,0 +1,26 @@ +# MNIST Digit Classifier with PyTorch + +This example trains an MNIST digit classifier using PyTorch on Baseten. + +**Resources:** CPU only (4 vCPU, 16Gi memory) + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples mnist-pytorch +cd mnist-pytorch +truss train push config.py +``` diff --git a/examples/nemotron3-30b-torchtitan/README.md b/examples/nemotron3-30b-torchtitan/README.md deleted file mode 100644 index 9a85c5d..0000000 --- a/examples/nemotron3-30b-torchtitan/README.md +++ /dev/null @@ -1,21 +0,0 @@ -# Nemotron-3 Nano 30B Training with TorchTitan - -This example demonstrates how to fine-tune NVIDIA's Nemotron-3 Nano 30B model using TorchTitan. - -## Serving the Model - -To serve this model after training: - -1. Generate the deployment config using the dry-run flag: - ```bash - truss train deploy_checkpoints --job-id= --dry-run - ``` - -2. Take the generated config and add the `--trust-remote-code` flag to the vLLM serve command. - -3. Deploy using `truss push`: - ```bash - truss push - ``` - -**Note:** Deploying from the UI does not currently work for this model. We are adding support for this soon. diff --git a/examples/nemotron3-30b-torchtitan/training/README.md b/examples/nemotron3-30b-torchtitan/training/README.md new file mode 100644 index 0000000..53cf69d --- /dev/null +++ b/examples/nemotron3-30b-torchtitan/training/README.md @@ -0,0 +1,46 @@ +# Nemotron-3 Nano 30B Training with TorchTitan + +This example demonstrates how to fine-tune NVIDIA's Nemotron-3 Nano 30B model using TorchTitan on Baseten. + +**Resources:** 1 node, 8x H200 GPUs + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples nemotron3-30b-torchtitan +cd nemotron3-30b-torchtitan +truss train push config.py +``` + +> **Note:** This example requires H200 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. + +## Serving the Model + +To serve this model after training: + +1. Generate the deployment config using the dry-run flag: + ```bash + truss train deploy_checkpoints --job-id= --dry-run + ``` + +2. Take the generated config and add the `--trust-remote-code` flag to the vLLM serve command. + +3. Deploy using `truss push`: + ```bash + truss push + ``` + +**Note:** Deploying from the UI does not currently work for this model. We are adding support for this soon. diff --git a/examples/orpheus-transformers/training/README.md b/examples/orpheus-transformers/training/README.md index d84b8fb..e7a31da 100644 --- a/examples/orpheus-transformers/training/README.md +++ b/examples/orpheus-transformers/training/README.md @@ -1,16 +1,26 @@ -# Orpheus + Transformers library -This example demonstrates how to train [Orpheus](https://github.com/canopyai/Orpheus-TTS) on a specific voice dataset by training a [LoRA module](https://www.ibm.com/think/topics/lora). This allows you to customize the powerful Text-To-Speech (TTS) model to use accents and tones that fit your use case best. +# Orpheus Fine-Tuning with Transformers -## Run instructions +This example fine-tunes the Orpheus audio model using the HuggingFace Transformers framework on Baseten. -### Update config.py -- Make sure the environment variables in `config.py` is updated with the same name of secrets as saved in Baseten secrets. -- `train.py` has the data processing calls, training code and hyper parameters defined. Update datasets, flags etc here or pass them as arguments through `run.sh` to change them. +**Resources:** 1 node, 1x H100 GPU -### Launch run +## Prerequisites -``` +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples orpheus-transformers +cd orpheus-transformers truss train push config.py ``` - -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. diff --git a/examples/oss-gpt-120b-axolotl/training/README.md b/examples/oss-gpt-120b-axolotl/training/README.md index 30fef07..aeba3c4 100644 --- a/examples/oss-gpt-120b-axolotl/training/README.md +++ b/examples/oss-gpt-120b-axolotl/training/README.md @@ -1,14 +1,28 @@ -## Run instructions +# OSS GPT 120B Fine-Tuning with Axolotl -### Update config.yaml -- If pushing your checkpoints to Huggingface, make sure to update the `hub_model_id` in `config.yaml`, else comment out the lines with `hub_model_id` and `hub_strategy`. -- If using Weights&Biases to track your run, update `use_wandb`, `wandb_project` and `wandb_entity` accordingly. -- Hyper parameters, dataset processing and other important fields are defined in this yaml. Please go over all fields to make sure they work for your training use-case. +This example fine-tunes the OSS GPT 120B model using the Axolotl framework on Baseten. -### Launch run +**Resources:** 2 nodes, 8x H100 GPUs each (16 GPUs total) -``` +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples oss-gpt-120b-axolotl +cd oss-gpt-120b-axolotl truss train push config.py ``` -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. \ No newline at end of file +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/oss-gpt-20b-axolotl/training/README.md b/examples/oss-gpt-20b-axolotl/training/README.md index 30fef07..1b31a95 100644 --- a/examples/oss-gpt-20b-axolotl/training/README.md +++ b/examples/oss-gpt-20b-axolotl/training/README.md @@ -1,14 +1,26 @@ -## Run instructions +# OSS GPT 20B Fine-Tuning with Axolotl -### Update config.yaml -- If pushing your checkpoints to Huggingface, make sure to update the `hub_model_id` in `config.yaml`, else comment out the lines with `hub_model_id` and `hub_strategy`. -- If using Weights&Biases to track your run, update `use_wandb`, `wandb_project` and `wandb_entity` accordingly. -- Hyper parameters, dataset processing and other important fields are defined in this yaml. Please go over all fields to make sure they work for your training use-case. +This example fine-tunes the OSS GPT 20B model using the Axolotl framework on Baseten. -### Launch run +**Resources:** 1 node, 4x H100 GPUs -``` +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples oss-gpt-20b-axolotl +cd oss-gpt-20b-axolotl truss train push config.py ``` - -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. \ No newline at end of file diff --git a/examples/oss-gpt-20b-fft-trl/training/README.md b/examples/oss-gpt-20b-fft-trl/training/README.md new file mode 100644 index 0000000..9296925 --- /dev/null +++ b/examples/oss-gpt-20b-fft-trl/training/README.md @@ -0,0 +1,28 @@ +# OSS GPT 20B Full Fine-Tuning with TRL + +This example performs full fine-tuning of the OSS GPT 20B model using the TRL framework on Baseten. + +**Resources:** 1 node, 8x H200 GPUs + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples oss-gpt-20b-fft-trl +cd oss-gpt-20b-fft-trl +truss train push config.py +``` + +> **Note:** This example requires H200 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/oss-gpt-20b-lora-trl/training/README.md b/examples/oss-gpt-20b-lora-trl/training/README.md index ae2fd26..7f236a4 100644 --- a/examples/oss-gpt-20b-lora-trl/training/README.md +++ b/examples/oss-gpt-20b-lora-trl/training/README.md @@ -1,31 +1,26 @@ -# GPT-OSS 20B LoRA Training with TRL +# OSS GPT 20B LoRA Fine-Tuning with TRL -This example demonstrates how to fine-tune [GPT-OSS 20B](https://huggingface.co/openai/gpt-oss-20b) using LoRA (Low-Rank Adaptation) with [TRL](https://github.com/huggingface/trl) and [PEFT](https://github.com/huggingface/peft). +This example fine-tunes the OSS GPT 20B model using LoRA with the TRL framework on Baseten. -## Running the example +**Resources:** 1 node, 2x H100 GPUs -### Install `truss` -Use the appropriate command for your package manager -```bash -# pip -pip install -U truss -# uv -uv add truss && uv sync --upgrade-package truss -``` - -### Create the workspace for your training project +## Prerequisites -```bash -# for the oss-gpt lora example -truss train init --examples oss-gpt-20b-lora-trl && cd oss-gpt-20b-lora-trl -``` +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` -### Kick off the job +## Getting Started -Make sure you've plugged in proper secrets (e.g. wandb api key, huggingface token) via Baseten Secrets and Environment Variables, and kick off your job +Initialize the example, navigate into the directory, and push the training job: ```bash +truss train init --examples oss-gpt-20b-lora-trl +cd oss-gpt-20b-lora-trl truss train push config.py ``` - -For more details, take a look at the [docs](https://docs.baseten.co/training/overview) diff --git a/examples/qwen3-0.6b-axolotl/training/README.md b/examples/qwen3-0.6b-axolotl/training/README.md index 30fef07..7fddf76 100644 --- a/examples/qwen3-0.6b-axolotl/training/README.md +++ b/examples/qwen3-0.6b-axolotl/training/README.md @@ -1,14 +1,26 @@ -## Run instructions +# Qwen3 0.6B Fine-Tuning with Axolotl -### Update config.yaml -- If pushing your checkpoints to Huggingface, make sure to update the `hub_model_id` in `config.yaml`, else comment out the lines with `hub_model_id` and `hub_strategy`. -- If using Weights&Biases to track your run, update `use_wandb`, `wandb_project` and `wandb_entity` accordingly. -- Hyper parameters, dataset processing and other important fields are defined in this yaml. Please go over all fields to make sure they work for your training use-case. +This example fine-tunes the Qwen3 0.6B model using the Axolotl framework on Baseten. -### Launch run +**Resources:** 1 node, 1x H100 GPU -``` +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-0.6b-axolotl +cd qwen3-0.6b-axolotl truss train push config.py ``` - -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. \ No newline at end of file diff --git a/examples/qwen3-0.6b-axolotl/training/config.py b/examples/qwen3-0.6b-axolotl/training/config.py index c202cfc..b668b6c 100644 --- a/examples/qwen3-0.6b-axolotl/training/config.py +++ b/examples/qwen3-0.6b-axolotl/training/config.py @@ -5,7 +5,7 @@ project_name = "demo/qwen3-0.6b" # 1. Define a base image for your training job -BASE_IMAGE = "axolotlai/axolotl:main-20250811-py3.11-cu126-2.7.1" +BASE_IMAGE = "pytorch/pytorch:2.7.0-cuda12.8-cudnn9-runtime" # 2. Define the Runtime Environment for the Training Job # This includes start commands and environment variables @@ -16,7 +16,7 @@ training_runtime = definitions.Runtime( start_commands=[ - f"axolotl fetch deepspeed_configs && torchrun --nproc-per-node={NUM_GPUS} train.py", + "/bin/sh -c 'chmod +x ./run.sh && ./run.sh'", ], environment_variables={ # Secrets (ensure these are configured in your Baseten workspace) @@ -35,7 +35,7 @@ training_compute = definitions.Compute( node_count=1, accelerator=truss_config.AcceleratorSpec( - accelerator=truss_config.Accelerator.H100, + accelerator=truss_config.Accelerator.H200, count=1, ), ) diff --git a/examples/qwen3-0.6b-axolotl/training/run.sh b/examples/qwen3-0.6b-axolotl/training/run.sh new file mode 100644 index 0000000..6ec7c07 --- /dev/null +++ b/examples/qwen3-0.6b-axolotl/training/run.sh @@ -0,0 +1,6 @@ +#!/bin/bash +set -eux + +pip install trl==0.29.0 peft==0.18.1 datasets==4.6.0 + +python train.py diff --git a/examples/qwen3-0.6b-axolotl/training/train.py b/examples/qwen3-0.6b-axolotl/training/train.py index a718b50..2788327 100644 --- a/examples/qwen3-0.6b-axolotl/training/train.py +++ b/examples/qwen3-0.6b-axolotl/training/train.py @@ -1,82 +1,79 @@ -from pathlib import Path import os - -from axolotl.utils.dict import DictDefault -from axolotl.cli.config import load_cfg -from axolotl.common.datasets import load_datasets -from axolotl.train import train +import torch +from datasets import load_dataset +from transformers import AutoModelForCausalLM, AutoTokenizer +from peft import LoraConfig +from trl import SFTTrainer, SFTConfig OUTPUT_DIR = os.environ.get("BT_CHECKPOINT_DIR", "outputs/qwen3-0.6b") +MODEL_NAME = "Qwen/Qwen3-0.6B" -def main(): - config = DictDefault( - adapter="qlora", - base_model="Qwen/Qwen3-0.6B", - bf16=True, - # chat_template="tokenizer_default_fallback_chatml", - # Data loader tweaks - dataloader_num_workers=2, - dataloader_pin_memory=True, - dataloader_prefetch_factor=8, - - datasets=[ - { - "path": "winglian/pirate-ultrachat-10k", - "type": "chat_template", - "field_messages": "messages", - } - ], +def main(): + # Load model — 0.6B fits easily on H200 without quantization + print("Loading model...") + model = AutoModelForCausalLM.from_pretrained( + MODEL_NAME, + torch_dtype=torch.bfloat16, + device_map="auto", + use_cache=False, + ) - # Eval/val - val_set_size=0.05, - eval_steps=10, + # Load tokenizer + print("Loading tokenizer...") + tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) + if tokenizer.pad_token is None: + tokenizer.pad_token = tokenizer.eos_token - # LoRA / QLoRA - load_in_4bit=True, + # LoRA config + lora_config = LoraConfig( + r=64, lora_alpha=32, - lora_r=64, - lora_mlp_kernel=True, - lora_target_modules="all-linear", + target_modules="all-linear", + lora_dropout=0.05, + bias="none", + task_type="CAUSAL_LM", + ) - # Optim & schedule - optimizer="adamw_torch", - learning_rate=1e-3, - lr_scheduler="cosine", - warmup_steps=5, - max_grad_norm=0.1, + # Load dataset + print("Loading dataset...") + dataset = load_dataset("winglian/pirate-ultrachat-10k", split="train") - # Training loop - micro_batch_size=1, + # SFT config + training_args = SFTConfig( + output_dir=OUTPUT_DIR, + max_steps=20, + save_steps=5, + per_device_train_batch_size=1, gradient_accumulation_steps=1, gradient_checkpointing=True, - gradient_checkpointing_kwargs={"use_reentrant": False}, - num_epochs=1, - max_steps=20, - sequence_len=2048, - - # Misc - embeddings_skip_upcast=True, + learning_rate=1e-3, + lr_scheduler_type="cosine", + warmup_steps=5, + bf16=True, logging_steps=1, - output_dir=OUTPUT_DIR, - saves_per_epoch=2, - sample_packing=True, - attn_implementation="flash_attention_2", - - # Plugins - plugins=[ - "axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin", - ], + max_length=2048, + packing=True, + report_to="none", + ) - # DeepSpeed (path can be relative) - deepspeed=str(Path("deepspeed_configs/zero1.json")), + # Train + print("Starting training...") + trainer = SFTTrainer( + model=model, + args=training_args, + train_dataset=dataset, + peft_config=lora_config, + processing_class=tokenizer, ) - cfg = load_cfg(config) + trainer.train() - dataset_meta = load_datasets(cfg=cfg) + print("Saving final model...") + trainer.save_model(os.path.join(OUTPUT_DIR, "final_model")) + tokenizer.save_pretrained(os.path.join(OUTPUT_DIR, "final_model")) + print("Training complete!") - model, tokenizer, trainer = train(cfg=cfg, dataset_meta=dataset_meta) if __name__ == "__main__": main() diff --git a/examples/qwen3-0.6b-classifier-hf-trainer/training/README.md b/examples/qwen3-0.6b-classifier-hf-trainer/training/README.md index 1fb7358..3244685 100644 --- a/examples/qwen3-0.6b-classifier-hf-trainer/training/README.md +++ b/examples/qwen3-0.6b-classifier-hf-trainer/training/README.md @@ -1,28 +1,26 @@ -## Qwen3-0.6B Sequence Classification with HuggingFace Trainer +# Qwen3 0.6B Classifier with HuggingFace Trainer -This example demonstrates how to fine-tune Qwen3-0.6B for binary text classification (IMDB sentiment analysis) using HuggingFace's `AutoModelForSequenceClassification` and `Trainer` API. +This example trains a classifier based on the Qwen3 0.6B model using the HuggingFace Trainer on Baseten. -### Key Features +**Resources:** 1 node, 1x H100 GPU -- Uses `AutoModelForSequenceClassification` instead of custom model architecture -- Leverages HuggingFace `Trainer` for simplified training loop -- Automatic handling of data collation, evaluation, and checkpointing -- Built-in metrics computation and evaluation +## Prerequisites -### Differences from the PyTorch version +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` -- **Model**: Uses `AutoModelForSequenceClassification` which automatically adds a classification head -- **Training**: Uses HuggingFace `Trainer` instead of manual PyTorch training loop -- **Data Handling**: Uses `DataCollatorWithPadding` for efficient batching -- **Evaluation**: Built-in evaluation with automatic metric computation +## Getting Started -## Run instructions +Initialize the example, navigate into the directory, and push the training job: -### Launch run - -``` +```bash +truss train init --examples qwen3-0.6b-classifier-hf-trainer +cd qwen3-0.6b-classifier-hf-trainer truss train push config.py ``` - -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. - diff --git a/examples/qwen3-0.6b-classifier-pytorch/training/README.md b/examples/qwen3-0.6b-classifier-pytorch/training/README.md index c1cde16..336f57e 100644 --- a/examples/qwen3-0.6b-classifier-pytorch/training/README.md +++ b/examples/qwen3-0.6b-classifier-pytorch/training/README.md @@ -1,8 +1,26 @@ -## Run instructions -### Launch run +# Qwen3 0.6B Classifier with PyTorch -``` +This example trains a classifier based on the Qwen3 0.6B model using PyTorch on Baseten. + +**Resources:** 1 node, 1x H100 GPU + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-0.6b-classifier-pytorch +cd qwen3-0.6b-classifier-pytorch truss train push config.py ``` - -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. diff --git a/examples/qwen3-0.6b-pytorch/training/README.md b/examples/qwen3-0.6b-pytorch/training/README.md index c1cde16..3278cb9 100644 --- a/examples/qwen3-0.6b-pytorch/training/README.md +++ b/examples/qwen3-0.6b-pytorch/training/README.md @@ -1,8 +1,26 @@ -## Run instructions -### Launch run +# Qwen3 0.6B Fine-Tuning with PyTorch -``` +This example fine-tunes the Qwen3 0.6B model using PyTorch on Baseten. + +**Resources:** 1 node, 1x H100 GPU + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-0.6b-pytorch +cd qwen3-0.6b-pytorch truss train push config.py ``` - -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. diff --git a/examples/qwen3-235b-mswift/training/README.md b/examples/qwen3-235b-mswift/training/README.md new file mode 100644 index 0000000..efb21e7 --- /dev/null +++ b/examples/qwen3-235b-mswift/training/README.md @@ -0,0 +1,28 @@ +# Qwen3 235B LoRA Fine-Tuning with MS-Swift + +This example fine-tunes the Qwen3 235B model using LoRA with the MS-Swift framework and MegatronLM on Baseten. + +**Resources:** 2 nodes, 8x H200 GPUs each (16 GPUs total) + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-235b-mswift +cd qwen3-235b-mswift +truss train push config.py +``` + +> **Note:** This example requires H200 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/qwen3-235b-mswift/config.py b/examples/qwen3-235b-mswift/training/config.py similarity index 100% rename from examples/qwen3-235b-mswift/config.py rename to examples/qwen3-235b-mswift/training/config.py diff --git a/examples/qwen3-235b-mswift/run.sh b/examples/qwen3-235b-mswift/training/run.sh similarity index 100% rename from examples/qwen3-235b-mswift/run.sh rename to examples/qwen3-235b-mswift/training/run.sh diff --git a/examples/qwen3-30b-mswift-multinode/training/README.md b/examples/qwen3-30b-mswift-multinode/training/README.md new file mode 100644 index 0000000..5b92fe8 --- /dev/null +++ b/examples/qwen3-30b-mswift-multinode/training/README.md @@ -0,0 +1,28 @@ +# Qwen3 30B Fine-Tuning with MS-Swift (Multi-Node) + +This example fine-tunes the Qwen3 30B model using the MS-Swift framework with MegatronLM across multiple nodes on Baseten. + +**Resources:** 2 nodes, 8x H100 GPUs each (16 GPUs total) + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-30b-mswift-multinode +cd qwen3-30b-mswift-multinode +truss train push config.py +``` + +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/qwen3-30b-mswift-singlenode/training/README.md b/examples/qwen3-30b-mswift-singlenode/training/README.md index ff4527b..d4cb808 100644 --- a/examples/qwen3-30b-mswift-singlenode/training/README.md +++ b/examples/qwen3-30b-mswift-singlenode/training/README.md @@ -1,22 +1,28 @@ -## Run instructions +# Qwen3 30B Fine-Tuning with MS-Swift (Single Node) -### Update config.py -Update `config.py` with environment variables to match your Baseten secrets. +This example fine-tunes the Qwen3 30B model using the MS-Swift framework with MegatronLM on a single node on Baseten. -### Update run_1node.sh -- If using Weights&Biases to track your run, update `wandb_project` and `wandb_exp_name` accordingly, or comment out these lines. -- Hyper parameters, dataset processing and other important fields are defined in this script. Please go over all fields to make sure they work for your training use-case. +**Resources:** 1 node, 8x H100 GPUs -### Launch run +## Prerequisites -``` -truss train push config.py -``` +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. +## Getting Started -### Accessing checkpoints -To get the location of your checkpoints, run `truss train get_checkpoint_urls --job-id your-job-id`, this downloads a file containing information about your checkpoint storage locations. +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-30b-mswift-singlenode +cd qwen3-30b-mswift-singlenode +truss train push config.py +``` -### Troubleshooting CUDA OOM Errors -CUDA out-of-memory (OOM) errors are often buried by Megatron and may manifest as NCCL errors instead. If you encounter errors like `RuntimeError: NCCL Error 1: unhandled cuda error … group.reduce_scatter_tensor_coalesced(outputs, inputs, reduce_opts)`, this typically indicates that your GPU memory usage is too high. In such cases, you should tweak some of the launch commands in `run_1node.sh` to reduce memory consumption (e.g., adjust batch size, sequence length, or model parallelism settings). \ No newline at end of file +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/qwen3-32b-fft-verl-multinode/training/README.md b/examples/qwen3-32b-fft-verl-multinode/training/README.md new file mode 100644 index 0000000..7a31cd1 --- /dev/null +++ b/examples/qwen3-32b-fft-verl-multinode/training/README.md @@ -0,0 +1,28 @@ +# Qwen3 32B Full Fine-Tuning with VeRL (Multi-Node) + +This example performs full fine-tuning of the Qwen3 32B model using the VeRL (Verifiable Reinforcement Learning) framework across multiple nodes on Baseten. + +**Resources:** 2 nodes, 8x H200 GPUs each (16 GPUs total) + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-32b-fft-verl-multinode +cd qwen3-32b-fft-verl-multinode +truss train push config.py +``` + +> **Note:** This example requires H200 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/qwen3-8b-fft-verl/training/README.md b/examples/qwen3-8b-fft-verl/training/README.md new file mode 100644 index 0000000..a4ca74a --- /dev/null +++ b/examples/qwen3-8b-fft-verl/training/README.md @@ -0,0 +1,28 @@ +# Qwen3 8B Full Fine-Tuning with VeRL + +This example performs full fine-tuning of the Qwen3 8B model using the VeRL (Verifiable Reinforcement Learning) framework on Baseten. + +**Resources:** 1 node, 8x H100 GPUs + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-8b-fft-verl +cd qwen3-8b-fft-verl +truss train push config.py +``` + +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/qwen3-8b-lora-dpo-trl/training/README.md b/examples/qwen3-8b-lora-dpo-trl/training/README.md new file mode 100644 index 0000000..9cc3853 --- /dev/null +++ b/examples/qwen3-8b-lora-dpo-trl/training/README.md @@ -0,0 +1,26 @@ +# Qwen3 8B LoRA DPO Fine-Tuning with TRL + +This example fine-tunes the Qwen3 8B model using LoRA with Direct Preference Optimization (DPO) via the TRL framework on Baseten. + +**Resources:** 1 node, 2x H100 GPUs + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-8b-lora-dpo-trl +cd qwen3-8b-lora-dpo-trl +truss train push config.py +``` diff --git a/examples/qwen3-8b-lora-verl/training/README.md b/examples/qwen3-8b-lora-verl/training/README.md new file mode 100644 index 0000000..03de819 --- /dev/null +++ b/examples/qwen3-8b-lora-verl/training/README.md @@ -0,0 +1,28 @@ +# Qwen3 8B LoRA Fine-Tuning with VeRL + +This example fine-tunes the Qwen3 8B model using LoRA with the VeRL (Verifiable Reinforcement Learning) framework on Baseten. + +**Resources:** 1 node, 8x H100 GPUs + +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples qwen3-8b-lora-verl +cd qwen3-8b-lora-verl +truss train push config.py +``` + +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job. diff --git a/examples/whisper-transformers/training/README.md b/examples/whisper-transformers/training/README.md index 098a667..d12f9d9 100644 --- a/examples/whisper-transformers/training/README.md +++ b/examples/whisper-transformers/training/README.md @@ -1,14 +1,28 @@ -## Run instructions +# Whisper Fine-Tuning with Transformers -### Update config.py -- Make sure the environment variables in `config.py` is updated with the same name of secrets as saved in Baseten secrets. -- In `train.py`, where all the data processing and training code is present, make sure the data processing function `load_and_process_common_accent_dataset` or `load_and_process_dataset` processes your dataset as desired. This file has 2 examples - `load_and_process_common_accent_dataset` or `load_and_process_dataset` - - This is also where hyper parameters for training are defined. Refer to `run.sh` to change them. +This example fine-tunes OpenAI's Whisper speech recognition model using the HuggingFace Transformers framework on Baseten. -### Launch run +**Resources:** 1 node, 8x H100 GPUs -``` +## Prerequisites + +1. [Create a Baseten account](https://baseten.co/signup) if you don't already have one. +2. Install the Truss CLI: + ```bash + # pip + pip install -U truss + # or uv + uv add truss + ``` + +## Getting Started + +Initialize the example, navigate into the directory, and push the training job: + +```bash +truss train init --examples whisper-transformers +cd whisper-transformers truss train push config.py ``` -Upon successful submission, the CLI will output helpful information about your job, including the job-id to track your run. \ No newline at end of file +> **Note:** This example requires more than 4x H100 GPUs. You may need to [contact Baseten](https://www.baseten.co/contact) to get approval for this instance type before running the job.