Skip to content

Feat: Add item_transform_random_crop for detection and segmentation items - #387

Open
DerrickUnleashed wants to merge 6 commits into
mlverse:mainfrom
DerrickUnleashed:feat/randomCropTransform
Open

Feat: Add item_transform_random_crop for detection and segmentation items#387
DerrickUnleashed wants to merge 6 commits into
mlverse:mainfrom
DerrickUnleashed:feat/randomCropTransform

Conversation

@DerrickUnleashed

@DerrickUnleashed DerrickUnleashed commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Added item_transform_random_crop for detection item, dataset and segmentation item, dataset

Closes #355

@DerrickUnleashed DerrickUnleashed changed the title Feat: Add item_transform_random_crop Feat: Add item_transform_random_crop for detection and segmentation items Aug 11, 2026
@DerrickUnleashed

Copy link
Copy Markdown
Contributor Author
url <- "https://upload.wikimedia.org/wikipedia/commons/b/b6/Felis_catus-cat_on_snow.jpg"

resize_pair <- function(a, b, size = c(600, 600)) {
  list(transform_resize(a, size), transform_resize(b, size))
}

# ========== 1. DETECTION ITEM ==========
img <- base_loader(url) |> transform_to_tensor()
boxes <- torch_tensor(matrix(c(600, 200, 2880, 1860), ncol = 4), dtype = torch_float32())
det_item <- list(x = img, y = list(boxes = boxes, labels = "CAT"))
class(det_item) <- c("image_with_bounding_box", "list")

det_cropped <- item_transform_random_crop(det_item, size = c(800, 1200))

p1 <- draw_bounding_boxes(det_item, colors = "blue", width = 10)$to(torch_float())$div(255)
p2 <- draw_bounding_boxes(det_cropped, colors = "red", width = 10)$to(torch_float())$div(255)
grid1 <- do.call(vision_make_grid, c(torch_stack(resize_pair(p1, p2)), list(scale = TRUE)))
tensor_image_browse(grid1)

# ========== 2. DETECTION DATASET ==========
ds_det <- pascal_detection_dataset(year = "2007", split = "trainval",
                                   transform = transform_to_tensor, download = TRUE)
orig_det <- ds_det[1]
ds_det_cropped <- item_transform_random_crop(ds_det, size = c(250, 350))
crop_det <- ds_det_cropped[1]

p3 <- draw_bounding_boxes(orig_det, colors = "blue", width = 5)$to(torch_float())$div(255)
p4 <- draw_bounding_boxes(crop_det, colors = "red", width = 5)$to(torch_float())$div(255)
grid2 <- do.call(vision_make_grid, c(torch_stack(resize_pair(p3, p4)), list(scale = TRUE)))
tensor_image_browse(grid2)

# ========== 3. SEGMENTATION ITEM ==========
img2 <- base_loader(url) |> transform_to_tensor()
h <- img2$shape[2]; w <- img2$shape[3]

mask1 <- torch_zeros(h, w, dtype = torch_bool()); mask1[, 1:150] <- TRUE
mask2 <- torch_zeros(h, w, dtype = torch_bool()); mask2[, 350:500] <- TRUE
masks <- torch_stack(list(mask1, mask2))
seg_item <- list(x = img2, y = list(masks = masks, labels = torch_tensor(c(1L, 2L)),
                                    image_height = h, image_width = w))
class(seg_item) <- c("image_with_segmentation_mask", "list")

seg_cropped <- item_transform_random_crop(seg_item, size = c(800, 1200))

p5 <- draw_segmentation_masks(seg_item, alpha = 0.5, colors = c("red", "blue"))$to(torch_float())$div(255)
p6 <- draw_segmentation_masks(seg_cropped, alpha = 0.5, colors = c("red", "blue"))$to(torch_float())$div(255)
grid3 <- do.call(vision_make_grid, c(torch_stack(resize_pair(p5, p6)), list(scale = TRUE)))
tensor_image_browse(grid3)

# ========== 4. SEGMENTATION DATASET ==========
ds_seg <- pascal_segmentation_dataset(year = "2007", split = "trainval",
                                      transform = transform_to_tensor, download = TRUE)
orig_seg <- ds_seg[1]
ds_seg_cropped <- item_transform_random_crop(ds_seg, size = c(250, 350))
crop_seg <- ds_seg_cropped[1]

p7 <- draw_segmentation_masks(orig_seg, alpha = 0.5)$to(torch_float())$div(255)
p8 <- draw_segmentation_masks(crop_seg, alpha = 0.5)$to(torch_float())$div(255)
grid4 <- do.call(vision_make_grid, c(torch_stack(resize_pair(p7, p8)), list(scale = TRUE)))
tensor_image_browse(grid4)

DETECTION ITEM

file341456d097e

DETECTION DATASET

file341798a52cd

SEGMENTATION ITEM

file34168424f11

SEGMENTATION DATASET

file34159551a90

@DerrickUnleashed
DerrickUnleashed marked this pull request as ready for review August 11, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add target_transform_random_crop

1 participant