Skip to content

Feat: add item_transform_random_erasing for detection and segmentation items - #389

Open
DerrickUnleashed wants to merge 8 commits into
mlverse:mainfrom
DerrickUnleashed:feat/randomEraseTransform
Open

Feat: add item_transform_random_erasing for detection and segmentation items#389
DerrickUnleashed wants to merge 8 commits into
mlverse:mainfrom
DerrickUnleashed:feat/randomEraseTransform

Conversation

@DerrickUnleashed

@DerrickUnleashed DerrickUnleashed commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Adds item_transform_random_erasing() for randomly erasing a rectangular region of a dataset item, covering object detection items/datasets, semantic segmentation items/datasets, and rotated-box items.

Currently only the image pixels are erased; boxes and masks are left unchanged, matching PyTorch's documented RandomErasing behaviour.

Question: Should we switch to dropping bounding boxes whose centre falls inside the erased region?

Closes #358

url <- "https://upload.wikimedia.org/wikipedia/commons/b/b6/Felis_catus-cat_on_snow.jpg"

# ========== 1. DETECTION ITEM ==========
img <- base_loader(url) |> transform_to_tensor()
boxes <- torch_tensor(matrix(c(600, 200, 2880, 1860), ncol = 4), dtype = torch_float32())
det_item <- list(x = img, y = list(boxes = boxes, labels = "CAT"))
class(det_item) <- c("image_with_bounding_box", "list")

det_erased <- item_transform_random_erasing(det_item)

p1 <- draw_bounding_boxes(det_item, colors = "blue", width = 10)$to(torch_float())$div(255)
p2 <- draw_bounding_boxes(det_erased, colors = "red", width = 10)$to(torch_float())$div(255)
grid1 <- vision_make_grid(torch_stack(list(p1, p2)), scale = TRUE)
tensor_image_browse(grid1)

# ========== 2. DETECTION DATASET ==========
ds_det <- pascal_detection_dataset(year = "2007", split = "trainval",
                                   transform = transform_to_tensor, download = TRUE)
orig_det <- ds_det[1]
ds_det_erased <- item_transform_random_erasing(ds_det)
erase_det <- ds_det_erased[1]

p3 <- draw_bounding_boxes(orig_det, colors = "blue", width = 5)$to(torch_float())$div(255)
p4 <- draw_bounding_boxes(erase_det, colors = "red", width = 5)$to(torch_float())$div(255)
grid2 <- vision_make_grid(torch_stack(list(p3, p4)), scale = TRUE)
tensor_image_browse(grid2)

# ========== 3. SEGMENTATION ITEM ==========
img2 <- base_loader(url) |> transform_to_tensor()
h <- img2$shape[2]; w <- img2$shape[3]

mask1 <- torch_zeros(h, w, dtype = torch_bool()); mask1[, 1:150] <- TRUE
mask2 <- torch_zeros(h, w, dtype = torch_bool()); mask2[, 350:500] <- TRUE
masks <- torch_stack(list(mask1, mask2))
seg_item <- list(x = img2, y = list(masks = masks, labels = torch_tensor(c(1L, 2L)),
                                    image_height = h, image_width = w))
class(seg_item) <- c("image_with_segmentation_mask", "list")

seg_erased <- item_transform_random_erasing(seg_item)

p5 <- draw_segmentation_masks(seg_item, alpha = 0.5, colors = c("red", "blue"))$to(torch_float())$div(255)
p6 <- draw_segmentation_masks(seg_erased, alpha = 0.5, colors = c("red", "blue"))$to(torch_float())$div(255)
grid3 <- vision_make_grid(torch_stack(list(p5, p6)), scale = TRUE)
tensor_image_browse(grid3)

# ========== 4. SEGMENTATION DATASET ==========
ds_seg <- pascal_segmentation_dataset(year = "2007", split = "trainval",
                                      transform = transform_to_tensor, download = TRUE)
orig_seg <- ds_seg[1]
ds_seg_erased <- item_transform_random_erasing(ds_seg)
erase_seg <- ds_seg_erased[1]

p7 <- draw_segmentation_masks(orig_seg, alpha = 0.5)$to(torch_float())$div(255)
p8 <- draw_segmentation_masks(erase_seg, alpha = 0.5)$to(torch_float())$div(255)
grid4 <- vision_make_grid(torch_stack(list(p7, p8)), scale = TRUE)
tensor_image_browse(grid4)

DETECTION ITEM

filefb0972860e6f

DETECTION DATASET

filefb093f1dd426

SEGMENTATION ITEM

filefb093f532b29

SEGMENTATION DATASET

filefb096d36b139

@DerrickUnleashed
DerrickUnleashed marked this pull request as ready for review August 13, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add item_transform_random_erasing

1 participant