Add Visualization utilities article - #397
Conversation
|
Instance segmentation is left out here because Mask R-CNN's masks never cross the 0.5 threshold, so the figures come out blank. Should I work on fixing that? @cregouby |
|
Yes, both Mask R-CNN and Faster R-CNN are very sensitive to score threshold and nms threshold (see examples or each function). |
cregouby
left a comment
cregouby
left a comment
There was a problem hiding this comment.
Praise Very nice vignette, thank you !
todo see inline
| quick way to eyeball a batch coming out of a `dataloader()`. | ||
|
|
||
| ```{r} | ||
| grid <- vision_make_grid(torch_stack(list(dog1, dog2)), num_rows = 2) |
| c(210, 150, 350, 430) | ||
| ), dtype = torch_float()) | ||
|
|
||
| boxed <- draw_bounding_boxes(dog1, boxes, colors = c("red", "yellow"), width = 5) |
There was a problem hiding this comment.
suggestion could we add labels = c("label_1", ...) to visually explain all the features of the function ?
| batch <- torch_stack(list( | ||
| transform_normalize(dog1, norm_mean, norm_std), | ||
| transform_normalize(dog2, norm_mean, norm_std) | ||
| )) |
There was a problem hiding this comment.
suggestion if I'm not wrong, transform_normalize() accepts batches, so I would demonstrate it and I would apply it once after torch_strack()
| draw_detection <- function(image, detection) { | ||
| keep <- as.logical(as.array(detection$scores > 0.5)) | ||
| draw_bounding_boxes( | ||
| image, | ||
| detection$boxes[keep, , drop = FALSE], | ||
| labels = coco_classes(as.integer(detection$labels[keep])), | ||
| colors = "yellow", | ||
| width = 5, | ||
| font_size = 30 | ||
| ) | ||
| } |
There was a problem hiding this comment.
thought I'm not in favor of defining new functions wrapping visualization utilities in an article where the intent is to demonstrate how good the visualization utilities are. That said, I've no better idea to not have a lot of boilerplate code.
suggestion We should at least explain the why of this function, and what each step does.
suggestion a small size, naive visualization of the direct model output (normalized image and unfiltered bbox) would allow a good justification of the why this function
| height and width. | ||
|
|
||
| ```{r} | ||
| resized <- lapply(list(dog1, dog2), transform_resize, size = c(520, 520)) |
There was a problem hiding this comment.
todo simplification clarity transform_resize is also accepting batches. So please demonstrate the piping of transforms after stacking the images.
2 participants
Fix #383.
Adds a Visualization utilities article mirroring PyTorch's tutorial:
vision_make_grid(),draw_bounding_boxes()with RF-DETR,draw_segmentation_masks()with FCN ResNet-50, anddraw_keypoints().Fix #113