I used instance segmentation to identify a person holding a board and wanted to obtain the outline of the person excluding outsiders. The person was partially obscured by the board and could only see their upper and lower bodies. When using results[0].masks.xy, I found that I could only obtain the coordinates of the person’s lower body outline and did not include their upper body. However, in the result map of results[0].plot(), I found that both the upper and lower body areas of the person were marked in blue. How can I obtain a complete mask of the person’s outline？
If I recall correctly (iirc), the output from results[0].masks.xy is list of Numpy arrays. This would mean that you’d need to extract each array from the list to get the mask. The reason this is a list and not a Numpy array, is because the mask contours might be different lengths, and would not be able to be stacked with mismatch array sizes.
result[0].masks.xy
>>> [
>>> array[i, 2],
>>> array[j, 2],
>>> array[k, 2],
>>> ...,
>>> array[N, 2],
>>> ]
# here, i, j, k represent different values for the first
# array dimension, with up to N different values
One important factor to consider (and possibly answer) is what do you wish to accomplish with the contours? Knowing this might provide better insight on how to accomplish your end goal (see this post for context).
What I want to ask is that when there is an instance of a target (you can think of it as the same ID) being detected, mask.xy only has part of its outer edge outline coordinate points, and the other parts are lost, just like I said it is obviously the head and lower body of the same person, but the split mask.xy only has the lower body of the person, but the head and lower body belong to the same person, this in the plot() result shows that the head and lower body are marked in blue as the same person, why does this difference occur, This problem does not arise when a person is not occluded, that is, complete. In the scenario I encountered, a person was obscured by the board he was holding and was divided into a head and lower body. The mask.xy coordinate only has the coordinates of his lower body, but the plot marks both the head and lower body. It appears that the mask in result.mask is different from the mask used in the plot() function.
The YOLO format doesn’t allow holes in mask. So when it’s converted to segments/polygons, the information is lost. Read this issue. The plotting and metrics use masks and not polygons. You can access the mask using result.masks.data.
1 Like