How to get a part(one point) of one segment?

Support YOLO
1

i use yolo11 seg, with my dataset, after trainning, model can get tongue mask. but one tip of tongue, it’s not good. how to improve it?

Thank you!

2

Hello there! :wave:

Great to hear that you’ve successfully trained a YOLO11 segmentation model to detect tongue masks. If you are noticing issues with specific areas, such as the tip of the tongue, here are a few tips to help improve your model’s performance:

  1. Enhance Your Dataset:
    Ensure that your dataset has sufficient and high-quality annotations for the tongue and specifically for the tip. You might want to add more images of tongues with diverse shapes, angles, and lighting conditions to give the model more examples of the specific features. Augmenting the dataset with techniques like flipping, scaling, and rotation can also help.

  2. Fine-Tune Annotations:
    If the tip of the tongue is not labeled accurately in your dataset, the model might struggle to learn this feature. Using tools like Label Studio for precise segmentation or validating your existing annotations could help improve outcomes.

  3. Adjust Image Size (imgsz):
    Use a higher resolution for training and prediction if the details like the tip of the tongue are small in the image. Increasing imgsz (e.g., imgs=1024) can sometimes capture finer details better.

    yolo segment train data=path/to/your_dataset.yaml model=yolo11n-seg.pt epochs=100 imgsz=1024

  4. Consider Tuning Hyperparameters:
    Adjusting hyperparameters like learning rate and batch size can yield better convergence. Profile your training setup (use smaller learning rates, if needed) and test various values to see significant changes.

  5. Inspect Predicted Masks Algorithmically:
    Post-process the predicted segmentation masks to refine them further. For instance, you can use morphological operations like dilation or erosion with OpenCV to clean up your masks, particularly for portions like the tongue tip.

    Example:

    import cv2

# Apply morphological transformations to enhance the tip region
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
refined_mask = cv2.dilate(predicted_mask, kernel, iterations=1)

  6. Increase Training Epochs:
    If your model is still underfitting, running the training for more epochs might allow it to learn finer details like the tongue tip better.

  7. Inspect Validation Metrics:
    Monitor your segmentation-specific metrics like map for masks during training and validation. Tools like the confusion matrix can help analyze whether your model accurately identifies smaller features. See this example for validation and interpreting metrics:

    metrics = model.val(split="val")
print("Mean Average Precision for masks:", metrics.seg.map)

  8. Isolate and Focus on the Tip:
    If the tip of the tongue is especially important, you could create a separate dataset or model pipeline to focus specifically on it. Alternatively, adding an additional class to differentiate the tip from the rest of the tongue could help.

For a deeper look into segmentation and model improvement techniques, you may find the following documentation helpful:

If these suggestions don’t resolve the issue, feel free to share more specifics (e.g., an example of the problematic predictions or training logs). We’d be happy to help further. Good luck! :rocket:

3

The segmentation data is returned in a list of [N, 2] shaped arrays. So if you know that your given point is, as an example, n=5 then you can use:

import torch
from ultralytics import YOLO, ASSETS

im = ASSETS / "bus.jpg"
model = YOLO("yolo11n-seg.pt")

result = model.predict(im)
segments: list[torch.Tensor] = result[0].masks.xy

point_to_isolate = 5

for mask in masks:
    # NOTE this will run for all masks 
    # depends on your use case if you want this
    point = mask[point_to_isolate]  # torch tensor (x, y)
    # ...add your logic