Yolo Segmentation Model Performance

Discussion
1

I have been working with a custom YOLO segmentation dataset that I annotated and exported from CVAT, but the results from my trained YOLO models (YOLOv8n, YOLO11m, YOLOE, and others) aren’t giving me the accuracy I need for detecting damages. The data was annotated from scratch in CVAT and adding more data hasnt helped in improving the accuracy of the model. Are there any suggestions in which direction I should move so I can get a new model with the best accuracy.

2

Can you post the results.jpg and labels.jpg files from your runs folder?

3

Hey there sorry for the delayed response here is what you asked for

labels
labels1600×1600 214 KB

4

results
results3600×1200 439 KB

The website didn’t let me post 2 images at once, since I am a new user.

5

Your objects are too small, so you would probably need to increase the imgsz to get better accuracy.

Do most images have damage? Or only some of them? The count of damage instance is a lot lower than the majority instance.

6

Had to increase the number of normal instances because I was getting a lot of false positives for damaged, so had to annotate them and use them as data for the new model, and yes the normal instances are more and not all images have damage.

7

Thanks for the details and the screenshots, Kevin. Agree with Toxite: your damage masks look tiny, so higher resolution is key. One more important point: don’t model “normal” as a class in segmentation. Keep “damage” as the only class and include many negative images with zero labels. Using a “normal” mask forces the model to decide between normal vs background vs damage, which often increases false positives and hurts small-mask learning.

Quick steps to try:

  • Re-label to a single class: damage; keep normal-only images as negatives (no masks).
  • Train at higher resolution and reduce mosaic so small masks aren’t minified.
  • If objects are still tiny, tile large images offline and adjust labels; this helps a lot for small objects as described in our guide on image tiling in the Insights page.

Example:

yolo segment train model=yolo11m-seg.pt data=data.yaml imgsz=1280 mosaic=0.1 copy_paste=0 epochs=100

If you need a reference for tiling and fine-tuning ideas, see the guidance on image tiling for small objects in the Ultralytics YOLO model evaluation insights.