Poor training results after tuning yolo11

Discussion
,
1

Hi,

I recently upgraded my model to use YOLO11l instead of YOLOv8n. Using the same training hyperparameters as I obtained from tuning my model on YOLOv8 i trained my YOLO11l model. This led to better results across the board. To get even more improvement I decided to tune the YOLO11l model as well. I performed 150 iterations with 30 epochs per iteration (same as for my YOLOv8 tuning). This resulted in a model with a higher fitness score then i obtained from tuning the YOLOv8. However, when I trained my YOLO11l model with the newly obtained hyperparameters for 300 epochs, it performed significantly worse. Even worse than my untuned YOLOv8n model with default hyperparameters.

Also notably, after only a few epochs I get an NMS time limit warning (see below) during the validation. Something that didn’t happen during the tuning. Perhaps this has something to do with the poor results

Does anyone know why this is happening? And more importantly, why the YOLO11l model trains so poorly with the tuned hyperparameters?

 Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      4/300      10.4G    0.04911      1.975      2.931        160        640: 100% ━━━━━━━━━━━━ 18/18 1.6it/s 11.4s
WARNING NMS time limit 2.800s exceeded
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 20% ━━────────── 1/5 0.1it/s 3.7s<49.2sWARNING NMS time limit 2.800s exceeded
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 40% ━━━━╸─────── 2/5 0.1it/s 7.9s<23.5sWARNING NMS time limit 2.800s exceeded
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 60% ━━━━━━━───── 3/5 0.1it/s 17.5s<16.6sWARNING NMS time limit 2.800s exceeded
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 80% ━━━━━━━━━╸── 4/5 0.1it/s 26.4s<8.5sWARNING NMS time limit 2.500s exceeded
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100% ━━━━━━━━━━━━ 5/5 0.1it/s 43.4s
                   all         74        540    0.00994     0.0199      0.006    0.00194

Hyperparameters for YOLO11l:

# 150/150 iterations complete ✅ (71604.88s)

# Best fitness=0.47402 observed at iteration 70
# Best fitness metrics are {'metrics/precision(B)': 0.70366, 'metrics/recall(B)': 0.67542, 'metrics/mAP50(B)': 0.72477, 'metrics/mAP50-95(B)': 0.47402, 'val/box_loss': 0.02901, 'val/cls_loss': 1.00207, 'val/dfl_loss': 2.45884, 'fitness': 0.47402}
 
lr0: 0.00522
lrf: 0.05804
momentum: 0.75242
weight_decay: 0.00025
warmup_epochs: 2.28673
warmup_momentum: 0.59203
box: 0.1998
cls: 0.52526
hsv_h: 0.01603
hsv_s: 0.77494
hsv_v: 0.39599
degrees: 14.81169
translate: 0.4
scale: 0.6
flipud: 0.0
fliplr: 0.35266
mosaic: 1.0
2

Can you post the output of yolo checks from terminal?