YOLO v8 consider the background as a target, What can I do about that?

Discussion
, ,
1

My dataset contains over 200,000 images. After training, the model achieved mAP@0.5 = 0.81 and mAP@0.5:0.95 = 0.51 with imgsz = 640, and I did not observe any false detections. However, when I switched the inference image size to imgsz = 448, the model started detecting background regions as the target, and the false positives had high confidence（0.76）.

To address this, I tried a simple sample-balancing approach: extracting additional frames from videos, converting them into images, and adding them to the training set. This helped temporarily, but the issue still occurs when the model is applied to new scenes.

Using the same dataset, I tested both YOLOv5 and YOLOv8 with official/default settings: pretrained weights, batch size 256, 4× RTX 4090 (24GB), 300 epochs, and no other parameter changes.

Has anyone encountered a similar issue?

误检1
误检11185×542 229 KB

2

I am not good at English.I’ m a student ,who can help me,thank you very much!

3

there are the yolo v5 training results and parameters

image
image677×753 21.6 KB

4

image
image1075×878 72.2 KB

5

This behavior is almost always an input-resolution mismatch: you trained/validated at imgsz=640, but at inference imgsz=448 the whole image is rescaled differently, so background textures can start to “look like” your target at that scale and the model becomes overconfident (you’ll usually see AP/precision drop if you validate at 448 too).

First, I’d confirm it by running validation at the same inference size (if metrics drop at 448, it’s expected):

yolo val model=best.pt data=your.yaml imgsz=448

If you need imgsz=448 in production, the fix is to train (or fine-tune) for that size or enable multi-scale so the model learns to be robust across sizes. You can also reduce false positives by increasing your confidence threshold, and by adding “hard negative” images from those new scenes (backgrounds that look similar but are not the target), which is the standard approach when precision is low as described in the Ultralytics note on improving precision by adjusting confidence and adding difficult negatives.

Example fine-tune (recommended to try Ultralytics YOLO11):

yolo train model=yolo11s.pt data=your.yaml imgsz=448 epochs=50 multi_scale=True

If you share your results.png (the one with P/R curves) and confirm whether this is single-class or multi-class, I can suggest the best conf starting point and whether this looks more like domain shift vs. label noise.

6

First,thank you very much for your help! Let me describe my previous workflow in detail: I trained a model with imgsz=640, then set mode=export to export an ONNX model with imgsz=448. When I ran inference with the ONNX model, I found false positives on new videos. If I extracted frames from a new video and added them to the training set ( added “hard negatives”), I could fix the false-positive issue for that specific video, but false positives would appear again when I switched to another new video. Previously, I also tried training directly with imgsz=448 using all the default parameters, without changing the confidence threshold, and I still got false positives.

After receiving your message, I just tried what you suggested. I found that when I validated my previously trained imgsz=640 best.pt model with imgsz=448 (using CUDA_VISIBLE_DEVICES=1 torchrun --nproc_per_node=1 val.py), mAP50 did drop significantly—from 0.80 down to 0.73. The PR plots produced during training and validation are shown below. This is a multi-class object detection task. If you need any other result plots, please leave me a message. I’m truly touched by your reply!

If possible, could you recommend a suitable starting confidence threshold, and also comment on whether this looks more like domain shift or label noise—plus how you would tell? I also have one more question: could these false positives be related to data augmentation? Could augmentation be causing them? The false positives have very high confidence, which feels very strange to me.

image
image1328×797 138 KB

image
image1216×708 126 KB

image
image1265×719 126 KB

image
image1441×739 138 KB

7

mode=val imgsz=448 results images

image
image1364×785 162 KB

image
image1303×821 75.3 KB

8

I found that when fine-tuning a 448 model from a best.pt trained at 640, enabling multi-scale training causes mAP@0.5 to drop significantly. I’m using the SGD optimizer with lr=0.01, batch size=256, and 8 RTX 4090 GPUs. Should the learning rate be adjusted accordingly?

9

You should train at the same imgsz as you’re exporting. Don’t use multi scale. And disable mosaic augmentation.

10

Could you help me ？I have already uploaded the corresponding results png

11

First,thank you very much for your help! Let me describe my previous workflow in detail: I trained a model with imgsz=640, then set mode=export to export an ONNX model with imgsz=448. When I ran inference with the ONNX model, I found false positives on new videos. If I extracted frames from a new video and added them to the training set ( added “hard negatives”), I could fix the false-positive issue for that specific video, but false positives would appear again when I switched to another new video. Previously, I also tried training directly with imgsz=448 using all the default parameters, without changing the confidence threshold, and I still got false positives.

After receiving your message, I just tried what you suggested. I found that when I validated my previously trained imgsz=640 best.pt model with imgsz=448 (using CUDA_VISIBLE_DEVICES=1 torchrun --nproc_per_node=1 val.py), mAP50 did drop significantly—from 0.80 down to 0.73. The PR plots produced during training and validation are shown below. This is a multi-class object detection task. If you need any other result plots, please leave me a message. I’m truly touched by your reply!

If possible, could you recommend a suitable starting confidence threshold, and also comment on whether this looks more like domain shift or label noise—plus how you would tell? I also have one more question: could these false positives be related to data augmentation? Could augmentation be causing them? The false positives have very high confidence, which feels very strange to me.

val imgsz=448, weights=best.pt(imgsz=640)

image

image

imagemode=val imgsz=448 results images

image

image

train 640 results.png

image

13

friend， could you tell me what are the benefits of disable mosaic augmentation

14

I am turning off mosaic augmentation. Should I use the official pre-training weights or the best.pt that I previously trained for 640? Do I need to modify the lr? I used bs=256, 8 cards, SGD optimizer before, lr=0.01

15

I still haven’t resolved this issue.I retrained the model with imgsz=448 and disabled mosaic augment throughout the entire process, but it still didn’t work—false positives persist.

16

You should check your ground truth labels. From the example image you shared, it seems like shadows are getting detected as cars, and I’d presume that there are ground truth annotations that include shadows.

You can see from the confusion matrix results plots there are many false positive and false negatives, which indicates there’s likely an issue with your ground truth labels. Whenever you see false positive detections like this, it’s very likely that the issue stems from the original data used in training. Additionally, the precision plot begins to decrease after maybe ~50 epochs, which means that the model is struggling to learn from the ground truth labels.

It’s a lot of work, but you need to verify your ground truth labels are correct. The only way to know for certain is to check the labels, but tools like Fiftyone might be able to help compare predictions versus ground truth so you can find what needs to be corrected. You might also try using the pretrained COCO YOLO models against the problematic images, as they are already trained on car, van, truck, etc. and you can compare how it performs against your custom trained model.

17

You should also use a larger model, not the n variant.

Or at least, you need to increase the depth of the n variant.

You can change depth from 0.5 to 1.0