YOLOv11 ONNX Export Error with Batch > 1 and NMS

Environment:
Ubuntu 22.04
Python 3.10.12
Ultralytics 8.3.155

Executing:

yolo export format='onnx' dynamic=True model=yolo11n.pt nms=True
yolo predict task=segment source=images batch=4 model=yolo11n.onnx

Expected Behavior:
4 segmented Images

Actual Behavior:
Only the first Image is segmented. The other 3 images have 0 detections / segmentations. This is also occurring in my own tensort inferencing routines.

If I export without nms, then the same predict cli command will segment all 4 images.

Hm, that’s odd. Where you have images is that a list of images or a directory? Have you tested inference with an exported model where nms=True without batching? Just curious to better understand where the problem is specifically.

CC: @Toxite

It’s because with nms=True, you have to also specify max batch during export. By default, that’s 1, so it will only return predictions for the first image during inference.

1 Like

Opened a PR to warn about this similar to TensorRT with dynamic

Thank you. That fixes the issue partially, but there is something odd going on I believe with the cli prediction side.

If I export using:
yolo export model=yolo11-boxseg.pt format='onnx' dynamic=True batch=12 nms=True

then this command works as expected:
yolo predict task=segment model=yolo11-boxseg.onnx source=images batch=4

but this command with batch omitted returns 0 detections in any of the images:
yolo predict task=segment model=yolo11-boxseg.onnx source=images

When I first exported a dynamic model yesterday with the nms and batch parameters set, I was using the cli to test predict on the onnx model and I could swear I had a case where there were only detections on the first image and not the others. In fact, I still have the run directory for that case, but I don’t know what export / predict calls generated it. I tried to recreate everything I did yesterday, but now I’m getting no detections on any of the images if batch is not set on the predict command, which I assume is not the expected behavior?

Hello! Thanks for the detailed follow-up and for providing such a clear description of the issue.

You’ve correctly identified a known requirement and what appears to be a bug. The initial problem you faced is because dynamic models exported with nms=True require a maximum batch size to be set during export, for example, batch=16. The default batch=1 is insufficient for the embedded NMS logic, which is why your second export command using batch=12 is the correct method. This behavior is highlighted in our exporter logic, as you can see in the Exporter class reference.

The new issue, where predict fails with an implicit batch=1 on the correctly exported model, is not expected behavior. A dynamic model should handle inference on any batch size up to the maximum it was exported with. This points to a potential edge case bug in the exported NMS operations when the inference batch size is exactly 1.

We appreciate you bringing this to our attention. The team will investigate this. In the meantime, the workaround is to explicitly set a batch size in your predict command, such as batch=2, even when processing a single image.

Does the logs also say no detection?

I am not able to reproduce the issue.