Actual Behavior:
Only the first Image is segmented. The other 3 images have 0 detections / segmentations. This is also occurring in my own tensort inferencing routines.
If I export without nms, then the same predict cli command will segment all 4 images.
Hm, that’s odd. Where you have images is that a list of images or a directory? Have you tested inference with an exported model where nms=True without batching? Just curious to better understand where the problem is specifically.
It’s because with nms=True, you have to also specify max batch during export. By default, that’s 1, so it will only return predictions for the first image during inference.
Thank you. That fixes the issue partially, but there is something odd going on I believe with the cli prediction side.
If I export using: yolo export model=yolo11-boxseg.pt format='onnx' dynamic=True batch=12 nms=True
then this command works as expected: yolo predict task=segment model=yolo11-boxseg.onnx source=images batch=4
but this command with batch omitted returns 0 detections in any of the images: yolo predict task=segment model=yolo11-boxseg.onnx source=images
When I first exported a dynamic model yesterday with the nms and batch parameters set, I was using the cli to test predict on the onnx model and I could swear I had a case where there were only detections on the first image and not the others. In fact, I still have the run directory for that case, but I don’t know what export / predict calls generated it. I tried to recreate everything I did yesterday, but now I’m getting no detections on any of the images if batch is not set on the predict command, which I assume is not the expected behavior?
Hello! Thanks for the detailed follow-up and for providing such a clear description of the issue.
You’ve correctly identified a known requirement and what appears to be a bug. The initial problem you faced is because dynamic models exported with nms=True require a maximum batch size to be set during export, for example, batch=16. The default batch=1 is insufficient for the embedded NMS logic, which is why your second export command using batch=12 is the correct method. This behavior is highlighted in our exporter logic, as you can see in the Exporter class reference.
The new issue, where predict fails with an implicit batch=1 on the correctly exported model, is not expected behavior. A dynamic model should handle inference on any batch size up to the maximum it was exported with. This points to a potential edge case bug in the exported NMS operations when the inference batch size is exactly 1.
We appreciate you bringing this to our attention. The team will investigate this. In the meantime, the workaround is to explicitly set a batch size in your predict command, such as batch=2, even when processing a single image.