Multi-Input Detection Problem

Discussion
1

Hello,

I’m working on an acoustic object detection task using the Ultralytics YOLOv11 framework. My dataset consists of paired spectrogram images for each sample: one representing the vertical axis and one representing the horizontal axis. These two images are inherently correlated (capturing the same acoustic event from orthogonal perspectives), and I want the model to leverage this relationship during feature extraction.

Input: Two synchronized images (vertical & horizontal) per training/inference sample.
Output: Separate bounding box coordinates and class predictions for each image.
Constraint: The model must process both images simultaneously, exploit their cross-correlation, but produce independent detection outputs per view.

I’m looking for guidance on the most robust and efficient way to implement this within the Ultralytics YOLO ecosystem.

Thank you.

2

You could merge the images to create a single image with the channels from both the image and perform multispectral training.

3

Thank you for your reply, but the problem is not merging images.

I can merge two images to 6 channels and can use yolov11. The problem is that we need to output separate bboxes for both images.

4

You can probably merge the labels. Make the classes distinct for the two so that you can tell from the output which of the two it belongs to.

The model can figure out which channels and their correlation to output classes automatically.

It’s the easy way that doesn’t require radically rewriting Ultralytics pipeline