Modifying yolo architecture

Support YOLO
, , , ,
1

Hi everyone, I’m working on object and lane detection using YOLO, and I’ve been trying to choose a topic for my research project. We currently train two separate models for the two tasks, but I’m wondering if I could use a single model for both. Could I modify the YOLO architecture to have one shared backbone and two separate heads? In the end, would this approach be more efficient, or would I lose accuracy?

2

YOLO segmentation models use two heads. One for detection, another for segmentation.

3

but at the end can ı have one .pt file as an example? or would ı be able to have high accuracy scores with fast inference?. I use 25-27 classes for detecting traffic signs. If it’s better for me to modify a high-accuracy and efficient model myself for 27 classes and lane detection, then I’ll do it that way.

4

Yes—you can keep a single .pt. Use a YOLO11 segmentation model; it already has a shared backbone with two heads (detection + masks), so one forward pass returns boxes for traffic signs and masks for lanes from the same checkpoint, as described in the Segment head reference.

Two important notes:

  • Training a segmentation model expects masks for all labeled instances. If you only have boxes for traffic signs, either generate masks for them or keep a second detection-only model.
  • One multi-head model is usually faster than running two models; accuracy is typically comparable if your data is balanced. If you need a bit more headroom, try the next model size up (e.g., yolo11m-seg).

Minimal setup:

# single model for lanes (mask) + signs (boxes)
yolo train task=segment model=yolo11s-seg.pt data=traffic.yaml epochs=100 imgsz=1280

from ultralytics import YOLO
m = YOLO('best.pt')  # result of training above
r = m.predict(source='video.mp4', conf=0.25)
# r[i].boxes -> traffic signs; r[i].masks -> lane masks

If you want to dig deeper into how the dual-head works, see the Segment head reference, and for setup details check the Train mode docs.

5

Thank you for your reply. But what if I have two different datasets for the two problems? I won’t be detecting objects and lanes in the same image.