YOLO26n RKNN inference speed on RK3588 (Orange Pi 5 Plus) + segmentation alternatives?

Discussion
1

Hi all,

I’m deploying YOLO26n (detect) on an Orange Pi 5 Plus (RK3588, 6 TOPS NPU) via RKNN export (end2end=False, per issue #23753) for a real-time edge use case with a ~500ms end-to-end latency budget (detection + tracking + embedding).

  1. Has anyone benchmarked real inference latency/fps for YOLO26n on RK3588/RK3588S via RKNN? Looking for actual on-device numbers (ms/frame for detect + NMS postprocessing), not just the 6 TOPS spec sheet.

  2. As I understand it, RKNN export currently only officially supports the detect task for YOLO26 (no seg/obb). Is that still accurate? If so, are there recommended alternatives for instance segmentation on this NPU — community RKNN export tooling, a different architecture, or anything else worth looking at?

Thanks!

2 Likes
2

Hello.

  1. There are benchmarks with FPS numbers on our docs page: YOLO26 RKNN Export for Rockchip NPU | Ultralytics
  2. That’s correct. Only detect models are currently supported. You can check this. It seems to support RKNN export for YOLO11-seg. ultralytics_yolo11/RKOPT_README.md at 0692e9297670acf4cc6d0cec773d7a9493cb8a5f · airockchip/ultralytics_yolo11 · GitHub
3

So OBB isn’t supported on RKNN/NPU at all? Or is there a working way to run an OBB model on the NPU without custom modifications?

4

The repo I linked supports YOLO11-OBB export on RKNN.

You can’t do it without modifying Ultralytics at all because Ultralytics doesn’t have official support for it. The linked repo is a fork of Ultralytics with modifications for RKNN support.

5

I don’t get your answer. Can you just say clearly: is OBB supported on Rockchip via RKNN or not? Yes or no.

If yes, please share one direct link to the official OBB → RKNN export docs or code.

If no, just say “no” so there’s no confusion.

6

Is OBB on RKNN officially supported by Ultralytics? No.

But you also asked if there are “community tooling” that supports it:

As I understand it, RKNN export currently only officially supports the detect task for YOLO26 (no seg/obb). Is that still accurate? If so, are there recommended alternatives for instance segmentation on this NPU — community RKNN export tooling, a different architecture, or anything else worth looking at?

which is why I linked you to an unofficial community developed alternative above in my earlier response.

If you want to ask if there’s official support by Ultralytics, then the answer is no currently

7

Good news, @user4 — yes, OBB → RKNN export is now officially supported in Ultralytics 8.4.110.

RKNN export now covers all current YOLO tasks: detection, instance segmentation, classification, pose, OBB, semantic segmentation, and depth. For OBB:

pip install -U "ultralytics>=8.4.110"
yolo export model=yolo26n-obb.pt format=rknn name=rk3588

Official documentation: YOLO26 RKNN Export for Rockchip NPU | Ultralytics
Implementation PR: Expand RKNN export support to all tasks - Pull Request #25490 - ultralytics/ultralytics - GitHub

We validated RKNN Toolkit conversion for all seven tasks in Linux Docker and added the same all-task smoke matrix to CI and the production worker image. One important distinction: this validates successful RKNN export/conversion; actual NPU inference latency and accuracy still need to be measured on Rockchip hardware.

1 Like
8

Has yolo26-obb NMS or it deleted like in yolo26detect?

9

All the YOLO26 models support inference without NMS including YOLO26 OBB models. However, RKNN doesn’t support the NMS-free head, which is why if you run inference using RKNN, you still need NMS. This a limitation with RKNN. It doesn’t support the required ops.

10

That’s unfortunate. It looks like we’ll have to implement the NMS block in C++ ourselves.

Do you already have any C++ inference examples for running YOLO26 OBB with NMS? Or what approach would you recommend?

Also, am I correct that inference on Rockchip will generally be faster in C++ than in Python?

11

We have C++ postprocessing function for all the tasks here:

12

Hi @user4 @pderrenger, One thing I would keep in mind when moving from YOLO26 FP32 to RKNN + custom C++ postprocessing is that export success does not always guarantee output parity. In your case, there are several independent variables that can affect final counting accuracy:

  • RKNN INT8/FP16 numerical changes may affect borderline detections.
  • The custom C++ NMS/postprocessing implementation may introduce differences compared with the original PyTorch pipeline.
  • Small confidence or box geometry changes can be amplified by tracking logic (especially under heavy occlusion).

A useful validation step before tuning the dataset further is to run the same field frames through:

PyTorch FP32 reference
vs.
RKNN + C++ postprocessing output

and compare:

  • confidence score deltas
  • box/OBB geometry shifts
  • dropped detections after thresholding
  • tracker association changes

This helps separate:

“the model needs more data”
from
“the deployment pipeline changed the model behavior.”

I am currently exploring a lightweight label-free deployment parity diagnostic workflow for exactly this type of cross-backend validation. Your RK3588 + YOLO26 + custom NMS setup would actually be an interesting real-world case to test this approach. Happy to compare notes if you reach the export stage.

13

To give a pragmatic update for when you reach the RKNN export and custom C++ NMS stage:
Since you are running tracking downstream (e.g. for real-time video/counting), even tiny confidence deltas between PyTorch FP32 and C++ postprocessing can cause “flickering” or dropped tracks under heavy occlusion. If you encounter unexpected missed detections on RK3588 later, here is a quick 3-step Parity Checklist to run before changing your dataset:

1.Pre-NMS Raw Tensor Alignment: Feed the exact same preprocessed image tensor to both PyTorch FP32 and RKNN NPU. Compare raw logit/confidence deltas to isolate quantization loss from NMS logic.
2.C++ NMS Threshold Boundary Check: Log detections that exist in PyTorch FP32 with confidence around . See if RKNN numerical drift pushed them just below your C++ NMS threshold.
3.Temporal Stability / Tracker Check: Pass PyTorch FP32 bounding boxes vs. RKNN C++ boxes into your tracker separately on a short video clip. Compare ID switch counts to verify if tracking instability is caused by geometry jitter.

Isolating these 3 layers early saves weeks of unnecessary re-annotation .

14

@pderrenger what do u think? maybe you have prepare inference scripts in c++ for inference yolo26n-obb.rknn ?

15

Not yet — we don’t currently have an official end-to-end C++ RKNN runtime example for yolo26n-obb.rknn.

The yolo_postprocess.hpp file linked above provides shared preprocessing and postprocessing, but it does not load or execute an RKNN model. I should also clarify that its current non-end-to-end OBB path uses axis-aligned proxy NMS, so it is not a complete reference for accurate rotated NMS on RKNN.

The supported ready-to-run path today is Python with YOLO("./yolo26n-obb_rknn_model"), as shown in the RKNN documentation. A C++ implementation would need to connect Rockchip’s rknn_api input/output buffers to preprocessing and a true rotated-box NMS implementation. I don’t want to point you to the existing header as though a verified turnkey C++ RKNN OBB example already exists.