NCNN vs Coral TPU

Hello, I am working on deploying a segmentation model on my Raspberry Pi 5, but I’ve encountered a performance issue. Surprisingly, the model performs better when converted to the NCNN format compared to using the Google Coral TPU.

With the NCNN format, the model processes frames in about 30–40 ms, whereas with the TPU, it takes 80–90 ms per frame. This result is unexpected, as I anticipated the TPU to deliver better performance.

I’m unsure if I’ve made a mistake in my setup or configuration. Any advice or insights would be greatly appreciated.

Can you try exporting with imgsz=512. It should be able to give you better performance results as with a higher imgsz some ops don’t get mapped to the tpu. You can also look at the benchmarks here

I tried exporting to 320, but the ncnn format is still faster.

Are you using yolo11? If so, try yolov8, it might be faster on the tpu.

I already tried, but the performance remained the same. It’s possible that the Raspberry Pi is running in low-power mode, which could explain the poor performance, as it restricts power usage for peripherals.

See also the RPi5 benchmarks in the docs.

I haven’t seen any benchmarks related to the Edge TPU there, but considering that the TF Lite format is used for the TPU, it makes sense for it to be slower than the NCNN format.

Hello Seby_R,

Thanks for reaching out. It’s great to hear you’re exploring different deployment options for your segmentation model on the Raspberry Pi 5.

You mentioned seeing better performance with the NCNN format compared to the Coral Edge TPU, which is unexpected. While you suspect low-power mode might be a factor, I recommend ensuring you’re using the high-frequency mode for the Edge TPU. You can find details on how to select the appropriate version in the Installation Walkthrough section of our Edge TPU guide. Also ensure you are using a USB 3.0 port.

Regarding your comment about TFLite and Edge TPU benchmarks, the Edge TPU is indeed designed to accelerate TFLite models. The Raspberry Pi guide provides a comparison between formats, including NCNN.