I’m reaching out to inquire about the best hardware solution for my object detection project. I’m aiming for a minimum of 10 FPS, high accuracy, and stable results.
models I think they can meet my requirments: Yolov5su, Yolov5m6u.
I’ve read that the Jetson Nano offers good performance and power efficiency, while the Raspberry Pi with Hailo accelerator seems to have a more complex setup and less community support.
I’m a bit confused about which option would be the most suitable for my needs. I would appreciate your expert advice on this matter.
Hailo isn’t officially supported by Ultralytics. Some people did get it working in the community, but it wasn’t straightforward. For Jetson, Ultralytics has several guides and also TensorRT support with int8 quantization which can help quite a bit with the FPS.
maybe you are right, but its possible to export the trained model as onnx, then convert it to .hef model that can work on hailo. as this video explaining step by step. Am I correct?
An NVIDIA Jetson would be something that would be more than capable of getting 10 FPS, although keep in mind that the Jetson Nano is a very old device and is not getting updated. If you go that route, you should considered something more recent for continued support and updates, as you will be reliant on the JetPack versions from NVIDIA (only critical if you want to update, if you don’t feel you’ll want updates then it won’t matter).
The RPi might be able to run a small model with 10-15 FPS without an accelerator. If I recall correctly, NCNN models performed best when tested by our embedded engineer. So maybe try that first? Depending on your needs, there are other options for embedded devices that could suffice. If you’re aiming to get it done in a budget and space isn’t an issue, maybe find a used PC with a decent CPU and get a used NVIDIA GPU (probably 2060 or higher).
Actually, I think you should test without an accelerator to start. I would suggest YOLO11n exported to NCNN format per Raspberry Pi - Ultralytics YOLO Docs
Additionally use half=True and the smallest imgsz possible (that would work for your application) and see how this performs. If you’re able to get what you need without an accelerator, it would be the simplest way to deploy. If you require an accelerator, you’ll have to check with the Hailo docs/repo on how to get it working, but AFAIK they have some conversion you can run for a model so that it is compatible with their platform.
Compile it for hailo8 and export as engine (for Tensorrt) for Jetson Orin NX 16 Gb. For Jetson I even use Half=True.
So I run same approaches by preparing really heavy logic.
My stats:
RP5+Haili8 = ~5,7 FPS
Jetson Orin NX = ~13 FPS
But first stack 10 times less cost than second which is annoying me.
I suppose to think that Jetson sold us marketing digits according to TOPS numbers.
So, in conclusion, yes…hailo harder to set up but it is really cool playmaker. I believe if Hailo create smth with same TOPS digits it will break a neck of Jetson.
Something seems off here. If you check out the docs on the TensorRT integration page, there’s a table showing the performance of YOLO models on the same Orin NX device. The longest inference speed (slowest) when using half=True (FP16) during export, is 3.2 ms which would be just over 300 FPS. So if you’re only seeing ~13 FPS on your Jetson device, something is not correct.
I don’t have one to check with, but I will ping someone who does. In the meantime, I suggest checking that your packages are properly updated (or try using one of the Jetpack Docker images from the repo) and trying to export the model again. Also, ensure that you export the model to .engine format (TensorRT) on the actual Jetson device as well, as the export process optimizes for the device the export is done from.
But TOPS measures integer operation throughput. It only matters when you’re running INT8. If the model isn’t exported to INT8, TOPS isn’t the metric to look at. When you export with half=True, you’re running FP16 inference which is measured in TFLOPs, not TOPs. FP16 inference has better accuracy than INT8.
Hailo executes everything in INT8 by design, which means you’re trading some accuracy for throughput, and the hardware doesn’t accelerate floating-point inference. A fair comparison is Jetson running TensorRT in INT8 mode (int8=True) rather than FP16 or FP32.
Someone has already reported more than 150 FPS on a Jetson Orin Nano Super, and that device is actually weaker than the Orin NX:
We have repeatedly tested YOLO model performance on Jetson devices, specifically the Jetson Orin NX 16GB you have, and obtained the benchmark numbers as mentioned in our Jetson documentation.
It seems that there is something off in your setup environment because 13FPS for inference on that device seems not right.
Could you please tell me which YOLO model size you are using?
Which JetPack version are you using?
Which Ultralytics version are you using?
Attach screenshots of “ALL” and “INFO” tabs of jetson-stats application after installing via:
Another thing to note is that, you need to enable MAXN power mode together with jetson_clocks, because some of the CPU, GPU cores are locked by default after flashing a new JetPack system.