Raspberry Pi 5 Crash When Running YOLOv11n NCNN

Support YOLO
,
1

Hey all,

I’m pretty new CV in general, and just started playing with YOLOv11n last week on a Raspberry Pi 5 (4GB RAM). So far, I’ve been really impressed with how quickly inference runs on this little machine, but I’m having some trouble with the Pi crashing when I try and run an exported NCNN model. Ultralytics has a great quick-start doc for working on Pi, and I was able to get the ARM Docker container pulled and running inference without a hitch. I was also able to install Ultralytics (v8.3.202) outside of the container in a Python venv, again without issue.

I followed the instructions to export in NCNN format inside the container, and was even more blown away by 70ms inference on 640p frames. This was easily 3x faster than running the standard PyTorch model, and when running inference as a background task in a larger app, it was over 5x faster (~90ms vs ~500ms). Again, I was able to run this inside the container, and also copy the model (the full yolo11n_ncnn_model directory generated at export) outside of the container to run on the host in my venv. When running inference in the context of my larger app, I don’t think I ever saw memory usage get above 1GB, or CPU spike above 50% across the 4 cores.

Here’s where things get funky. I only played with it for an afternoon, and a few days later, I tried using it again and found that trying to load the NCNN model I’d copied out of the container would seize the Pi. Entering the container, I could still export the NCNN model (using CLI command in doc), but when trying to run the NCNN model, the Pi would always seize (fully unpingable on the network) and require a power cycle.

Things I tried…

  • Logging system resource usage at the time of the 100%-reproducible crash, but didn’t capture anything that would indicate exhausting RAM or CPU
  • Removing and re-pulling the image
  • Perusing forums, installing older versions of Ultralytics and dependencies, both inside and outside of the container
  • Installing a fresh Bookworm image to an SD card, installing in the Pi, pulling the container

Some sanity checks…

  • The standard PyTorch model always worked on the Pi without a problem, inside and outside of the container; it’s just slower, as expected
  • I installed the (non-ARM) container on my Ubuntu 22.04 laptop, exported NCNN, and ran – no problems here; it was isolated to the Pi
  • In the exported yolo11n_ncnn_model directory, model.ncnn.bin is ~10.6MB, and model.ncnn.param is ~22.7kB

I came back to it again yesterday and was able to re-pull the image (there was an update in recent days). I spun up a container and lo-and-behold, I could export NCNN and run it inside and outside of the container without any problems. Ultralytics in is now at v8.3.204, which had me thinking maybe there was some bug in 202, although that still didn’t explain how it went from working to not-working.

Maybe 2 hours after NCNN was working without issue, both inside and outside the container, through Raspberry Pi power cycles, etc, I tried running NCNN and it was crashing the Pi again. Again, I went through fully removing the image and re-pulling, but no dice.

If you’ve made it this far, thanks for reading my novel! Any hunches on what could be happening here? I’m utterly perplexed, especially considering that the Docker image should be able to reproduce the same exact container again and again (disclaimer: not even close to a Docker expert). I have to imagine I’m missing something simple; hoping one of you can tell me what that is!

2

Tagging in @lakshanthad to help out since he’s got the most insight (and capability to test) everything embedded and RPi

1 Like
3

Hello @afbowen,

Sorry to hear about your situation. I will help you further debug this issue and find a solution.

First of all, thank you for the detailed explanation about your issue. It gave me a lot of clarity about the issue. Before we move forward, could you please help me answer the following:

  1. Power supply used to power the RPi 5?
  2. Micro-SD or SSD used for the system?
  3. After how long of running the model, the RPi 5 crashes?
  4. Screenshot of htoputility at the time of crash?
  5. Any example code you could provide so that I could reproduce here on my 4GB Raspberry Pi 5? (I would appreciate if you could send me the model that is currently crashing, mentioning about the Ultralytics version which is used to export the model as well).

I have all 4GB, 8GB and 16GB Pi 5s to proceed with the testing.

Thank you!

4

Thanks for getting me routed, @BurhanQ! I appreciate the quick response @lakshanthad :raising_hands:

I got the chance to do a few more clean-slate cycles today, trying fresh installs of both the desktop and headless versions of Bookworm 64-bit. In the first attempt this morning, I was able to run inference on the headless, but was no longer able to after rebooting the Pi. Inference was also running slower than I remember it, consistently taking ~100ms instead of the ~70ms I was seeing earlier this week. Running the PyTorch model still appears to be ~330ms.

I’ve done my best to stay consistent and not introduce other variables, running through the following steps 3 times:

  1. Install the OS on the SD using Raspberry Pi Imager v1.9.6
  2. Insert SD, power up Pi, shell in over WiFi
  3. Install Docker
  4. Pull the docker image
  5. Run a container, then run yolo export and predict (per the doc)

The Pi has seized each time with NCNN. After rebooting, I find I can run the PyTorch model without a problem. I’ve explicitly tried running straight from the fresh OS install, and after running apt updateand upgrade.

To answer your questions…

  1. I’m using a 67W Anker power supply. Running vcgencmd get_throttled yields throttled=0x0 (no undervoltage, etc conditions since boot). Using an inline power meter, when I run inference, I see a spike to about 5.2W before it drops to 0.1W after the Pi crashes.
  2. I’m using 2 different MicroSD cards of the same type – SanDisk UHS-I (64GB, 256GB).
  3. It crashes when calling yolo predict with the CLI, or ncnn_model() for Python (cut and pasted commands/code from the doc, also tried with local image to rule out a download issue).
  4. See attached screenshot of htoprunning in separate shell (this is how it freezes after inference seizes the Pi).
  5. For all recent testing trying to diagnose, I’ve just run the CLI commands provided in the doc.

I’ve posted the PyTorch and NCNN models in this folder. Nothing special about them; it’s just the standard yolo11n.pt you get with the container, and yolo11n_ncnn_model is the directory you get after exporting. The ultralytics module is at 8.3.204.

yolo predict Output

root@7e8af2bea5c8:/ultralytics# yolo predict model='yolo11n_ncnn_model' source='https://ultralytics.com/images/bus.jpg'
WARNING ⚠️ Unable to automatically guess model task, assuming 'task=detect'. Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'.
Ultralytics 8.3.204 🚀 Python-3.11.2 torch-2.8.0+cpu CPU (aarch64)
Loading yolo11n_ncnn_model for NCNN inference...

Downloading https://ultralytics.com/images/bus.jpg to 'bus.jpg': 100% ━━━━━━━━━━━━ 134.2KB 942.3KB/s 0.1s
Timeout, server pi5.local not responding.

HTOP

pi5_ncnn_htop
pi5_ncnn_htop1662×601 159 KB

I’m having trouble getting a good screenshot at the peak of when the yolo11n.pt model runs, but it also pins a single core to 100% (the others are at 0%) then drops to ~99.5% in the next frame (others in single-digits %).

Let me know if there’s anything more I can provide!

5

Thanks for the super detailed repro and for sharing the models—very helpful. Given the full-board freeze (SSH drops) right after “Loading yolo11n_ncnn_model…”, this looks like a kernel-level hang rather than a Python OOM. A few quick next steps to isolate and get you unblocked:

  • Please grab the previous-boot kernel logs after a crash: run journalctl -k -b -1 | tail -n 200 and paste the output. This will tell us if there’s a kernel panic, watchdog, or undervoltage just before the lock.
  • Share the exact docker run command you use (we recommend --ipc=host per the guide). Also share versions: python -c "import ultralytics, ncnn; print(ultralytics.__version__, getattr(ncnn,'__version__','n/a'))".
  • Pin the NCNN wheel and retest outside Docker to rule out a wheel regression:
    • pip install --force-reinstall "ncnn==<a previous date-tag version>" (check your current with pip show ncnn and try one older/newer).
  • As an immediate workaround, try OpenVINO. On Pi 5 it’s typically as fast or faster than NCNN (our Pi guide shows ~85 ms for YOLO11n vs ~94 ms for NCNN):
    • yolo export model=yolo11n.pt format=openvino
    • yolo predict model=yolo11n_openvino_model source='https://ultralytics.com/images/bus.jpg'
      See the Raspberry Pi quick-start benchmarks in the Ultralytics docs for context.

Two environment checks that have helped others with hard locks under burst CPU load:

  • Test once with the official 27 W Raspberry Pi PSU (PD behavior differs by charger even if “67 W” capable).
  • After reboot, monitor in a second shell: watch -n1 vcgencmd get_throttled and run NCNN again to see if flags change just before the freeze.

We’ll try to reproduce on our 4 GB Pi 5 with 8.3.204 and the same container. If you can post the journalctl -k -b -1 tail, pip show ncnn, and your docker run … line, we can zero in quickly. For reference, the setup and format trade-offs are summarized in the Ultralytics Raspberry Pi guide under NCNN and the benchmark table.

6

I have also same experienced with 16GB model. under this environments I had, no docker, yes virtual environment(venv), python v3.13(default). yolo11n, pccamera v2. 256 SAN SSD. 64bits OS. I will have test and disclose the result as leader puts. during installation of ultralytics[export] which generates error, instead of it, I used “pip install ultralytics –no-deps”.

7

Interesting. What was the dependency error you had when trying to install ultralytics[export]?

8

@afbowen Thank you for the very detailed information! Sorry to get back so late.

I have done multiple tests of running NCNN exported models both on host and Docker container and they seem to work fine on all Raspberry Pi 5 4GB/ 8GB/ 16GB.

Could you now try with the latest Ultralytics v8.3.231 on host with a Python 3.11 venv which comes with Bookworm OS.

We have also recently updated our Docker image for arm64. Could you pull the latest and try again?

t=ultralytics/ultralytics:latest-arm64
sudo docker pull $t && sudo docker run -it --ipc=host $t

@calmroad To answer your question, currently Ultralytics only supports up to Python 3.12. Since you are in a Python 3.13 environment, there could be some dependency issues during the installation process. So we suggest you to proceed with Python 3.12.