use ultralytics_inference::{YOLOModel, InferenceConfig};
fn main()
{
// Load your custom NCNN model (auto-detects format)
let mut model = YOLOModel::load("custom_ncnn_model/").unwrap();
// Configure exactly like Python predict params
let mut config = InferenceConfig::default();
config.confidence_threshold = 0.6; // Confidence threshold
//config. = true; // Real-time display window
//config.line_thickness = 2; // Box line thickness
config.save = false; // No output saving
//config.stream = true; // Process video frame-by-frame
// Run prediction (handles video streaming + visualization automatically)
//model.predict_config("video.mp4", config);
model.predict("video.mp4").unwrap();
}
However I get this error:
thread 'main' (100562) panicked at src/main.rs:6:59:
called `Result::unwrap()` on an `Err` value: ModelLoadError("Failed to load model: Load model from custom_ncnn_model/ failed:Protobuf parsing failed.")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Doesn’t this crate support ncnn models? Do I have to use oxnn instead?
According to the project Readme, only ONNX is supported at the moment. I know there are plans to include as many/all of the formats, but the project is still in early development (really experimental), so not certain when all formats will have support.
I’m not familiar with Rust personally, so I can’t review the project directly. I will double check with the person working on the project and let you know as soon as possible, but they had some time off recently so it may take a bit of time.
Checked with Copilot and the Ultralytics Assistant, and it appears that as of now, the [Ultralytics Rust inference project] (GitHub - ultralytics/inference: Rust inference package experiments) only supports ONNX models. I’ll follow up with info about ncnn support when I hear back.
Heard back. Sounds like ncnn is planned much later in the roadmap, and since the project is still considered experimental, it could be a while. If you’d like to use the ultralytics_inference Rust crate, you’ll have to stick with ONNX for now. Of course, if you would like to contribute support for ncnn, a PR is welcome, but no guarantee it’ll be accepted.
Both formats are good for running on RPi, and ncnn does have a lower inference time than onnx. It’s a compute and power constrained device, so it won’t be fast but it can be reasonable. The question really becomes, what’s acceptable for your use case? It can be nice to have the faster option, but the question really would be, how fast do you need it to be?
I can’t say from experience, and I’m not sure if anyone has run any tests. Most of Ultralytics developers are familar and use Python, but obviously there are a few Rustacians out there. If I were to guess, I suspect the model inference time for onnx and ncnn might not change much between Python and Rust, since usually that’s a model/hardware constraint. The end-to-end time (including pre- and post-processing, as well as the image/video loading) is likely to be the bigger distinguishing factor. I suspect, I don’t know, that Rust will do this faster, however I don’t know how much faster.
I recommend doing some quick testing to find out, both for yourself and others in the community. I’d wager that if you compare the Rust inference using onnx and Python using ncnn, that they might come out to be relatively close overall, since the ncnn inference time is much lower than onnx for RPi. If that’s the case, then I’d recommend going with which ever language you’re more familar with, as it’ll simplify the work you have to do going forward. Even if one is faster than the other, if the difference isn’t enough to impact your use case (you’re the only one who will know the answer to that), I’d still recommend using whichever language you’re more familar with.
