Slimming with onnxslim 0.1.50 fails with yolo export on Jetson 6.2 L4T

Discussion
, ,
1

Greetings, I have been struggling for the past couple of days to understand and resolve the following error when trying to convert a stock YOLO11 Pytorch model to native TensorRT for the Jetson Orin AGX. The error I receive, regardless of which module source I use:

‘’’
yolo export model=“yolo11s.pt” format=engine half=True dynamic=True device=0
Ultralytics 8.3.115 :rocket: Python-3.10.12 torch-2.7.0 CUDA:0 (Orin, 62841MiB)
YOLO11s summary (fused): 100 layers, 9,443,760 parameters, 0 gradients, 21.5 GFLOPs

PyTorch: starting from ‘yolo11s.pt’ with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (18.4 MB)

ONNX: starting export with onnx 1.17.0 opset 19…
ONNX: slimming with onnxslim 0.1.50…
/opt/rh/gcc-toolset-14/root/usr/include/c++/14/bits/stl_vector.h:1130: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator [with _Tp = unsigned int; _Alloc = std::allocator; reference = unsigned int&; size_type = long unsigned int]: Assertion ‘__n < this->size()’ failed.
Aborted (core dumped)
‘’’

The replacement models for pytorch were installed by:
‘’’
pip3 install --force --no-cache-dir torch torchvision torchaudio --index-url https://pypi.jetson-ai-lab.dev/jp/cu126
‘’’

Which, checking versions and GPU availability on the Orin AGX 64GB, produces:
‘’’
python3 -c “import torch, torchvision, torchaudio ; print(torch.version, torchvision.version,torchaudio.version) ; print(f’GPU available? {torch.cuda.is_available()}')”

2.7.0 0.22.0 2.7.0
GPU available? True
‘’’

I tried to resolve the error, by installing the following onnxruntime_gpu versions:

  1. ‘pip3 install --force onnxruntime_gpu-1.17.0-cp310-cp310-linux_aarch64.whl’
  2. pip3 install --force onnxruntime_gpu-1.22.0-cp310-cp310-linux_aarch64.whl

After I had originally installed, per the instruction guide doc: pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.20.0-cp310-cp310-linux_aarch64.whl

Where I get the exact same error.

I can remove the module altogether and the export conversion completed albeit without the slimming through onnxslim (auto install was attempted but failes). Could anyone recommend a fix for this bug? Should I re-install from scratch an older version of JetPack? I was having problems with older versions of Pytorch and ultralytics, which was resolved by using the correct JetPack versions provided by nvidia, according to previous nvidia community user support messages.

thanks in advance for anyone’s time & advice!

-ted

2

Hi Ted, thanks for the detailed report on the issue you’re encountering with onnxslim during export on JetPack 6.

It seems like there might be an incompatibility with onnxslim version 0.1.50 in your specific Jetson environment setup.

Could you please try updating to the latest ultralytics package first to see if the issue persists?

pip install -U ultralytics

If the error continues, you can bypass the slimming step by adding simplify=False to your export command. This will skip the problematic onnxslim step while still allowing you to generate the TensorRT engine file.

yolo export model=yolo11s.pt format=engine half=True dynamic=True device=0 simplify=False

This should allow the export process to complete successfully. Let us know if this helps!

3

Maybe try one of the Ultralytics JetPack Dockerfiles? Those should help isolate an system variables and give a more reproducible environment for troubleshooting.

I can’t tell where the error is coming from with the error posted (I also don’t know C++ at all). If you think there’s an issue specifically with onnxslim you might want to open an Issue on their GitHub.

Since it appears that the error is coming from gcc-toolset-14 you might want to verify compatibility with onnxslim as well.

4

Hi pderrenger,
Thanks for the suggestions. Yes, setting simplify=False, did export to the .engine model. I also solved the initial problem after realizing that onnyxslim module was referencing the pip3 wheel compiled onnyxruntime instead of the Ultralytics supplied Jetson L4T Jetpack one, by first noting from pip3 freeze produced:

absl-py==2.2.2 appdirs==1.4.4 apturl==0.5.2 astunparse==1.6.3 attrs==25.3.0 bcrypt==3.2.0 beniget==0.4.1 blinker==1.4 Brlapi==0.8.3 Brotli==1.0.9 cachetools==5.5.2 cattrs==24.1.3 certifi==2020.6.20 chardet==4.0.0 chex==0.1.89 click==8.0.3 colorama==0.4.4 coloredlogs==15.0.1 coremltools==8.2 cpuset==1.6 cryptography==3.4.8 cupshelpers==1.0 cycler==0.11.0 dbus-python==1.2.18 decorator==4.4.2 defer==1.0.6 distro==1.7.0 distro-info==1.1+ubuntu0.2 duplicity==0.8.21 etils==1.12.2 exceptiongroup==1.2.2 fasteners==0.14.1 filelock==3.18.0 flatbuffers==25.2.10 flax==0.10.4 fonttools==4.29.1 fs==2.4.12 fsspec==2025.3.2 future==0.18.2 gast==0.6.0 google-auth==2.39.0 google-auth-oauthlib==1.2.2 google-pasta==0.2.0 grpcio==1.71.0 h5py==3.13.0 httplib2==0.20.2 humanfriendly==10.0 humanize==4.12.2 idna==3.3 importlib-metadata==4.6.4 importlib_resources==6.5.2 jax==0.4.34 jaxlib==0.4.34 jeepney==0.7.1 Jetson.GPIO==2.1.7 Jinja2==3.1.6 joblib==1.4.2 keras==2.15.0 keyring==23.5.0 kiwisolver==1.3.2 language-selector==0.1 launchpadlib==1.10.16 lazr.restfulclient==0.14.4 lazr.uri==1.0.6 libclang==18.1.1 linkify-it-py==2.0.3 lockfile==0.12.2 louis==3.20.0 lxml==4.8.0 lz4==3.1.3+dfsg macaroonbakery==1.3.1 Mako==1.1.3 Markdown==3.8 markdown-it-py==3.0.0 MarkupSafe==3.0.2 matplotlib==3.5.1 mdit-py-plugins==0.4.2 mdurl==0.1.2 ml-dtypes==0.3.2 monotonic==1.6 more-itertools==8.10.0 mpmath==1.3.0 msgpack==1.1.0 nest-asyncio==1.6.0 networkx==3.4.2 numpy==1.26.4 oauthlib==3.2.0 olefile==0.46 onboard==1.4.1 onnx==1.17.0 onnxruntime==1.21.1 <------------------------ wrong module for Jetson ----- onnxruntime-gpu @ https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.20.0-cp310-cp310-linux_aarch64.whl#sha256=0c418beb3326027d83acc283372ae42ebe9df12f71c3a8c2e9743a4e323443a4 onnxslim==0.1.50 opencv-python==4.11.0.86 openvino==2025.1.0 openvino-telemetry==2025.1.0 opt_einsum==3.4.0 optax==0.2.4 orbax-checkpoint==0.11.5 packaging==23.2 pandas==1.3.5 paramiko==2.9.3 pexpect==4.8.0 pillow==11.2.1 platformdirs==4.3.7 ply==3.11 protobuf==4.25.6 psutil==7.0.0 ptyprocess==0.7.0 py-cpuinfo==9.0.0 pyaml==25.1.0 pyasn1==0.6.1 pyasn1_modules==0.4.2 pycairo==1.20.1 pycups==2.0.1 Pygments==2.19.1 PyGObject==3.42.1 PyJWT==2.3.0 pymacaroons==0.13.0 PyNaCl==1.5.0 PyOpenGL==3.1.5 pyparsing==2.4.7 pyRFC3339==1.1 python-apt==2.4.0+ubuntu4 python-dateutil==2.8.1 python-dbusmock==0.27.5 python-debian==0.1.43+ubuntu1.1 pythran==0.10.0 pytz==2022.1 pyxdg==0.27 PyYAML==5.4.1 requests==2.25.1 requests-oauthlib==2.0.0 rich==14.0.0 rsa==4.9.1 scikit-learn==1.6.1 scipy==1.15.2 seaborn==0.13.2 SecretStorage==3.3.1 simplejson==3.20.1 six==1.16.0 sympy==1.13.3 systemd-python==234 tensorboard==2.15.2 tensorboard-data-server==0.7.2 tensorflow==2.15.1 tensorflow-cpu-aws==2.15.1 tensorflow-decision-forests==1.8.1 tensorflow-estimator==2.15.0 tensorflow-hub==0.16.1 tensorflow-io-gcs-filesystem==0.37.1 tensorflowjs==4.22.0 tensorrt==10.3.0 tensorrt_dispatch==10.3.0 tensorrt_lean==10.3.0 tensorstore==0.1.73 termcolor==3.0.1 textual==3.1.1 tf_keras==2.15.1 threadpoolctl==3.6.0 toolz==1.0.0 torch==2.7.0 torchaudio==2.7.0 torchvision==0.22.0 tqdm==4.67.1 treescope==0.1.9 typing_extensions==4.13.2 ubuntu-drivers-common==0.0.0 ubuntu-pro-client==8001 uc-micro-py==1.0.3 ufoLib2==0.13.1 ultralytics==8.3.116 ultralytics-thop==2.0.14 unicodedata2==14.0.0 urllib3==1.26.5 urwid==2.1.2 wadllib==1.3.6 Werkzeug==3.1.3 wrapt==1.14.1 wurlitzer==3.1.1 xdg==5 xkit==0.0.0 zipp==1.0.0

And by reproducing the same crash with:
LD_DEBUG=all onnxslim yolo11s.onnx yolo11s-slimmed.onnx

where the calling reference always pointed to the cpu-based, onnxruntime==1.21.1 instead of the Jetson GPU one: onnxruntime_gpu.

Simply removing the cpu one fixed the problem. :wink:

I didn’t try the most recent onnxruntime-gpu module from the Jetson JetPack 6 whl packages.

-ted