# Total execution time of \`model.predict()\` is way higher than inference time

**URL:** <https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365>\
**Category:** Discussion\
**Tags:** code\
**Created:** [August 18, 2025, 10:06am UTC](https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365 "2025-08-18T10:06:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kallinteris-Andreas](https://sea1.discourse-cdn.com/flex001/user_avatar/community.ultralytics.com/kallinteris-andreas/32/838_2.png) [@Kallinteris-Andreas](https://community.ultralytics.com/u/Kallinteris-Andreas)\
**Post date:** [August 18, 2025, 10:06am UTC](https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365/1 "2025-08-18T10:06:59Z")

</div>

Given this simple benchmark of inference speed

```py
import os
import time
import psutil
from PIL import Image
from ultralytics import YOLO

detector = YOLO("yolo12m.pt", "cpu")

image_path = "debug/images/face_4096.jpg"
image = Image.open(image_path).convert("RGB")

process = psutil.Process(os.getpid())

while True:
    start_time = time.time()
    results = detector.predict(image)
    end_time = time.time()

    print(f"Detection took: {(end_time - start_time) * 1000:.4f} milliseconds")

```

i get a sample output of

```sh
...
0: 640x640 1 person, 2 chairs, 52.4ms
Speed: 3.4ms preprocess, 52.4ms inference, 1.2ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 226.1436 milliseconds

0: 640x640 1 person, 2 chairs, 52.4ms
Speed: 3.3ms preprocess, 52.4ms inference, 1.1ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 227.5164 milliseconds

0: 640x640 1 person, 2 chairs, 52.5ms
Speed: 3.7ms preprocess, 52.5ms inference, 1.3ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 226.4342 milliseconds

0: 640x640 1 person, 2 chairs, 52.5ms
Speed: 3.4ms preprocess, 52.5ms inference, 1.2ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 231.9701 milliseconds

0: 640x640 1 person, 2 chairs, 52.4ms
Speed: 3.7ms preprocess, 52.4ms inference, 1.3ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 235.8549 milliseconds

0: 640x640 1 person, 2 chairs, 52.4ms
Speed: 3.8ms preprocess, 52.4ms inference, 1.3ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 234.9603 milliseconds

0: 640x640 1 person, 2 chairs, 52.5ms
Speed: 3.6ms preprocess, 52.5ms inference, 1.2ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 232.1908 milliseconds

0: 640x640 1 person, 2 chairs, 52.4ms
Speed: 3.8ms preprocess, 52.4ms inference, 1.4ms postprocess per image at shape (1, 3, 640, 640)
Detection took: 237.7448 milliseconds
...

```

here we can see that the total execution time (~225-230ms) is much longer than the total process + inference speed time (~56-57ms)

Note: I have tested with `stream=True` argument, and I get the same behavior

---

<div class="post-metadata">

**Author:** ![BurhanQ](https://sea1.discourse-cdn.com/flex001/user_avatar/community.ultralytics.com/burhanq/32/7_2.png) [@BurhanQ](https://community.ultralytics.com/u/BurhanQ)\
**Post date:** [August 18, 2025, 10:47am UTC](https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365/2 "2025-08-18T10:47:10Z")

</div>

The major difference is that you’re measuring at the application level, not at the execution level of the code. That difference includes a lot of additional overhead, which very easily will increase the time measured.

The times reported by the `ultralytics` library are measured here:

> <https://github.com/ultralytics/ultralytics/blob/bc62fa95aec22e70910459de750830fbb69a989d/ultralytics/engine/predictor.py#L324-L337>

and the `Profile` class used to measure can be found here:

> <https://github.com/ultralytics/ultralytics/blob/bc62fa95aec22e70910459de750830fbb69a989d/ultralytics/utils/ops.py#L18>

---

<div class="post-metadata">

**Author:** ![Kallinteris-Andreas](https://sea1.discourse-cdn.com/flex001/user_avatar/community.ultralytics.com/kallinteris-andreas/32/838_2.png) [@Kallinteris-Andreas](https://community.ultralytics.com/u/Kallinteris-Andreas)\
**Post date:** [August 20, 2025, 3:02pm UTC](https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365/3 "2025-08-20T15:02:13Z")

</div>

After reading `predictor.stream_inference` it is not clear to me what takes all those milliseconds outside of the `preprocess`, `inference`, `postprocess` functions

Regardless, is there a way to speed up that code, or is it just expected

Note: I have found a lot of information for speeding up `inference` by exporting to other format such as `tensorRT` and `ONNX` and quantizing etc, but no information on speeding up the surrounding code.

---

<div class="post-metadata">

**Author:** ![Toxite](https://sea1.discourse-cdn.com/flex001/user_avatar/community.ultralytics.com/toxite/32/123_2.png) [@Toxite](https://community.ultralytics.com/u/Toxite)\
**Post date:** [August 20, 2025, 3:14pm UTC](https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365/4 "2025-08-20T15:14:57Z")

</div>

You can try bypassing the intermediate steps and perform inference directly:

> <https://github.com/ultralytics/ultralytics/issues/14855#issuecomment-2262134660>
>
> \### Search before asking
> 
> \- \[X\] I have searched the Ultralytics YOLO \[issues\](…https://github.com/ultralytics/ultralytics/issues) and \[discussions\](https://github.com/ultralytics/ultralytics/discussions) and found no similar questions.
> 
> 
> \### Question
> 
> simple code :
> \`\`\`
> import time
> import sys
> import cv2
> from ultralytics import YOLO
> pos\_model = YOLO('models/yolov8n-pose.onnx', task='pose')
> 
> cap = cv2.VideoCapture(
> 'rtsp://xxx:xxxx!@0xxxx:1xx77/stream1')
> 
> ret, frame = cap.read()
> 
> st = time.time()
> 
> results = pos\_model.predict(
> frame, conf=0.5, iou=0.6 )#, imgsz=640, verbose=False)
> print(time.time()-st)
> \`\`\`
> 
> 
> 
> Issue :
> output showing 
> Ultralytics YOLOv8.1.34 🚀 Python-3.11.9 torch-2.0.1+cpu CPU (Intel Xeon Platinum 8370C 2.80GHz)
> Setup complete ✅ (64 CPUs, 256.0 GB RAM, 92.6/126.4 GB disk)
> Loading models\\yolov8n-pose.onnx for ONNX Runtime inference...
> 
> 0: 640x640 (no detections), 46.5ms
> Speed: 60.5ms preprocess, 46.5ms inference, 1.0ms postprocess per image at shape (1, 3, 640, 640)
> 1.9199035167694092
> 
> what issue is there 
> as per 60.5ms +46.5ms +1.0ms =0.108 sec 
> but next line printing 1.91 sec 
> 
> \### Additional
> 
> I am running this code in azure windows vm in cpu

---

<div class="post-metadata">

**Author:** ![Kallinteris-Andreas](https://sea1.discourse-cdn.com/flex001/user_avatar/community.ultralytics.com/kallinteris-andreas/32/838_2.png) [@Kallinteris-Andreas](https://community.ultralytics.com/u/Kallinteris-Andreas)\
**Post date:** [August 20, 2025, 3:41pm UTC](https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365/5 "2025-08-20T15:41:13Z")

</div>

Thanks, your response let me to a path of testing the source type

turns out a lot of the time is spent converting `Pil.Image` to `np.ndarray` every time `predict` was called

```py
image = np.array(image)

```

I originally assume data type conversion would be handled by the `preprocess` function

---

<div class="post-metadata">

**Author:** ![pderrenger](https://sea1.discourse-cdn.com/flex001/user_avatar/community.ultralytics.com/pderrenger/32/73_2.png) [@pderrenger](https://community.ultralytics.com/u/pderrenger)\
**Post date:** [August 21, 2025, 1:27pm UTC](https://community.ultralytics.com/t/total-execution-time-of-model-predict-is-way-higher-than-inference-time/1365/6 "2025-08-21T13:27:40Z")

</div>

Great find — source type matters a lot.

Ultralytics does handle conversions in preprocess, but giving it a NumPy array or Torch tensor avoids repeated PIL→NumPy work and path/loader setup each call. For tight loops, pass a preconverted np.ndarray (HWC, BGR uint8) or a preallocated torch.Tensor (BCHW, float32 in [0,1]) and reuse the same YOLO instance so predictor/dataset aren’t rebuilt. Also disable extras like save/show/verbose.

Minimal example:  
from ultralytics import YOLO  
import cv2, torch

model = YOLO(“yolo11n.pt”) # recommend YOLO11 for best performance  
im\_bgr = cv2.imread(“debug/images/face\_4096.jpg”) # np.ndarray

# or: im\_t = torch.from\_numpy(im\_bgr[…, ::-1]).permute(2,0,1)[None].float()/255

for \_ in range(1000):  
\_ = model.predict(im\_bgr, stream=False, verbose=False, save=False, imgsz=640)

If you still see large gaps vs. the internal speed breakdown, try stream=True for generators on videos/dirs, and profile outside of preprocess/inference/postprocess; some overhead can come from dataset setup, callbacks, plotting/logging, and Python loop timing. Reference: predictor preprocess and call flow are documented in the Ultralytics engine predictor docs. See the Ultralytics Predictor reference for preprocess and **call** details.