I’m training the YOLOx v12 Ultralytics model on my custom dataset. I have created my yaml file (18 classes of fish species) and have my train/test/val directory structure as needed by the model. In the directories, I have the images and associated text files (class, x1, y1, x2, y2). After training the model, I’m using the best.pt model in the runs directory to predict fish species on a video stream. All of this works fine.
What does not make sense to me are the results I get. When I see the video generated by the trained model’s predictions, I see fish being identified as classes in the COCO dataset. So, an example would be a fish identified as a bird or kite. If I provided the yaml file with no reference to any of the COCO dataset classes, why would the prediction still refer to any of the COCO categories? Any help clarifying this? Thank you much,
Ralf
Code snippet:
# train downloaded model
print('training model...')
model = YOLO("yolo12x.pt")
results = utils.train_model(model, epochs=300, imgsz=640, yaml_file="data/data.yaml") #comment out if no retraining
model_pred = YOLO('runs/detect/train/weights/best.pt') # model I'm using for predictions
Can you provide the actual training code? You’re using a method that’s not part of Ultralytics and you haven’t posted the code for it either. It doesn’t show what your actual command for training is.
Of course. Below is the training code I’m having the issue above. Again, the issue is that the trained model (on my custom dataset) is still predicting COCO classes, none of which is in my yaml file (snippet is shown above).
# This is a model to extract fish biomass and richness/biodiversity of reef ecosystems
import pandas as pd
import numpy as np
import os
from pathlib import Path
import torch
import cv2
import csv
from ultralytics import YOLO, solutions, SAM
def train_model(model: object,
epochs: int = 300,
imgsz: int = 640,
yaml_file: str = None) -> None:
"""
Train the YOLO model
Parameters:
model (object): The pre-trained model downloaded from Ultralytics.
epochs (int): The number of training epochs.
imgsz (int): The image size.
yaml_file (str): The path to the customized yaml file formatted according to YOLO.
Returns:
None.
"""
results = model.train(data=yaml_file,
epochs=epochs,
imgsz=imgsz,
cfg='yolov12x.yaml',
weights='yolo12x.pt',
device=[0])
def gen_model_results(trained_model: object,
previous_run_results: object,
model_data_file: csv,
source_image: str = None,
source_video: str = None) -> None:
"""
Generate a csv file with pertinent results for a trained model.
Parameters:
trained_model (object): The trained model (training based on custom dataset of annotated fish images).
previous_run_results (object): The 'results' object returned from previous training run.'.
model_data_file (csv): The csv file to save the trained model results in.
source_image (str): The path to a prediction image.
source_video (str): The path to a prediction videl.
Returns:
None.
"""
# run inference on the source image/video
# result = trained_model(source=source_image) # uncomment for single image prediction
results = trained_model(source=source_video)
# Open the CSV file in write mode ('w') or append mode ('a')
with open(model_data_file, 'w', newline='') as file:
# Create a writer object
writer = csv.writer(file)
writer.writerow(['x1', 'y1', 'x2', 'y2', 'confidence', 'class', 'frame'])
frm = 0
for res in results:
boxes = res.boxes.cpu().numpy()
frm += 1
# Write the row to the CSV file
for box in boxes.data:
out = np.frombuffer(box.data, dtype=np.float32) # box.data is a memoryview data type
out = np.append(out, frm)
writer.writerow(out)
if __name__ == '__main__':
print('Training task started...')
#set some vars
source_video = 'videos/ES032024RV01_V1_GX020106.MP4'
dest_video = 'reef_fish_counted.mp4'
source_image = 'moorish-idol.jpg'
dest_image = 'moorish_idol_ided.jpg'
dest_image_dir = 'frames/'
#train downloaded model
print('training model...')
model = YOLO("yolo12x.pt") #in my main directory (downloaded from Ultralytics)
train_model(model, epochs=300, imgsz=640, yaml_file="data/data.yaml")
model_pred = YOLO('runs/detect/train/weights/best.pt')
results = model_pred(source=source_video)
#generate model results
print('generating model results...')
model_data_file = 'model_results.csv'
gen_model_results(model_pred,
results,
model_data_file,
source_image,
source_video)
print('done')
You should remove the cfg and weights args. cfg is not for model yaml. And weights isn’t an actual argument. I am not sure how the training even works because it should throw an error due to the non-existent argument (maybe it doesn’t and you’re loading an old checkpoint which explains it showing COCO classes).
Also the best.pt would not always be in the runs/detect/train folder because the train folder gets incremented each run unless you use exist_ok=True.
Sorry, the config argument was a holdover and is not used. Thank you for the tip on overwriting the runs directory. As I was running the detection for the trained model, it indeed kept incrementing the runs. I did, however, always use the latest trained model in the runs/detect/train{latest_training_run}/weights directory.
Assuming that the model got trained and I’m using the correct ‘best.pt’ file, would you have any other hints as to why it still classifies COCO images? Thx again for your assistance,
It wouldn’t show COCO classes if those assumptions are correct. Either it didn’t train on the correct dataset or the best.pt is wrong. What are the classes shown in the train_batch and val_batch jpg files inside the train directory?
OS Windows-11-10.0.26100-SP0
Environment Windows
Python 3.12.6
Install pip
Path C:\Users\USA\AppData\Local\Programs\Python\Python312\Lib\site-packages\ultralytics
RAM 255.52 GB
Disk 1096.8/7452.0 GB
CPU Intel Xeon w9-3495X
CPU count 112
GPU NVIDIA RTX 6000 Ada Generation, 49140MiB
GPU count 1
CUDA 12.4
I might have figured out what the issue was. See the code below. I was tinkering with the ObjectCounter method to extract information on number of objects by class to estimate species richness. When I instantiate the counter I’m using oc.ObjectCounter(), but nowhere I initialize ‘oc’. It must be inheriting the ‘oc’ object from the model trained on the COCO dataset or something. What I need to do is to link the counter object to my trained model. Once I figure that out, I’m good to go. Anyhow, I’d like to thank you for your help. Without your input, I’d still be spinning my wheels. All the best.
frame_num = 1
object_counter = oc.ObjectCounter()
[f.unlink() for f in Path(dest_image_dir).glob("*") if f.is_file()]
while video_capture.isOpened():
success, frame = video_capture.read()
if not success:
break
tracks = trained_model.track(frame, persist=True, show=False, classes=[0, 2])
# frame = object_counter.start_counting(frame, tracks)
res = object_counter.process(frame)
res_dict = res.classwise_count
video_writer.write(frame)
cv2.imwrite(os.path.join(dest_image_dir + f"frame_{frame_num}.png"), frame)
frame_num += 1
I think you figured it out already, but just following up for completeness. The ObjectCounter class inherits from the BaseSolution class, which if not provided a model weights file during initialization, will use a COCO pretrained model by default.
You should be able to use your model by updating your code:
You’re on the right track! The ObjectCounter solution indeed needs to be initialized with the path to your custom-trained model. If you instantiate it without specifying a model argument, like solutions.ObjectCounter(), it defaults to using a standard pretrained model (e.g., yolo11n.pt), which is trained on the COCO dataset.
To use your custom model for counting, you should pass the path to your best.pt file when creating the counter object, for example:
from ultralytics import solutions
# Initialize ObjectCounter with your custom model
counter = solutions.ObjectCounter(model='runs/detect/train/weights/
You’re on the right track! The ObjectCounter solution indeed needs to know which model to use.
If you initialize it like solutions.ObjectCounter(), without specifying the model argument, it defaults to using yolo11n.pt, which is trained on the COCO dataset. This explains why you were seeing COCO class predictions.
To use your custom model, you need to pass the path to your best.pt file during initialization:
from ultralytics import solutions
# Initialize the counter with your custom model
counter = solutions.ObjectCounter(model='runs/detect/train/weights/best.pt', region=your_region_points)
# Then use it in your loop
# ...
results = counter(im0) # Pass the frame to the counter instance
# ...
Also, when using model.track(), ensure the classes argument uses indices corresponding to your custom 18 classes (0-17), not COCO indices like [0, 2], unless those happen to be the specific fish classes you want to track from your custom list.
We’re glad to hear that the information was helpful and that your results are improving. The credit goes to your diligent work and the collaborative spirit of the YOLO community.
Python-3.12.6 torch-2.4.1+cu124 CUDA:0 (NVIDIA RTX 6000 Ada Generation, 49140MiB)
(112 CPUs, 255.5 GB RAM, 1096.8/7452.0 GB disk)