@Symbadian your dataset looks fine. Missing classes in a validation set will not cause any errors but also no metrics will be output for classes that have no labels.
If you want a histogram of your val labels you could switch your val and train sets in your data yaml, start training and then take a look at your labels.png file
Hey @Glenn, top of the morning pal, can you help me to understand what my confusion is with this aspect, please?
Isnāt the validation coming from the specified location?
for example when I investigate the
train: images/train2017
the data inside of that folder has no separation between what is set for training and validation.
it is one folder of dataā¦
So I followed the same perspective with my data for ( - images/Knife_DeployTr ) here
train: images/train2017
- images/Knife_DeployTr
What I am saying is, that I am not seeing a validation separation of the data for the original data in the images/train2017 folder.
Is this specified somewhere? and if yes can you guide me, Please??!!??
I have been searching all of the .py and configuration files for such a statement but thus far, my attempts have been futile.
I am sure that I am doing something wrong and my limited knowledge prevents me from detecting what that solution is. Please assist, really willing to learn about these operations.
Hi @Glenn, Some bit of information I am missing that is preventing me from understanding and I still donāt get it!
check my image,
the validation data is there as specified deliberately by BeatingpyTval both in the training and validations folder so that it does not confuse me further.
Maybe we can start here to bridge my gaps in understanding the processes. Hey @Glenn J, I am really really grateful for your time and efforts in trying to guide me in the right direction. Thanx a mill, immensely grateful pal!!
To add to my previous statement, pal, check the outputs from the processing the validation is being recognized but not outputted.
@Symbadian for guidelines on formatting YAMLs see Train Custom Data Ā· ultralytics/yolov5 Wiki Ā· GitHub
Splits are up to you, you define them in your data.yaml file.
Hi @Glenn, I am trying to get the understanding that I need to move forward and this is not getting me anywhere. With the link you provided, Iāve already past that stage. I have the data I want to use in the format that it should be in.
What I am trying to understand according to your instructions previously, is how to add the dataset to what is already there in coco128.yaml. I followed everything to the best knowledge I can but something is missing as the errors is continuously complaining about the dataset increasing from 80 to 81.
I would like to think that I edited all of the data files concerning the configurations conversion from 80 to 81. When that was done, I checked the dataset to see if that was in the right location and it is as per my screenshots above.
After this, I initialized the operations but I am not seeing the ONE ADDITIONAL CLASS in the verbose output. But this is shown in the confusion matrix output image with no numeric results.
The training and validation files are therein their locations or else they wonāt be showing up partially in the verbose output.
I am not the professional here, but do you think it has something to do with the training procedures?
The model is trained on the original 80 classes but somehow is not seeing the new class to make this 81 at this stage!!! I am not sure please assistā¦ really need your assistance here as this is a tad frustrating trying to get an understanding of the processes.
This is what I am trying to gain your experience on, Please?
Please guide meā¦
Thanx loads Sym
@Symbadian multi-directory training works correctly. You must format your dataset and YAML as in the above guides. Iāve already instructed you to follow GlobalWheat YAML format which you appear to not be doing:
Can we use 2 trained models of different classes inside one detect.py file?
Detection
Use the class(1) trained model to detect cars on the road in the image, crop that image of the cars save it, and then use the class(2) trained model(2) to detect people inside that car.
The result of one would be used for the input of 2nd one.
Is there any nested method to solve this?
Sorry @Glenn J, this was just an example of me explaining how the files are not separated as a folder for validation and a folder for training within the images/train2017 folder.
This is how I have it structured now but the same problem persists;
path: ../datasets/coco128 # dataset root dir
train:
- images/train2017
- images/Knife_DeployTr
val:
- images/train2017
- images/Knife_DeployVal
test: # test images (optional)
# Classes
nc: 81 # number of classes
names: ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',
'hair drier', 'toothbrush', 'Knife_DeployTr'] # class names
I am not certain if the train.py is causing the issues as somewhere it is not pulling the extra class āKnife_DeployTrā in the training sessions. hence the output is continuously recalling the 80 classes rather than 81ā¦???
this is my thinking I am sure that I am wrong here?!??!
I got through, with this issue! I reassemble the dataset by merging the classes in roboflow. That provided consistency with the labels and had a positive impact on the entire operations immediately. Thanx for all of your efforts and guidance.
Cheer
Sym
Hello sir! i have an problem thats my code as u said. I want to change that class 0 to 1 (i have an another trained model which has 13 class so iāll ad one more class as 14 so i need to change that 0 to 14 then i can merge them) as ur say but i couldnāt manage it can u help me ? thanks.
Hello again, iāve solved my problem. by that u can change the class as u want (for only one line but u can multiple for lines which is f.readlines() then line[0] =ā1ā line[5]= ā3ā
Hi, cross-dataset training is a intersting topic and I have a question.
If I have two dataset label cat and dog respectively, the cat dadaset contains dog but without label, and the same goes for the dog dataset. When train using the setting above, the dog prediction in cat dataset are judged to be wrong because the dogs havenāt been labeled. These will lead to incorrect loss calculation and damage the network performance.
Is there a solution to this problem? @Glenn @KalenMichael
You should label the respective classes that are missing in each dataset. The alternative is to create two single class models, e.g. catModel.pt and dogModel.pt. Then run dogModel.pt as inference against your cat dataset to annotate the dogs and run catModel.pt against your dog dataset to label the cats. Once all cats and dogs are labeled, retrain a two class model.
Hi @Glenn, I have a question about adding a new class to a existing model.
I first trained a model that can only detect dogs on about 5000 images, and it performs well (mAP 0.5 is around 0.9).
Then I took the weights (runs/train/dog/weights/best.pt) and trained them on 8 new cat images.
my new data.yaml file looks like this:
train:
nc: 2
names: [ādogā,ācatā]
I trained the weights on 8 cat images for only 1 epoch, and since my batch_size is bigger than 8, there is only 1 batch and the weights only get updated once during the training, but the model can no longer detect dogs at all. the mAP0.5 for dogs drops from 0.9 to 0.0005.
I do not know why this happened, I was expecting the model to be not as good at detecting dogs as before but not completely forgets about dogs in just one update.
Am I missing anything here? I also tried freezing backbone layers and freezing all layers but the results are still bad.
I want to create a object detection model using yolov8 or yolov5 and train it on my custom dataset. here are few problems i ran into
Can anyone please tell me a way such that I can retain the pretrained weights of yolov8/v5 and train it on a custom dataset such that the new model can detect classes for coco+custom dataset classes.
Thanks in advance
I will avhieve this kinda model actually, I have pretrained model with only 2 classes and I have dataset that has different 7 classes than pretrained model and I want to just use datasetās labels at the end, but in that case do I really take advantage of the pretrained model in transfer learning? Or is this transfer learning?
Hi! We are moving the Ultralytics community to Discord. To receive support for your questions, problems, issues, etc. please join us on the new server: Ultralytics
Hey, qq: are the labels case sensitive? Trying to achieve a cross-datasets training and I have some ( kind of ) overlapping labels but in different cases. How does this work?
thank you!
Hi @Glenn
I have DATASET A with class ANIMAL, I have DATASET B with class MACHINE,and I have DATASET C class PLANT.
Question1 : Can I train 3 models independently(on different computers), one on each dataset?
Question2 : Can I train each model on dataset making each class index is 0 ?
Question3 : Can I Combine Weights to Detect each class? Or donāt combine weights to detect each class.
