Can YOLO be trained using raw images? Because the purpose of training the model is to detect raw images. My raw images are typically 10-bit and 12-bit.
By default, Ultralytics YOLO currently only provides support for loading 8-bit (uint8) images. You can always clone the repo from GitHub and make modifications to the source code to process additional image formats, but this is not presently supported.
If you must keep 10/12-bit pixel values, my recommendation is that you convert your data prior to passing it to Ultralytics YOLO (for training or inference). During training, images are read from disk with the expectation that they will be 8-bit (pixel values in [0, 254]) but eventually converted into 32-bit floating point (fp32) values. So for training, you may need to convert your raw images to fp32 and resave to disk (rawpy might be a good option), then modify the Ultralytics dataloader to read fp32 data, that or you could put the conversion process into the training dataloader but that’s going to add a lot more overhead.
During inference, images are expected to read in as having 8-bit pixel values, but are converted to 16-bit floating point (fp16) values (by default, but can use fp32 when using the argument half=False). Here you can load your raw images and then pass the data into PyTorch with fp32 values, which can be provided directly to Ultralytics YOLO for inference.
For the fastest test, I would test out converting your image data to 8-bit, and training a model. If the truncation of data is minimal, as in there’s not a significant loss of detail in the objects you’re looking to detect, then the model will likely perform reasonably well given you have a sufficiently large annotated dataset. I can tell you from personal experience that converting 16-bit images to 8-bit for use in training a model worked reasonably well for extremely precise defect classification. There was some loss of detail, mostly imperceptible, but the images were only single channel and there was still sufficient information to distinguish various defects at a high precision.
Thank you for your detailed explanation.
I’m glad to hear that converting 16-bit images to 8-bit can still work quite well for extremely precise defect classification, as long as there is a sufficiently large annotated dataset.
Based on your suggestion, I’ve decided to start by converting my raw images to 8-bit and give that a try.
1 Like
Sounds like a solid plan, @kang_pan! Converting to 8-bit is often a practical starting point, especially given the broad compatibility with standard image processing libraries and tools.
Let us know how it goes or if you encounter any issues during the process. Good luck with your training!