YOLO- NONE object detection from thermal images

Hi there!

I am a Mining Engineering student. I am working on the research which is focused on AI-Driven Frameworks for Mitigating Rockfall Hazards. It is about detection of loose rock through thermal images in underground mining to prevent falls or other unfortunate situations.

I found out that YOLO can detect not just with shapes but also with colors in the pixels. Detection of traffic light, or fire can be as a prove.

Thermal images in underground mining also consist of RGB colors, for example, blue is unstable rock, which might fall, red is stable rock and so on. Every time the shapes of the unstable rock will be different, can YOLO still detect the unstable rock or blue color? And how accurate it can be? Thermal image will be taken with UAV drones.

I tried couple of tests, first made an annotation, then trained the YOLO, but when I tested with another thermal image and video, YOLO didn’t detect anything. Of course, it needs more training and more images. But I wanted to make sure that I am on the right way, or maybe some other additional methods would work with thermal images.

I appreciate if someone could share information about that.
Thanks)

It uses color. So as long as you have enough data, it should work.

Like Toxite mentioned, it should work. The key is that you’ll need a lot of images and annotations to get a result that will generalize well (meaning it works on data it’s never seen before). You’ll also need to be consistent with your annotations (make sure to label all instances of a class) and keep tight bounds around each object.

Hi, thanks for your reply,

I used square shaped annotation for thermal image, but it would be more precise to use canny edge detection. But do you think I can use edge detection from canny to train the YOLO model?
For the square shaped annotation, I had a labels with points of the square like;
0 0.915070 0.501111 0.169861 0.997778
0 0.045797 0.948049 0.091394 0.101518
1 0.498884 0.500000 0.476693 1.000000
1 0.143267 0.439390 0.283267 0.733469

can I get this kind of labels from canny edge detection and use to train YOLO?

Maybe. The question you should be considering first is, “am I searching for a problem to fit a solution?” So far in your posts, you’ve throw a lot of information out but it’s not clear or obvious what problem you’re looking to solve.

Think about the end goal, as in what’s the final project/product you’re targeting. Then map out what are the things that are required to deliver your target. For example:

Project: Detect when someone approaches a door, send a notification during specific times, and log all detections.
Required:

  • network camera with view of door
  • person detection model
    • start with pertained, custom train as needed
  • database to log activity
  • notification service (Gmail)
  • schedule for notifications
  • code for service (detect, log, and notify)
  • server to host service
  • (optional) dataset to improve detection
    • image/video data
    • data storage
    • annotations of people
    • annotation tool (bounding boxes or segmentation)
    • (additional details about what’s needed for annotations)

A breakdown in this manner will help guide your efforts. You can get more granular in your breakdown as needed.

You might wonder why I started with all of that for what sounds like a simple question. The issue is for someone disconnected from what you’re doing, I have extremely limited information about what’s going on. That’s not to say I need to know that kind of breakdown of your project, that’s 100% for you.

What information I do have is what you’ve posted here. You mentioned using thermal images for finding unstable rock, which is usually indicated by a specific color. You mentioned attempting to use a YOLO model for detection, but not seeing decent performance with only a few annotations. Recently you asked about using Canny Edge detection for annotations, mentioning it would be more precise than a square annotation and asking if you could train a YOLO model using Canny Edge detection data.

I suspect what you are looking to do is train a YOLO segmentation model. This is likely because the bounding boxes are not precise enough since you’re using thermal imaging which tends to have less detail of objects and contour boundaries.
Since thermal images usually consist of only a few different colors, if you can determine what the color is that represents the objects you’re looking to detect, you can likely select all of those pixels directly from the image. This might not require the use of a model to accomplish what you’re looking to do if the color is always there same. If the color might vary significantly and if there’s no way to identify the objects otherwise, the training a YOLO segmentation model might make more sense. To generate the annotations, you could try using the OpenCV function for finding contours, maybe try using a classical segmentation technique, or use a pretrained segmentation model like SAM2 to annotate your data.

Still, I recommend that you document out your project requirements first, as this can help anchor your efforts. Your idea of using Canny Edge detection isn’t a bad one at all, but it’s more important that you understand (and can communicate) what the goal is as opposed to asking if something will work. See this post to learn more about the XY problem which is a common occurrence when asking questions.

1 Like

Hi,
I appreciate your effort. You are right, that my questions are not concentrated enough and difficult to understand. Sorry about that, it is just I am new in this concept and project. But I will try to do what you mentioned and come back soon. Thank you!

1 Like

Hello Axror_Narmatov,

It’s great you’re exploring YOLO for your research on mitigating rockfall hazards.

YOLO primarily focuses on identifying patterns and features, which can include color variations like those in thermal images. The accuracy of detection will depend significantly on the quality and consistency of your training data. Since the shapes of unstable rocks vary, focus on collecting a diverse dataset that captures these variations in thermal signatures.

Regarding your initial tests, non-detection in new images is common after limited training. Continue expanding your dataset, and make sure annotations are consistent.

For using Canny edge detection outputs, YOLO requires bounding box or polygon annotations rather than edge maps directly. You might consider using Canny to help guide your manual annotations or explore instance segmentation, if detailed outlines are needed. Explore more about instance segmentation in Ultralytics documentation on segmentation.

Keep experimenting and refining your approach. Engaging with the community, such as through GitHub Issues can provide additional support and insights.