Best approach for a system that detects acts of violence/ violent behaviour from cctv footage

Discussion
, ,
1

This project has been on my mind for a while now but it probably is far from my technical abilities i would just like to know the best approach

Thanks

2

It’s challenging because violence is temporal, as in you can’t rely on a single frame to determine it. You need to rely on multiple frames. On top of that, it is anomalous. Anomalous events by their nature have data scarcity issue. But you don’t just need images of violence, you need clips which is even harder to get. And you don’t just need few clips, you need multiple, hundreds if not thousands for it to generalize well. Not just clips of the same thing from different angle. But different environment, lighting conditions, etc.

So if you don’t have data, this is very challenging. It’s not about approach, you need data first.

The best you could do without data is combine YOLO as preliminary detection of violent poses and use a VLM as the filter to determine if there’s violence taking place. VLMs are large and expensive and do not run real time. But they are the best at doing this zero shot without data. You would still need some data here to train the violent pose detection model. But in general, it will flag anything like extended or raised hand as violence since it’s just relying on single frame, so you need the VLM to filter those.

1 Like