Post Your Answer

Why do we face flickering issues in the bounding box during object detection?

2 years ago in Image Processing By Shahruddin Jafri

Algorithms

Hi everyone, my question to you all is why the bounding box flickers during the object detection only when it is live streaming and not during the object detection in the image? Could it be an algorithm issue that can be fixed?

I am using the YOLOv5s model which is based on the ONNX hardware system. I want to know if I can fix the flickering issue by working on an algorithm or should I just use some other method for object detection.

All Answers (3 Answers In All) Post Your Answer

By Finn Answered 2 years ago

There could be a few reasons which can cause the bounding boxes to flicker. Lighting conditions, variety, speed, occlusion, deformation, background (whether clustered or textured), and different viewpoints. Considering that the object detection in an image is fine, then the cause could be speed, deformation or most probably the difference in the viewpoint. It is because an object might not look the same from all angles. For example, a pyramid may look triangular from the sides but may look square. For further reference, I am attaching a link here. You can look through it. https://www.exposit.com/blog/computer-vision-object-detection-challenges-faced/

Replied 2 years ago

By Shahruddin Jafri

Yes Finn, this I know. I am actually looking to see if it can be solved with algorithms or something. Anyways, thank you for the answer.

Reply To Finn

By Sahil Answered 2 years ago

Hi Shahruddin, I would like to point out that your point of comparison between an image and live video is wrong on so many measures. No matter if you are using YOLO or any other algorithm for that purpose. When it comes to object detection in an image, you are dealing with a static box. There is no motion in the object when it is in an image. In the case of live streaming, the situation is different. In the live stream, here are successive frames. This means that the ROIs (objects) are moveable (in motion), and so is the case with the camera. In such a case, the algorithms focus on the ROIs as well as the box boundaries which are also known as confidence. This change in focus can make the boundary boxes flicker. To check whether this is the issue or not, you can put your camera on a fixed position and use an object that doesn’t move. See if the flickering happens still. If that is the case, then as Finn mentioned above, it could be one of the other reasons that are causing the boxes to flicker.

Replied 2 years ago

By Shahruddin Jafri

Hi Sahil, thank you for your answer. I agree with the points you have mentioned but I have a different situation. Let me explain it further for a better understanding. I tried it in two ways: One where I used the test application of the EVK board. In this I ran the test through an image sensor and used it during the live stream. In this case, the flickering issue was here. Next I used RTSP and input the live streaming to the python code I developed in my laptop from the evk board. When I did that, there was no flickering issue. The model I used in both these cases remains the same; i.e.YOLOv5s. Due to this I wonder if there is some algorithm issue and maybe, by changing the codes, it can be solved. I hope I made myself and the question clearer, So, please suggest some solutions in this regard. I will be looking forward to your response.

Replied 2 years ago

By Sahil

Oh okay, I see. Maybe then the problem is not related to the ones mentioned above but it could be an issue occurring during the acquisition stage. Maybe the sensor or the program deployed on your EVK board is causing the issue. Check your electronics or maybe the program on which you are running the tests. I am sorry but I don’t know much about this and could not be of help. But again, if the second way that you have explained is providing you an output where there is no flickering in the boxes, then why are you not going with that one? Isn’t it the one you will be using for processing and analysing the data and objects? Well, in any case, I can suggest you experts who can help. Maybe they can help you better in this situation. http://www.fivevidya.com

Replied 2 years ago

By Shahruddin Jafri

Yes, I definitely agree that the issue could be in the program. That is why I am trying to see what could be the possible reason. Also, it is not that I am not using my code but more like I can’t use it. The reason is that we have been provided some test applications which we need to check. The codes in these applications have already been given in the evk board and so, I am trying to solve the issues on it first. Using the code I created is like my last option in case I fail to figure out and solve the issue. Thank you for the reference and contributing your time in the discussion. I really appreciate it.

Reply To Sahil

By Lisa Hedgewell Answered 2 years ago

Hello Dear, I don’t have much to say about how you to get the algorithms right but I would like to share a paper with you. Soft-NMS - Improving Object Detection with One Line of Code Actually recently I reviewed a paper based on stabilizing the Non-Maximum Suppression (NMS), and just like you asked, the author of that paper also pointed out the fact that it is mainly the issue with algorithm coding which leads the bounding boxes to flicker continuously. In that paper, the author pointed out the ways of stabilizing it too but because it is not published yet, I can not name the paper or even share the link with you. The paper given above was one of the major references used in that paper. I hope it turns out to be helpful for you too. Best wishes.

Replied 2 years ago

By Shahruddin Jafri

Hello Lisa, thank you so much for sharing the reference. I will definitely go through it once.

Reply To Lisa Hedgewell