angle detection of a particular object in a recorded video

Question

I am trying to find out the angle of the black stripe in the below photo for the entire recorded video. I was trying to find code online but was only able to find code for images or requiring manual input of drawing lines on image and finding angle. Do you know any projects or opensource code which is similar to this problem?

enter image description here

Tried to find opensource codes , found github projects on pose estimation and angle detection of objects in an image. Didn't find anything for recorded videos and object tracking

If you are able to segment the black stripe the you can use cv.minAreaRect to get the orientation. This works with many kind of objects. If shape isnt suitable bit texture is available, keypoints like SIFT can give you an orientation. — Micka, Jul 18 '23 at 05:24
Please provide enough code so others can better understand or reproduce the problem. — Community, Jul 18 '23 at 08:01
if you 1. capture a single click (1 user mouse click needed) inside the black strip of interest, then you can 2. build a raster that moves left/right up/down along adjacent pixels with nearly same colour. You tell the raster to stop when next pixel changes colour sharply. 3. once you have all pixels belonging to such strip it's easy to calculate attitude. — John BG, Jul 18 '23 at 19:01

score 1 · Answer 1 · answered Jul 17 '23 at 21:43

I think the first step would just be to first decompose your video into a series of images, then you could process them individually (e.g. following this method). You most likely wouldn't need to process them all as I would imagine you are interested in lean angle of the rider and I doubt you would be able to measure angle changes with timescales on the order of 1/frame rate with this setup.

Assuming you are just interested in the angle with respect to the horizontal or vertical of the video frame (I didn't see any other fiducials present) you could then use a Canny edge detection:

import cv2

img = cv2.imread("image.png", flags=0)
img_blur = cv2.GaussianBlur(img, (3, 3), 0, 0)

edges = cv2.Canny(image=img_blur, threshold1=100, threshold2=200)

cv2.imshow("Edges", edges)
cv2.waitKey(0)

Which would allow you to detect the edges of the stripe on the vest. You could use a region of interest to only consider a certain section of the frame and then look for the corners or edges and find the angle of those edges with respect to the horizontal.

I'm not sure if there is a drop in place method to do this as I think with other object tracking methods you would just be finding the centroid of that area without additional steps.

Thank you. I shall try this out and let you know asap – Doofenmirtz Mirtz Jul 23 '23 at 18:32 — Doofenmirtz Mirtz, Jul 23 '23 at 18:32

score 0 · Answer 2 · answered Jul 19 '23 at 17:01

1.- Input Image

A=imread('001.jpg');
hf=figure(1);
ax=gca
imshow(A);

2.- Adjusting Contrast

th_low=.3  % low contrast threshold
th_high=.7  % high contrast threshold

A2=imadjust(A,[th_low th_low th_low; th_high th_high th_high],[]);

Without adjusting contrast there's some 'ringing'. Showing what I call 'ringing' along a sample cross section

hold(ax,'on');
plot(ax,[130:190],200,'b*')

 figure(2);
 stem([130:190],A(200,[130:190],1),'Color','r');
 grid on
 hold on
 title('without contrast adjustment')
 stem([130:190],A(200,[130:190],2),'Color','g');
 stem([130:190],A(200,[130:190],3),'Color','b');

 figure(3);
 stem([130:190],A2(200,[130:190],1),'Color','r');
 grid on
 hold on
 title('with contrast adjustment')
 stem([130:190],A2(200,[130:190],2),'Color','g');
 stem([130:190],A2(200,[130:190],3),'Color','b');

The variance of a cross section of the are of interest is far lower along inside the are of interest, and the endges are sharper, after improving contrast

3.- Take Single Point reference inside area of interest

I know you'd like it without any interaction, who would disagree.

However, a single point is only to skip discerning the area of interest, the targeted strip.

Such sorting is not difficult, but laborious and I hope you agree upon the fact that such classification would need at least another question, for instance: That would be 'find among this segmented zones the helmet, the road limits, the byke .. until getting to the strip of interest.

figure(4)
ax4=gca
imshow(A2)
hold(ax4,'on');

print('click on 1 point inside area of interest');
[y0,x0]=ginput(1);

x0=floor(x0);y0=floor(y0);

plot(ax4,y0,x0,'g*');

Ar=A2(:,:,1);  % red
Ag=A2(:,:,2);  % green 
Ab=A2(:,:,3);  % blue

ar0=double(Ar(x0,y0))
ag0=double(Ag(x0,y0))
ab0=double(Ab(x0,y0))

er_r=.05;er_g=.05;er_b=.05;  % error threshold 1%

d1=5;  % NOT euclidean, ABS distance

nxr=1;nxl=1;nyu=1;nyd=1;

A0=[Ar(x0,y0) Ag(x0,y0) Ab(x0,y0)]

A_right=[Ar(x0+nxr,y0) Ag(x0+nxr,y0) Ab(x0+nxr,y0)]
A_left=[Ar(x0-nxl,y0) Ag(x0-nxl,y0) Ab(x0-nxl,y0)]
A_up=double([Ar(x0,y0+nyu) Ag(x0,y0+nyu) Ab(x0,y0+nyu)])
A_down=double([Ar(x0,y0-nyd) Ag(x0,y0-nyd) Ab(x0,y0-nyd)])

while abs(double(A0(1))-double(A_right(1)))<d1 && ...       % right
        abs(double(A0(2))-double(A_right(2)))<d1 && ...
        abs(double(A0(3))-double(A_right(3)))<d1 && ...
        y0+nxr<size(Ar,2)

        plot(ax4,y0+nxr,x0,'b*')
        nxr=nxr+1;
        A_right=[Ar(x0,y0+nxr) Ag(x0,y0+nxr) Ab(x0,y0+nxr)]
        plot(ax4,y0+nxr,x0,'r*')
end

while abs(double(A0(1))-double(A_left(1)))<d1 && ...      % left
        abs(double(A0(2))-double(A_left(2)))<d1 && ...
        abs(double(A0(3))-double(A_left(3)))<d1 && ...
        y0-nxl>1

        plot(ax4,y0-nxl,x0,'b*')
        nxl=nxl+1;
        A_left=[Ar(x0,y0-nxl) Ag(x0,y0-nxl) Ab(x0,y0-nxl)]
        plot(ax4,y0-nxl,x0,'r*')
end

while abs(double(A0(1))-double(A_down(1)))<d1 && ...      % down
        abs(double(A0(2))-double(A_down(2)))<d1 && ...
        abs(double(A0(3))-double(A_down(3)))<d1 && ...
        x0+nyd<size(Ar,1)

        plot(ax4,y0,x0+nyd,'b*')
        nyd=nyd+1;
        A_down=[Ar(x0+nyd,y0) Ag(x0+nyd,y0) Ab(x0+nyd,y0)]
        plot(ax4,y0,x0+nyd,'r*')
end

while abs(double(A0(1))-double(A_up(1)))<d1 && ...      % up
        abs(double(A0(2))-double(A_up(2)))<d1 && ...
        abs(double(A0(3))-double(A_up(3)))<d1 && ...
        x0-nyu>1

        plot(ax4,y0,x0-nyu,'b*') % check
        nyu=nyu+1;
        A_up=[Ar(x0-nyu,y0) Ag(x0-nyu,y0) Ab(x0-nyu,y0)]
        plot(ax4,y0,x0-nyu,'r*') % check
end

The resulting cross

the plot lines inside the while loops are just to make sure 'the cross' end up where it should.

What you call the 'orientation' can now be found further processing the upper side of 'the cross'. This is, it should be easy to find whether the upper side 'turns' left or right hence the 'visual orientation' of the driver, which is likely what you need to obtain just from the attitude of the driver's back marked with the strip.

In black & white it's easier, but now you have the possibility to directly look for charcteristic hi-vi vest fluorescent green to solve the corner I cut aiming with a single mouse click.

I would to further elaborate and add more script precisely developing these closing comments, but perhaps it's better to get some feed back first.

For the following steps, if you need further help just post another question and let me know the link to next question.

Thank you very much for the detailed explanation and the code. I will definitely try this out and let you know!! — Doofenmirtz Mirtz, Jul 23 '23 at 18:33

angle detection of a particular object in a recorded video

2 Answers2