1

I am using a DirectShow filtergraph to grab frames from videos. The current implementation follows this graph:

SourceFilter->SampleGrabber->NullRenderer

This works most of the time to extract images frame by frame for further processing. However I encountered issues with some videos that do not have a PAR of 1:1. These images occur stretched in my processing steps.

The only way to fix this I have found for now is to use a VMR9 renderer in windowless mode that uses GetCurrentImage() to extract a bitmap with the correct aspect ratio. But this method is not very useful for continuous grabbing of thousands of frames.

My question now is: what is the best way to fix this problem? Has anyone run into this issue as well?

japedo
  • 368
  • 3
  • 15

1 Answers1

1

Sample Grabber gets you frames with original pixels. It is not exactly a problem if there is aspect ratio attached and the pixels are not "square pixels". To convert to square pixels you simply need to stretch the image respectively. It would be easier for you to do this scale step outside of DirectShow pipeline, and you have all data you need: pixels and original media type. You can calculate the corresponding resolution with square pixels and resample the picture.

Roman R.
  • 68,205
  • 6
  • 94
  • 158
  • I guess you are the author of http://alax.info/blog/? This page helped me a lot! Indeed I thought about your proposed method and already tried to build a way to retrieve the true PAR by reading the videos frame aspect ratio, its resolution and its format. Just thought there would be a more elegant way to solve this. Thank you! – japedo Jan 08 '15 at 09:20
  • A more elegant way would be to embed scaling into filter form factor and have a filter that scales to square pixels. To some extent, stock Video Resizer DMO does this, but I suppose there might be certain problems using it directly. Since you are using Sample Grabber + Null Renderer, it suggests that you want to escape DirectShow pipeline as soon as possible, in which case scaling outside is more appropriate. – Roman R. Jan 08 '15 at 09:34
  • Thank you again! I may try this but... first back to your first answer: I am struggling here. The way I did it before (very ugly) is: open the video with VMR9, retrieve resolution (1440x1080), calulcate FAR from this resolution (4:3), read PAR from VIDEOINFOHEADER2, and if FAR and PAR don't match calculate the horizontal stretching factor that results (in my case) in a resolution of 1980x1080. After that I opened it with my actual filtergraph. – japedo Jan 08 '15 at 09:43
  • This first approach is obviously ugly and very likely wrong. So how would you get all the required information from VIDEOINFOHEADER of the sample grabber? All I can get for now is the resolution and from there calculate the frame aspect ratio. Where would you extract the key information that leads to the horizontal stretching factor? – japedo Jan 08 '15 at 09:44
  • Media type with aspect ratio does not fit `VIDEOINFOHEADER` structure. Instead, [`VIDEOINFOHEADER2`](http://msdn.microsoft.com/en-us/library/windows/desktop/dd407326%28v=vs.85%29.aspx) is typically used target AR is different from simply pixel width/height ratio. However `VIDEOINFOHEADER2` is not supported by standard Sample Grabber, and your ability to connect all together suggest that you have other conversion in the pipeline, custom grabber or legacy media type without AR. There should be something not mentioned. – Roman R. Jan 08 '15 at 09:54
  • `VIDEOINFOHEADER2`'s AR and pixel resolution - this is sufficient for calculations (you either stretch up along "shorter" dimension, or stretch down the only which is too long). As I mentioned, on Sample Grabber media type AR information might be however already stripped. Basically, it is not a problem because you can check AR on some upstream media type, but you need to figure out how at all all your filters are connected provided that stock SG is unable to accept `VIDEOINFOHEADER`. – Roman R. Jan 08 '15 at 09:57
  • Thats exactly the problem: SampleGrabber does not accept VIDEOINFOHEADER2. To circumvent this I used two filter graphs: (1) Source->VMR9 and (2) Source->SampleGrabber->NullRenderer. The first one is to retrieve VIDEOINFOHEADER2 and the second one to do the actual grabbing. That is what I meant when I said "ugly". I read serveral times somewhere on the net that you can extend the SampleGrabber to accept VIDEOINFOHEADER2 but I could not find examples. Information about PAR on the net is really rare :) – japedo Jan 08 '15 at 10:01
  • Your method only works if Source can connect with both VIH and VIH2, or you have a filter in the system which can automatically convert VIH2 to VIH. Both ways it is somewhat unreliable in terms that it might stop working in different environment. A "proper" solution is pretty clear: a custom filter which does the transformation in question. As you try to build it without custom filter, using available filters only, I suppose you cannot make it much more elegant compared to what you have now. – Roman R. Jan 08 '15 at 10:37