[gst-devel] proposal: support for row-stride in gstreamer

[Rob Clark]

On Aug 4, 2009, at 4:49 AM, Jan Schmidt wrote:

> Hi,
>
> On Thu, 2009-07-30 at 11:45 -0500, Rob Clark wrote:
>> so, my next steps (at least my current thinking about next steps) are
>> to start adding some rowstride related stuff in gst-plugins-base:
>>
>>
>> gst-libs/gst/video/video.c:
>> 1) add gst_video_format_get_size_strided()
>> 2) change gst_video_format_parse_caps() to understand strided caps,
>> and add a gst_video_format_parse_caps_strided() function  (it seems
>> there are enough places already using gst_video_format_parse_caps()
>> that I probably don't want to change the signature of this function)
>>
>> and then add a GstStrideTransform element under gst/stride
>
> I haven't put enough thought into designing a stride system to say
> anything for certain. Particularly, I'm not sure whether we'll be able
> to safely and successfully integrate it into the 0.10 series without
> thinking about it harder.
>
> I am pretty sure, however, that adding a separate stride adjust  
> element
> is the wrong way to go. None of the existing pipelines will include  
> it,
> so it would no be useful without changes to the applications/playbin
> etc.
>
> My hunch is that it will be better to add a new 'video/x-raw-yuv-full'
> format and add stride as a new parameter, and adjust  
> ffmpegcolorspace to
> support conversions to/from the unstrided/strided formats.


Hmm.. my idea for stridetransform was mainly just an element for  
testing and debugging with manually constructed gst-launch pipelines.   
We need some way to run codecs both with and without rowstide and  
verifying the output.  For example:


   ... ! decoder ! video/x-raw- 
yuv,format=(fourcc)YUY2,width=320,height=240,framerate=30/1 ! filesink  
location=file1.dat

and then

   ... ! decoder ! video/x-raw-yuv- 
strided 
,format 
=(fourcc)YUY2,width=320,height=240,rowstride=700,framerate=30/1 !  
stridetransform ! video/x-raw- 
yuv,format=(fourcc)YUY2,width=320,height=240,framerate=30/1 ! filesink  
location=file2.dat


But if you think this should be part of a normal pipeline, then I  
think it would make sense to merge in with the colorspace conversion,  
so you have one memory copy instead of two.

But do you think this is required?  At least in the cases that I have  
in mind, the video sink will also support non-strided buffers, but  
will be falling back to a less optimal mechanism.


------

btw, at the risk of starting a bikeshed discussion, is 'video/x-raw- 
yuv-full' preferred to 'video/x-raw-yuv-strided'?  So far I've been  
using the latter, but I don't mind changing the code that I've written  
so far.  Should the functions added to video.c have the _full suffix  
instead of _strided (ie. gst_video_format_parse_caps_full() instead of  
gst_video_format_parse_caps_strided())?

If we go with -full, are there any other fields we should add to the  
caps at the same time?  Offhand, the only thing I can think of that is  
a mandatory field would be rowstride, but I haven't thought too much  
about, for example, interlaced buffers.

------

fyi, I've begun making some changes in my private tree at http://github.com/robclark/gst-plugins-base 
  ... but nothing that I've changed so far is set in stone.  But we do  
have to start adding support in the codec and other elements that  
we'll use pretty soon, so comments and suggestions now are greatly  
appreciated.  We can stay on our own tree, or a special rowstride  
branch, for now if integration to master is post-0.10.  But it would  
be nice to not have to *completely* re-write things later ;-)



BR,
-R