IMX Scaler / CSC m2m driver.

[Ian Molton]

On 25/03/15 15:33, Nicolas Dufresne wrote:
> Le mercredi 25 mars 2015 à 15:04 +0000, Ian Molton a écrit :
>> Oh, some more info:
>>
>> My best (so far) pipeline:
>>
>> gst-launch-1.0 filesrc location= Downloads/big_buck_bunny_1080p_h264.mov ! qtdemux ! h264parse ! v4l2video3videodec capture-io-mode=dmabuf ! v4l2video0convert output-io-mode=dmabuf-import ! video/x-raw,width=480,height=272 ! ximagesink sync=false
>>
>> As I understand things so far:
>>
>> * filesrc read data into a buffer that qtdemux uses (no copy?)
>
> Well, reading from file to ram is usually considered a copy. Though
> encoded data is 40 times smaller generally.

Ah, I meant to indicate the data is read directly into the buffer that
qtdemux uses?

>> * qtdemux fills another buffer that h264parse uses (no copy?)
>
> qtdemux will frame and parse the output. It may requires copies.

presumably to pick out frame numbers, sizes, that sort of thing...

>> * h264parse fills another buffer (no copy?)
>
> In this case no copy, it will only copy is very special conditions.
>
>> * v4l2video3dec copies [in userspace] that last buffer to the
>>     buffers it gets from the kernel.
>
> A copy is made to the device mmap pool. This is only avoidable if your
> driver supports output-io-mode=userptr. demuxers don't do allocation
> query, so they can't write into the decoder pool.

If it did support output-io-mode=userptr, then the driver would see a
vitrually-contiguous but (likely?) physically discontg buffer, though?
so it'd need to support s/g DMA for there to be any gain?

>> * v4l2videoconvert gets the completed buffers from v4l2video3dec
>>     via dmabuf-import (no copy?) and produces the scaled/CSC'd output.
>
> No copy, and no CPU mampping (mmap). That makes the expensive flush
> suspicious btw.

I'll have a look and see what thats doing...

>> Im a bit hazy on what happens next. Where does v4l2videoconvert get
>> its output (capture?) buffers from? presumably they arent the same
>> ones it used for input as the sizes wont [necessarily] match?
>
> By default, the driver will allocate

On the fly? presumably from a pool then.

>  and will be mapped to userspace
> using mmap(). So there is not copy at that element, it just push mmap()
> buffers.

right, the copy happens in ximagesink?

>> if these are allocted in ximagesink, and passed in from there, then
>> it would need capture-io-mode=userptr is that right?
>
> Right.

AIUI then, this means that ximagesink wont cost a copy if the preceeding
element can use capture-io-mode=USERBUF?

>> presumably without USERPTR, the buffers would have been allocated
>> in kernel (via ?), and would need to have been copied to userspace?
>
> Via an allocator, could be CMA, scatter gatter, or vmalloc. They are not
> copied to userspace, they are mapped. The problem is that to pass that
> memory to the X11 server you need that memory to be XShm allocated. So
> ximagesink will copy the mmap buffer to an XShm segment.
>
>>
>> I did notice some /really/ odd behaviour from this pipeline:
>>
>> gst-launch-1.0 filesrc location= Downloads/big_buck_bunny_1080p_h264.mov  ! qtdemux ! h264parse ! v4l2video3videodec capture-io-mode=dmabuf ! v4l2video0convert output-io-mode=userptr ! video/x-raw,width=1920,height=1088 ! ximagesink sync=false
>>
>> with the video going back in time, etc. (actually, I think I could
>> see the IPU IC tiling, as it seemed like there were 4 regions of the
>> display, and not always in sync with each other.)
>
> In older Gst (that means anything not master) there was some issue that
> could do that. If you are on git master, then you have a broken driver,
> or userptr is not really supported.

I'm on master as of about last week...

-Ian