[Mesa-dev] [PATCH] st/va: move YUV content to deinterlaced buffer when reallocated for encoder

[Andy Furniss]

Leo Liu wrote:
> 
> 
>>>>>>>> +      }
>>>>>>> 
>>>>>>> Should we bail out with an error here when it's the other
>>>>>>> way around?
>>>>>> Although I cannot think of any of case that to get buffer 
>>>>>> Interlaced now, It's still a good idea to bail out here
>>>>>> when it happnens Will add it in v4.
>>>>> 
>>>>> It's not a error when case like buffer is deinterlaced, and 
>>>>> interlaced result from query. What we need to do is to do
>>>>> nothing, just ignores. I have sent out v4, please ignore it,
>>>>> it won't work.
>>>> 
>>>> Well that's not correct either.
>>>> 
>>>> When the buffer is allocated as progressive and the codec
>>>> doesn't supports that we should certainly do something.
>>>> 
>>>> Either bail out as an error when we encode because we can't
>>>> convert progressive->interlaced (just the other way around)
>>>> and/or reallocate for decoding.
>>> Here is current situation for  transcode
>>> 
>>> Decoder allocate I buffers as preferred, then encoder prefers as
>>> P buffers , so re-allocated them to P buffers. and then next
>>> frame, decoder take P buffer, but not as preferred.
>>> 
>>> 3 possible ways for decoder to go:
>>> 
>>> 1. ignores the the Preferred, and keep buffer as P, and pipe
>>> goes. V3 2. go for Preferred, and then do endless alloc/dealloc
>>> frame by frame. V2 3. Bailout as error, the pipeline stops. V4
>> 
>> Won't have time to test till tomorrow but just getting one of the
>> cases I thought may work with this, that couldn't work with the
>> env, out.
>> 
>> ffmpeg can (in theory anyway) do -
>> 
>> hwdec -> hw deinterlace -> hw encode.
>> 
>> Possible?
> 
> Not sure about FFMpeg. Have you tried that before? I always use 
> gstreamer for encode/transcode.

The env existed before ffmpeg vaapi could do it, so I expected and got
failiure.

IIRC with the env = 0 hwdec -> hw deint -> copy back raw nv12 worked,
but trying to get encode failed as expected without env = 1. It was a
while ago I tried, IIRC it was possible to hang GPU. I guess the deint

I don't know how, so haven't tried a gstreamer command that would do the
same.