[Libva] (H264) Reference Picture selection

[Gwenole Beauchesne]

Hi,

2014-05-14 10:01 GMT+02:00 Xiang, Haihao <haihao.xiang at intel.com>:
> On Wed, 2014-05-14 at 15:49 +0800, Zhao, Yakui wrote:
>> On Sun, 2014-05-11 at 23:34 -0600, Gwenole Beauchesne wrote:
>> > 2014-05-12 7:29 GMT+02:00 Zhao, Yakui <yakui.zhao at intel.com>:
>> > > On Sun, 2014-05-11 at 22:41 -0600, Gwenole Beauchesne wrote:
>> > >> Hi,
>> > >>
>> > >> 2014-05-12 3:34 GMT+02:00 Zhao Yakui <yakui.zhao at intel.com>:
>> > >> > On Fri, 2014-05-09 at 03:06 -0600, Yuan, Feng wrote:
>> > >> >> Hi,
>> > >> >>
>> > >> >> >>> I have a scheme where the decoder ack's received frames - With this
>> > >> >> >>> information I could resync without an IDR by carefully selecting the
>> > >> >> >>> reference frame(s).
>> > >> >> >
>> > >> >> >> Will you please describe your idea in detail?
>> > >> >> >
>> > >> >> >>> long term references would typically be employed as well, but it
>> > >> >> >>> seems ref-pic-marking was removed may'13?
>> > >> >> >
>> > >> >> >> Why do you hope to use the long-term reference? As you know, the
>> > >> >> >> current encoding is based on hardware-acceleration. When more the
>> > >> >> >> reference frames can be selected, the driver will be more complex and
>> > >> >> >> then the encoding speed is also affected.
>> > >> >> >
>> > >> >> >It's for implementing Cisco's clearpath technology
>> > >> >> >(http://www.cisco.com/c/dam/en/us/td/docs/telepresence/endpoint/softwa
>> > >> >> >re/clearpath/clearpath_whitepaper.pdf)
>> > >> >> >Basically one would  periodically mark a P frame as long term reference, in
>> > >> >> >case of packet loss one can thus refer to this one instead of restarting the
>> > >> >> >stream with an idr.
>> > >> >> >
>> > >> >> >
>> > >> >> >I'm not too concerned about the encoding speed as long as it's above
>> > >> >> >realtime (60fps 1920x1080)
>> > >> >>
>> > >> >> Maybe it's more convenient If libva can have a switch for slice_header made between driver and app. Then some specially cases(ref-reorder, long-term-ref) may let app to manage slice-header.
>> > >> >
>> > >> > Thanks for your suggestion. If the slice-header info is also generated
>> > >> > by the app and then be passed to the driver, it can be easy to do the
>> > >> > operation of "re-order and long-term-ref"
>> > >> >
>> > >> > But this will depend on the policy how the slice header info is
>> > >> > generated. (by app or the driver). Now this is exported by querying the
>> > >> > capability of the driver. If we move this into the app, the
>> > >> > infrastructure will be changed a lot.
>> > >>
>> > >> Infrastructure of what? The application? No, it doesn't really change
>> > >> anything but generating the extra packed slice header.
>> > >
>> > > Currently the app will query the capability of encoding and determine
>> > > which kind of header should be passed by the upper application(For
>> > > example: PPS/SPS).
>> > > If the Slice_header info is exposed, the driver will assume that it is
>> > > the responsibility of generating the slice header info. In such case it
>> > > is difficult to mix the two working modes without a lot of changes.(One
>> > > is that the driver generates the slice header info. Another is that the
>> > > app generates the slice header info).
>> >
>> > Then the driver needs to be fixed to not assume anything. The API and
>> > operation points look clear enough, aren't they?
>>
>> Sorry that something is not described very clearly in my previous email
>> because of one typo error.
>>
>>    If the Slice_header info is exposed, app should be responsible of
>> generating the slice header info and then passed it into the driver.
>>    If the slice_header info is not exposed, the driver will be
>> responsible of generating the slice header info.
>>
>> And it is difficult to mix the above two working modes without a lot of
>> changes.
>
> Another thing is for CBR, the QP might be changed per frame in the
> driver and app/middleware doesn't not know the final QP. How does app
> get the right slice_qp_delta to build the right packed slice header ?
> does app always use a fixed slice_qp_delta ?

AFAIK, slice_qp_delta does not really matter. You can start off QP =
26 (defaults with pic_init_qp_minus26 = 0 + slice_qp_delta = 0), and
the driver can still generate per-MB QP that fits. i.e. you have an
additional delta at the macroblock layer IIRC. That should fit most
purposes.

Regards,
Gwenole.

>> > The changes to the codec layer are minimal, and this was done already,
>> > in multiple instances of codec layers, and with another driver.
>> >
>> > >> > Another problem is that it is possible that one frame has multiple
>> > >> > slices. In such case the user should pass the multiple slice header
>> > >> > info.
>> > >>
>> > >> This is not an an issue. The API mandates that the packed headers are
>> > >> provided in order.
>> > >
>> > >
>> > >
>> > >>
>> > >> Regards,
>> > >> Gwenole.
>> > >
>> > >
>>
>>
>> _______________________________________________
>> Libva mailing list
>> Libva at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/libva
>
>