[Libva] (H264) Reference Picture selection

[Zhao, Yakui]

On Sun, 2014-05-11 at 23:34 -0600, Gwenole Beauchesne wrote:
> 2014-05-12 7:29 GMT+02:00 Zhao, Yakui <yakui.zhao at intel.com>:
> > On Sun, 2014-05-11 at 22:41 -0600, Gwenole Beauchesne wrote:
> >> Hi,
> >>
> >> 2014-05-12 3:34 GMT+02:00 Zhao Yakui <yakui.zhao at intel.com>:
> >> > On Fri, 2014-05-09 at 03:06 -0600, Yuan, Feng wrote:
> >> >> Hi,
> >> >>
> >> >> >>> I have a scheme where the decoder ack's received frames - With this
> >> >> >>> information I could resync without an IDR by carefully selecting the
> >> >> >>> reference frame(s).
> >> >> >
> >> >> >> Will you please describe your idea in detail?
> >> >> >
> >> >> >>> long term references would typically be employed as well, but it
> >> >> >>> seems ref-pic-marking was removed may'13?
> >> >> >
> >> >> >> Why do you hope to use the long-term reference? As you know, the
> >> >> >> current encoding is based on hardware-acceleration. When more the
> >> >> >> reference frames can be selected, the driver will be more complex and
> >> >> >> then the encoding speed is also affected.
> >> >> >
> >> >> >It's for implementing Cisco's clearpath technology
> >> >> >(http://www.cisco.com/c/dam/en/us/td/docs/telepresence/endpoint/softwa
> >> >> >re/clearpath/clearpath_whitepaper.pdf)
> >> >> >Basically one would  periodically mark a P frame as long term reference, in
> >> >> >case of packet loss one can thus refer to this one instead of restarting the
> >> >> >stream with an idr.
> >> >> >
> >> >> >
> >> >> >I'm not too concerned about the encoding speed as long as it's above
> >> >> >realtime (60fps 1920x1080)
> >> >>
> >> >> Maybe it's more convenient If libva can have a switch for slice_header made between driver and app. Then some specially cases(ref-reorder, long-term-ref) may let app to manage slice-header.
> >> >
> >> > Thanks for your suggestion. If the slice-header info is also generated
> >> > by the app and then be passed to the driver, it can be easy to do the
> >> > operation of "re-order and long-term-ref"
> >> >
> >> > But this will depend on the policy how the slice header info is
> >> > generated. (by app or the driver). Now this is exported by querying the
> >> > capability of the driver. If we move this into the app, the
> >> > infrastructure will be changed a lot.
> >>
> >> Infrastructure of what? The application? No, it doesn't really change
> >> anything but generating the extra packed slice header.
> >
> > Currently the app will query the capability of encoding and determine
> > which kind of header should be passed by the upper application(For
> > example: PPS/SPS).
> > If the Slice_header info is exposed, the driver will assume that it is
> > the responsibility of generating the slice header info. In such case it
> > is difficult to mix the two working modes without a lot of changes.(One
> > is that the driver generates the slice header info. Another is that the
> > app generates the slice header info).
> 
> Then the driver needs to be fixed to not assume anything. The API and
> operation points look clear enough, aren't they?

Sorry that something is not described very clearly in my previous email
because of one typo error.

   If the Slice_header info is exposed, app should be responsible of
generating the slice header info and then passed it into the driver. 
   If the slice_header info is not exposed, the driver will be
responsible of generating the slice header info. 

And it is difficult to mix the above two working modes without a lot of
changes.   

> 
> The changes to the codec layer are minimal, and this was done already,
> in multiple instances of codec layers, and with another driver.
> 
> >> > Another problem is that it is possible that one frame has multiple
> >> > slices. In such case the user should pass the multiple slice header
> >> > info.
> >>
> >> This is not an an issue. The API mandates that the packed headers are
> >> provided in order.
> >
> >
> >
> >>
> >> Regards,
> >> Gwenole.
> >
> >