[Libva] (H264) Reference Picture selection

[Zhao, Yakui]

On Wed, 2014-05-14 at 03:50 -0600, Sreerenj wrote:
> On 14.05.2014 10:51, Xiang, Haihao wrote:
> > On Mon, 2014-05-12 at 07:34 +0200, Gwenole Beauchesne wrote:
> >> 2014-05-12 7:29 GMT+02:00 Zhao, Yakui <yakui.zhao at intel.com>:
> >>> On Sun, 2014-05-11 at 22:41 -0600, Gwenole Beauchesne wrote:
> >>>> Hi,
> >>>>
> >>>> 2014-05-12 3:34 GMT+02:00 Zhao Yakui <yakui.zhao at intel.com>:
> >>>>> On Fri, 2014-05-09 at 03:06 -0600, Yuan, Feng wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>>>>> I have a scheme where the decoder ack's received frames - With this
> >>>>>>>>> information I could resync without an IDR by carefully selecting the
> >>>>>>>>> reference frame(s).
> >>>>>>>> Will you please describe your idea in detail?
> >>>>>>>>> long term references would typically be employed as well, but it
> >>>>>>>>> seems ref-pic-marking was removed may'13?
> >>>>>>>> Why do you hope to use the long-term reference? As you know, the
> >>>>>>>> current encoding is based on hardware-acceleration. When more the
> >>>>>>>> reference frames can be selected, the driver will be more complex and
> >>>>>>>> then the encoding speed is also affected.
> >>>>>>> It's for implementing Cisco's clearpath technology
> >>>>>>> (http://www.cisco.com/c/dam/en/us/td/docs/telepresence/endpoint/softwa
> >>>>>>> re/clearpath/clearpath_whitepaper.pdf)
> >>>>>>> Basically one would  periodically mark a P frame as long term reference, in
> >>>>>>> case of packet loss one can thus refer to this one instead of restarting the
> >>>>>>> stream with an idr.
> >>>>>>>
> >>>>>>>
> >>>>>>> I'm not too concerned about the encoding speed as long as it's above
> >>>>>>> realtime (60fps 1920x1080)
> >>>>>> Maybe it's more convenient If libva can have a switch for slice_header made between driver and app. Then some specially cases(ref-reorder, long-term-ref) may let app to manage slice-header.
> >>>>> Thanks for your suggestion. If the slice-header info is also generated
> >>>>> by the app and then be passed to the driver, it can be easy to do the
> >>>>> operation of "re-order and long-term-ref"
> >>>>>
> >>>>> But this will depend on the policy how the slice header info is
> >>>>> generated. (by app or the driver). Now this is exported by querying the
> >>>>> capability of the driver. If we move this into the app, the
> >>>>> infrastructure will be changed a lot.
> >>>> Infrastructure of what? The application? No, it doesn't really change
> >>>> anything but generating the extra packed slice header.
> >>> Currently the app will query the capability of encoding and determine
> >>> which kind of header should be passed by the upper application(For
> >>> example: PPS/SPS).
> >>> If the Slice_header info is exposed, the driver will assume that it is
> >>> the responsibility of generating the slice header info. In such case it
> >>> is difficult to mix the two working modes without a lot of changes.(One
> >>> is that the driver generates the slice header info. Another is that the
> >>> app generates the slice header info).
> >> Then the driver needs to be fixed to not assume anything. The API and
> >> operation points look clear enough, aren't they?
> >>
> >> The changes to the codec layer are minimal, and this was done already,
> >> in multiple instances of codec layers, and with another driver.
> > Where is your code for codec layer with packed slice header/packed raw
> > data, I want to know the point where is packed slice header / packed raw
> > data inserted into the bitstream in the codec layer.
> I have these patches for the codec-layer (gstreaemr-vaapi)  but not yet 
> integrated to gstreamer-vaapi master.
> Submitting all packed_headers in the following order  just after the 
> submission of VAEncSequenceParameterBuffer to vaRenderPicture() : 
> "packed_sps, packed_pps, packed_slice".
> And then submitting Miscparam, PictureParam and SliceParam. Is it not 
> enough?

It is enough under the single slice. But it will still change where the
slice header is generated.

The current design is based on the following points:
   1. The driver exposes the SPS/PPS/MISC and the user-space app is
responsible for generating the corresponding packed ata.
   2. The driver is responsible for generating the slice header. 

If the generation of slice header is moved from the driver to the
user-space app, the driver needs a lot of changes and some working cases
will be affected. Another is  the complexity of handling the generation
of slice_headers in multislice. 

> 
> Anyway the intel driver seems to be using fixed size arrays for 
> referencing packed headers (packed_header_param[4] and 
> packed_header_data[4]) which needs to be changed if there are
> multiple slices per frame since we have to submit packed headers for 
> each slice. isn't it?
> 

Yes. If more packed data is passed, the driver needs to be changed. 

> 
> >
> >>
> >>>>> Another problem is that it is possible that one frame has multiple
> >>>>> slices. In such case the user should pass the multiple slice header
> >>>>> info.
> >>>> This is not an an issue. The API mandates that the packed headers are
> >>>> provided in order.
> >>>
> >>>
> >>>> Regards,
> >>>> Gwenole.
> >>>
> >> _______________________________________________
> >> Libva mailing list
> >> Libva at lists.freedesktop.org
> >> http://lists.freedesktop.org/mailman/listinfo/libva
> >
> >
> > _______________________________________________
> > Libva mailing list
> > Libva at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/libva
>