Follow-Up: v4l2h264dec gstreamer 1.18.4 not accepting stream

Mon Mar 7 18:39:07 UTC 2022

Le lundi 07 mars 2022 à 15:53 +0100, Pascal Speck via gstreamer-devel a écrit :
> Am 07.03.22 um 15:41 schrieb Nicolas Dufresne via gstreamer-devel:
> > Le lundi 07 mars 2022 à 12:29 +0100, Pascal Speck via gstreamer-devel a écrit :
> > > Hi together...
> > > 
> > > After a bit more of digging I found out that it seems to be the profile cap which is not accepted here:
> > 
> > Ok.
> > 
> > > 
> > > I see the "gst_caps_set_subset" beeing called with the following:
> > > 
> > > caps video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)800, height=(int)600, framerate=(fraction)0/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
> > > bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)3.1
> > > 
> > > template_caps: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string){ 1, 1b, 1.1, 1.2, 1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4, 4.1 }, profile=(string){ baseline, main, high }
> > 
> > This is a driver bug, mainline encoder must support constrained baseline. The
> > driver must be fixed because it is stateful and it will not mark the constraint
> > in the bitstream otherwise.
> 
> So if I get you right the v4l2h264dec needs to report something like  { constrained-baseline, basline, constrained-main, main, constrained-high } instead of { baseline, main, high } in his caps, so
> the above stream-caps would get accepted.
> I'm wondering where the regression from the old setup happened.. I'll try to show the caps of the old setup which are reported by the driver and find the point where they're reported wrong.

Older GStreamer version was completely ignoring V4L2 profiles. We needed to
exposed the profile in the template so that auto-plugger can fallback if the
selected HW (encoder or decoder) is not capable of the the profile requested in
the caps. This feature was enabled in 1.18.0 (8 September 2020), so in theory,
RPi folks knew about this and just ignored it. Or they just don't test much the
multimedia side.

It has been the same for many years, each time we start using a feature in V4L2,
we hit drivers with partial or incorrect implementation. We support the "not
implemented" cases (unless we have a bug), but not the partially implemented
one. As we speak, we started working on Kernel CI, so that we can test newer
Kernel against current GStreamer and vis-versa, but RPi folks won't benefit of
this effort until they actually land their decoder driver(s) upstream. So they
highly rely on users for the testing and debugging.

> 
> > 
> > > 
> > > As this function does only compare for "equal" features, I would assume that it breaks here.
> > > 
> > > I'm currently wondering who does the mistake here:
> > > 
> > > Should the h264parse element also add the profile="baseline" feature or should the decoder also answer "constrained-baseline" in it's caps to fullfil this issue.?!
> > > 
> > > Some help of the gst-experts what should be the expected behaviour is appreciated here, so I would know at which module I should search for changes.
> > > 
> > > Best Regards
> > > 
> > > Pascal
> > > 
> > > Am 07.03.22 um 08:18 schrieb Pascal Speck via gstreamer-devel:
> > > > 
> > > > Currently I'm wondering about 3 things:
> > > > 
> > > > 1: Is it really the colorimetry that is not accepted?
> > > > I don't see the "ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
> > > > Device '/dev/xxx' does not support xxx colorimetry" message here. I was assuming into the direction of "baseline" vs "constrained-baseline" because v4l2h264enc is only returning baseline in accepted caps.
> > > > 
> > > > 2: Can one give me an idea how to gain more logging information about which capability is causing the "non-acceptance"? Would it be a deal to generally having this information in the gst-error-log?
> > > > 
> > > > I'll try to dig in further meanwhile but as I'm not a gstreamer-expert I first have to find the file where it's worth starting to dig into the code... :-)
> > > > 
> > > > 3: Is there a simple solution to just override the caps in the pipeline only "downstream" to pretend to have a different colorimetry or profile.. If so I would maybe findout more which of the caps is
> > > > not accepted...
> > > > 
> > > > Regards Pascal
> > > > 
> > > > Am 04.03.22 um 14:52 schrieb Nicolas Dufresne via gstreamer-devel:
> > > > Am 04.03.22 um 14:52 schrieb Nicolas Dufresne via gstreamer-devel:
> > > > 
> > > > > Le vendredi 04 mars 2022 à 11:21 +0100, Pascal Speck via gstreamer-devel a
> > > > > écrit :
> > > > > > 
> > > > > > 0:00:03.690824695  1500  0xffa000 INFO                    v4l2 gstv4l2object.c:4469:gst_v4l2_object_probe_caps:<v4l2h264dec0:sink> probed caps: video/x-h264, stream-format=(string)byte-stream,
> > > > > > alignment=(string)au, level=(string){ 1, 1
> > > > > > b, 1.1, 1.2, 1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4, 4.1 }, profile=(string){ baseline, main, high }, width=(int)[ 48, 1920 ], height=(int)[ 16, 1088 ], colorimetry=(string){ bt709, bt601, smpte240m,
> > > > > > 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12
> > > > > > :8, bt2020, 2:0:0:0 }, parsed=(boolean)true
> > > > > > 
> > > > > > I can see that with the old setup the caps seem very similar but are accepted:
> > > > > > 
> > > > > > /GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)800, height=(int)600, framerate=(fraction)0/1,
> > > > > > interlace-mode=(string)progressive, chrom
> > > > > > a-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)3.1
> > > > > > 
> > > > > > Does someone have an Idea where to dig further?
> > > > > 
> > > > > Looks quite similar to:
> > > > > 
> > > > > https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1056
> > > > > 
> > > > > We *just* need someone with free time to fix it.
> > > > > 
> > > > > Nicolas
> > 