The identity plugin not being called per frame?

Fri Aug 2 13:00:06 UTC 2019

On Fri, Aug 2, 2019, 7:51 AM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:

> Le vendredi 02 août 2019 à 06:45 -0400, pisymbol . a écrit :
> > Nicolas:
> >
> > Can you please please pretty please answer me the following questions:
> >
> > 1) Would you expect the number of I420 frames to be equal to the H.264
> ones?
> >
> > i.e. nvcamerasrc ! identity name=count420 ! blah ! omxh264dec !
> h264parse ! identity name=count264 ! mux out...
> >
> > Again: Would you expect the number of "handoff" calls to be equal
> between the two identity plugins above?
>
>
> For most cases it should. There exist of course corner cases, when
> omxh264dec qos property is on, frames could be dropped at the input to
> ensure real-time operation. On very ancient gst-omx, I'm pretty sure
> there is random miss-behaviour.
>

What's ancient? I assume 1.8+ is not.

>
> >
> > 2) In the above pipeline would you expect the number of frames counted
> by "count264" equal the number of frames muxed to a multifilesink? i.e. a
> 1-to-1 match between frames IN the file and frames caught by identity
> "count264"?
> >
> > The problem I am seeing is that on stopping the pipeline, it looks like
> a few frames hit my handoff callback that don't make it out to the muxer
> and subsequently in the file stream. Is this because stopping is
> asynchronous, i.e. some frames hit handoff but never made it out to the
> muxer because the element is shutting down? Is there an idiom where I can
> guarantee a 1-to-1 match between number of frames that hit "handoff" vs
> number of frames that make it into the stream?
>
> Looks like a bad "drain/finish" implementation. It's possibly an OMX
> implementation issue though, please report to NVidia as this bit is
> proprietary, hence cannot be fixed by if broken.
>

Could be. Does multifilesink drop any frames if you use max-file-duration?
i.e. I assume splitting every 60s shouldn't inherit frame loss. However,
there is a thread somewhere in the NVidia forums that claims we need to use
splitmuxsink instead?

Sorry for the lack of clarity.

>
> >
> > 3) Is there any way to coordinate a multifilesink rollover
> (max-file-duration) event with a handoff function? I suspect the answer is
> no because there seems to be an inherit race condition between when my
> callback sees the event and when a handoff callback gets called for the
> "next frame" (or a frame that is past the file rollover)?
>
> No, I'd say use appsink and implement your own, it's really trivial
> thing to do, and then you can fully customized for your needs without
> any race.
>

Via new-sample?

>
>
> >
> > Thanks!
> >
> > -aps
> >
> > On Thu, Aug 1, 2019 at 8:12 PM Nicolas Dufresne <nicolas at ndufresne.ca>
> wrote:
> > > I'm looking at this thread and couldn't make my mind what pipeline we
> are referring to, it makes it's hard to help. Could meany things, could be
> QoS doing as this is live encoding. A shot in the dark, but you could try
> QoS=false on your sink maybe ?
> > >
> > > Le jeu. 1 août 2019 14 h 25, pisymbol . <pisymbol at gmail.com> a écrit :
> > > >
> > > > On Thu, Aug 1, 2019 at 12:18 PM pisymbol . <pisymbol at gmail.com>
> wrote:
> > > > >
> > > > > On Thu, Aug 1, 2019 at 11:12 AM pisymbol . <pisymbol at gmail.com>
> wrote:
> > > > > > Here is a short log of a pipeline starting and stopping:
> > > > > >
> > > > > > Available Sensor modes :
> > > > > > 4104 x 3046 FR=30.000000 CF=0x1009208a10 SensorModeType=4
> CSIPixelBitDepth=12 DynPixelBitDepth=12
> > > > > >
> > > > > > NvCameraSrc: Trying To Set Default Camera Resolution. Selected
> sensorModeIndex = 0 WxH = 4104x3046 FrameRate = 30.000000 ...
> > > > > >
> > > > > >
> > > > > > Available Sensor modes :
> > > > > > 4104 x 3046 FR=30.000000 CF=0x1009208a10 SensorModeType=4
> CSIPixelBitDepth=12 DynPixelBitDepth=12
> > > > > >
> > > > > > NvCameraSrc: Trying To Set Default Camera Resolution. Selected
> sensorModeIndex = 0 WxH = 4104x3046 FrameRate = 30.000000 ...
> > > > > >
> > > > > > Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen
> : Block : BlockType = 4
> > > > > > ===== MSENC =====
> > > > > > NvMMLiteBlockCreate : Block : BlockType = 4
> > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance
> violation
> > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance
> violation
> > > > > > Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen
> : Block : BlockType = 4
> > > > > > ===== MSENC =====
> > > > > > NvMMLiteBlockCreate : Block : BlockType = 4
> > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance
> violation
> > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance
> violation
> > > > > > ===== MSENC blits (mode: 1) into tiled surfaces =====
> > > > > > ===== MSENC blits (mode: 1) into tiled surfaces =====
> > > > > > 2019-08-01 11:01:40,635 INFO: Stop recording...
> > > > > > 2019-08-01 11:01:40,672 DEBUG: Stopping recorder pipeline...
> > > > > > [21, 17]
> > > > > >
> > > > > > The above is the number of numbers the "handoff" callback was
> called for the recording. It should be the total number of frames in the
> stream that landed on the filesystem, right?
> > > > > >
> > > > > > But it's not...
> > > > > >
> > > > > > $ ffprobe -v fatal -count_frames -select_streams v:0
> -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1
> capture.mkv
> > > > > > 18
> > > > > > $ ffprobe -v fatal -count_frames -select_streams v:1
> -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1
> capture.mkv
> > > > > > 14
> > > > > >
> > > > > > What am I missing?
> > > > > >
> > > > > > The pertinent pipeline bits are:
> > > > > >
> > > > > > nvcamearsrc ! identity name=tapX ...
> > > > > >
> > > > > > Why are the number of frames LESS than the number of times
> "handoff" was called?
> > > > > >
> > > > > >
> > > > >
> > > > > Is there a way to tell if a GstBuffer is actually frame data?
> (assuming an element is sending buffers that are NOT frame data???)
> > > > >
> > > >
> > > > If I move my callback deeper in my pipeline, I still see "drift" for
> lack of a better term. The number of callbacks seem to always be higher
> than the number of frames in the stream. I originally thought this was due
> to the fact that my stream is not fixed at 30fps but ffprobe does indeed
> count the frames one by one (
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190802/525f8577/attachment.html>