The identity plugin not being called per frame?

Fri Aug 2 17:18:41 UTC 2019

Le vendredi 02 août 2019 à 09:00 -0400, pisymbol . a écrit :
> 
> 
> On Fri, Aug 2, 2019, 7:51 AM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> > Le vendredi 02 août 2019 à 06:45 -0400, pisymbol . a écrit :
> > > Nicolas:
> > > 
> > > Can you please please pretty please answer me the following questions:
> > > 
> > > 1) Would you expect the number of I420 frames to be equal to the H.264 ones?
> > > 
> > > i.e. nvcamerasrc ! identity name=count420 ! blah ! omxh264dec ! h264parse ! identity name=count264 ! mux out...
> > > 
> > > Again: Would you expect the number of "handoff" calls to be equal between the two identity plugins above?
> > 
> > 
> > For most cases it should. There exist of course corner cases, when
> > omxh264dec qos property is on, frames could be dropped at the input to
> > ensure real-time operation. On very ancient gst-omx, I'm pretty sure
> > there is random miss-behaviour.
> 
> What's ancient? I assume 1.8+ is not.

In term of gst-omx, 1.8 is pretty ancient, most of the recent work
landed in 1.16. I still don't understand why people keep using OMX
though.

> > > 
> > > 2) In the above pipeline would you expect the number of frames counted by "count264" equal the number of frames muxed to a multifilesink? i.e. a 1-to-1 match between frames IN the file and frames caught by identity "count264"?
> > > 
> > > The problem I am seeing is that on stopping the pipeline, it looks like a few frames hit my handoff callback that don't make it out to the muxer and subsequently in the file stream. Is this because stopping is asynchronous, i.e. some frames hit handoff but never made it out to the muxer because the element is shutting down? Is there an idiom where I can guarantee a 1-to-1 match between number of frames that hit "handoff" vs number of frames that make it into the stream?
> > 
> > Looks like a bad "drain/finish" implementation. It's possibly an OMX
> > implementation issue though, please report to NVidia as this bit is
> > proprietary, hence cannot be fixed by if broken.
> 
> Could be. Does multifilesink drop any frames if you use max-file-duration? i.e. I assume splitting every 60s shouldn't inherit frame loss. However, there is a thread somewhere in the NVidia forums that claims we need to use splitmuxsink instead?

Splitmux sink is a better choice yes. But in theory there should be no
lost caused by this sink, it mean yield an invalid file though.

> 
> Sorry for the lack of clarity.
> > > 
> > > 3) Is there any way to coordinate a multifilesink rollover (max-file-duration) event with a handoff function? I suspect the answer is no because there seems to be an inherit race condition between when my callback sees the event and when a handoff callback gets called for the "next frame" (or a frame that is past the file rollover)?
> > 
> > No, I'd say use appsink and implement your own, it's really trivial
> > thing to do, and then you can fully customized for your needs without
> > any race.
> 
> Via new-sample?
> 
> 
> 
> 
> > 
> > 
> > > 
> > > Thanks!
> > > 
> > > -aps
> > > 
> > > On Thu, Aug 1, 2019 at 8:12 PM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> > > > I'm looking at this thread and couldn't make my mind what pipeline we are referring to, it makes it's hard to help. Could meany things, could be QoS doing as this is live encoding. A shot in the dark, but you could try QoS=false on your sink maybe ?
> > > > 
> > > > Le jeu. 1 août 2019 14 h 25, pisymbol . <pisymbol at gmail.com> a écrit :
> > > > > 
> > > > > On Thu, Aug 1, 2019 at 12:18 PM pisymbol . <pisymbol at gmail.com> wrote:
> > > > > > 
> > > > > > On Thu, Aug 1, 2019 at 11:12 AM pisymbol . <pisymbol at gmail.com> wrote:
> > > > > > > Here is a short log of a pipeline starting and stopping:
> > > > > > > 
> > > > > > > Available Sensor modes :
> > > > > > > 4104 x 3046 FR=30.000000 CF=0x1009208a10 SensorModeType=4 CSIPixelBitDepth=12 DynPixelBitDepth=12
> > > > > > > 
> > > > > > > NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 0 WxH = 4104x3046 FrameRate = 30.000000 ...
> > > > > > > 
> > > > > > > 
> > > > > > > Available Sensor modes :
> > > > > > > 4104 x 3046 FR=30.000000 CF=0x1009208a10 SensorModeType=4 CSIPixelBitDepth=12 DynPixelBitDepth=12
> > > > > > > 
> > > > > > > NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 0 WxH = 4104x3046 FrameRate = 30.000000 ...
> > > > > > > 
> > > > > > > Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
> > > > > > > ===== MSENC =====
> > > > > > > NvMMLiteBlockCreate : Block : BlockType = 4
> > > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance violation
> > > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance violation
> > > > > > > Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
> > > > > > > ===== MSENC =====
> > > > > > > NvMMLiteBlockCreate : Block : BlockType = 4
> > > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance violation
> > > > > > > NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance violation
> > > > > > > ===== MSENC blits (mode: 1) into tiled surfaces =====
> > > > > > > ===== MSENC blits (mode: 1) into tiled surfaces =====
> > > > > > > 2019-08-01 11:01:40,635 INFO: Stop recording...
> > > > > > > 2019-08-01 11:01:40,672 DEBUG: Stopping recorder pipeline...
> > > > > > > [21, 17]
> > > > > > > 
> > > > > > > The above is the number of numbers the "handoff" callback was called for the recording. It should be the total number of frames in the stream that landed on the filesystem, right?
> > > > > > > 
> > > > > > > But it's not...
> > > > > > > 
> > > > > > > $ ffprobe -v fatal -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1  capture.mkv
> > > > > > > 18
> > > > > > > $ ffprobe -v fatal -count_frames -select_streams v:1 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 capture.mkv
> > > > > > > 14
> > > > > > > 
> > > > > > > What am I missing?
> > > > > > > 
> > > > > > > The pertinent pipeline bits are:
> > > > > > > 
> > > > > > > nvcamearsrc ! identity name=tapX ...
> > > > > > > 
> > > > > > > Why are the number of frames LESS than the number of times "handoff" was called?
> > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > Is there a way to tell if a GstBuffer is actually frame data? (assuming an element is sending buffers that are NOT frame data???)
> > > > > > 
> > > > > 
> > > > > If I move my callback deeper in my pipeline, I still see "drift" for lack of a better term. The number of callbacks seem to always be higher than the number of frames in the stream. I originally thought this was due to the fact that my stream is not fixed at 30fps but ffprobe does indeed count the frames one by one (
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190802/0160efd0/attachment.sig>