<div dir="ltr"><div>I need to confess, I work on IP cameras all day long. I work for a company that writes Security Camera software.</div><div><br></div><div>In the beginning, there was Analog (bear with me)<br></div><div> Very low latency, no buffering, very slow searching that ole VCR.</div><div><br></div><div>then IP camera on the scene with same resolution but was frame by brame compression.</div><div> Variable framerate, on demand to control of bandwidth,</div><div> Storage was predictable and pruning was linear</div><div><br></div><div>Then Mp4,H264,H265, B frames, VPn, <br></div><div> Gop based video - a periodic key frame and then difference frames 1/10 their size allowing Mega-Pixel bandwidth and storage</div><div> But now the encoding delay becomes important. on 30 fps video it is not uncommon for the encoding of the keyframe to take longer than sample time <br></div><div> This requires operating in the past, ever so slightly.</div><div> Lets say encoding the keyframe takes 25ms and and diff frames take 8ms. no problem 29 * 8 + 35 = 267 ms , much less than the second it spans</div><div> But You need to buffer and operate with a latency of 18ms downstream to stay smooth AND IMPORTANT FOR FFMPEG. not complain about frames to late to display and drop right after the keyframe.</div><div><br></div><div>Now there is HLS</div><div> This introdeces Segments. and the first segment must start with a keyframe and usually every segment has segments made on n number of complet gops, makes searching easier.</div><div> Now you must buffer 1.5 segments and the viewer operates with multipl seconds of delay. <br></div><div><br></div><div>In the beginning apple speced HLS with 10 second long segments You watched video that was alomst always 12 seconds behind. <br></div><div>I ran it as low as 2 seconds long segments before dash came around with...you guessed it, shorter segemnts. <br></div><div>These work by having a manifest, essitially a virtual filesystem in memory for live, But the delay is imho unacceptable.</div><div><br></div><div>Newer protocols on newer browsers focus on reducing the latency but good luck getting all browsers to cooperate on a standard.</div><div><br></div><div>BTW in just the last 5 years IP cameras have gone from 100's of ms of encoding delay on their side to less than 10ms.<br></div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 3, 2022 at 6:58 PM Peter Maersk-Moller via gstreamer-devel <<a href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Some webcams, like the old-timer Logitech C920, can deliver H.264 encoded video over V4L2.</div><div><br></div><div>That said, while cameras like the C920 camera records frames at a steady rate evenly spaced apart in time, these arrive in the driver/kernel level unevenly spaced apart and may need a mechanism to timestamp them with an even spaced time stamps. However the uneven spacing may only occur either between each second or perhaps between each I-frame. Usually I-frames will be late and the following P and B frames may arrive in a bulk or in some cases, the fP and B frames following an I frame, may have the same timestamp or nearly the same timestamp, If this is the case and the hlssink2 muxer does not correct for the timestamps being evenly spread out (it most likely do not), the mixer may have a hard time to mux audio with perhaps correct timestamps and video with basically faulty timestamps. You can check that by adding the "identity silent=false" element to your video pipeline before the hlssink2 video input. You may have to add "-v" to gst-launch-1.0.</div><div><br></div><div>Regards</div><div>Peter MM</div><div><br></div>On Mon, Jan 3, 2022 at 11:31 PM Jeff Shanab via gstreamer-devel <<a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a>> wrote:<br><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Webcams are raw yuv video, arn't they? gstreamer in this case is using ffmpegs x-h264 to encode it <br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 3, 2022 at 4:51 PM Nicolas Dufresne via gstreamer-devel <<a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi James,<br> <br> Le lundi 03 janvier 2022 à 07:16 +0800, James via gstreamer-devel a écrit :<br> > gstreamer seems very nice in concept. The fact that I've been trying for 3 months and can get no help is a big deterent.<br> > <br> > I've got a 4 core i7 NUC clocked to 4.8G and I get a stream of QoS messages telling me the computer is too slow.<br> > (GST_DEBUG=2,pulsesrc:6)<br> <br> Sorry if previous message were missed. Perhaps your issue is specific to your<br> WebCam ? I don't owned myself a webcam that encodes to H264, so I've used<br> vaapih264enc, and performance was decent.<br> <br> Asking question on public channels is an art, make sure to narrow down as much<br> as possible your issue, and try to think on how others will be able to<br> reproduce, if you can't make sure to share extra information that would allow<br> controlling the variable when simulating with audiotestsrc and similar.<br> <br> regards,<br> Nicolas<br> <br> > <br> > The machine is idle running a single pipeline.<br> > The stream stutters. ffmpeg shows dup and often a 100 dropped frames on each segment.<br> > <br> > Using audiotestsrc renders perfectly.<br> > <br> > #! /bin/bash<br> > <br> > IP=`hostname -I`<br> > <br> > gst-launch-1.0 -e -v v4l2src device=/dev/video2 ! \<br> > video/x-h264,width=1920,height=1080,framerate=30/1 ! \<br> > h264parse ! \<br> > tee name=vt \<br> > vt. ! queue ! hlssink2 max-files=5 name=hl \<br> > playlist-root=http://$IP playlist-location=/dev/shm/channel1.m3u8 location=/dev/shm/segment_%05d.ts \<br> > pulsesrc device=0 ! audioconvert ! avenc_aac ! \<br> > tee name=at \<br> > at. ! queue ! aacparse ! hl.audio<br> > <br> > The redundant tee's are for use later.<br> > unless I see a euroka moment I'll have to try somethink else<br> > James<br> > <br> <br> </blockquote></div> </blockquote></div></div></div> </blockquote></div>