<div dir="ltr"><div><div><div><div><div><div><div><div><div><div><div><div><div>Experimenting a bit further with gstreamer-1.1.90 and shm, I have found through trial and error the following.<br><br></div>1) If the the shmsink option 'wait-for-connection' is true (1), the shmsink option shm-size must be a size equal to or greater than 6 frames. If the shm-size is less than that, the pipeline with the shmsink may and quite often will freeze whether or not another process connects in the other end. If the shmsink does not have a queue element upstream, the shm-size must be equal to or greater than 3 frames.<br> <br>2) If the the shmsink option 'wait-for-connection' is false (0) and the shmsink has a queue element upstream, the shmsink option shm-size must be a size equal to or greater than 3 frames. If the shm-size is less than that, the pipeline with the shmsink may and quite often will freeze whether or not another process connects in the other end. If the shmsink does not have a queue element upstream, the shm-size must be equal to or greater than 2 frames.<br><br></div>Is this what we want ? Especially the 6 frames can be a bit excessive.<br> <br></div>Here is how it was tested.<br><br></div>Sender side with queue element:<br>gst-launch-1.0 -v videotestsrc is-live=true do-timestamp=false ! $RAWVIDEO ! queue ! identity silent=false ! shmsink socket-path=/tmp/control-pipe shm-size=22118400 wait-for-connection=1 sync=true<br> <br></div>Sender side without queue element<br>gst-launch-1.0 -v videotestsrc is-live=true do-timestamp=false ! $RAWVIDEO ! identity silent=false ! shmsink socket-path=/tmp/control-pipe shm-size=22118400 wait-for-connection=1 sync=true<br> <br></div>RAWVIDEO was defined as<br>$ RAWVIDEO='video/x-raw, format=(string)BGRA, width=(int)1280, height=(int)720, framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false'<br><br></div> Receiver side was defined as<br>while true ; do gst-launch-1.0 -v shmsrc do-timestamp=true is-live=true socket-path=/tmp/control-pipe ! identity silent=false ! fakesink sync=true ; sleep 2 ;done<br><br></div>With the defined frame type and size, the sizes tested was as follows<br> </div>1 frame = 3686400<br></div>2 frames = 7372800<br></div>3 frames = 11059200<br></div><div>4 frames = 14745600<br></div>6 frames = 22118400<br></div>7 frames = 25804800<br><div><div><div><div><div><div><div><br></div> <div>In the above example, the videotesrc element was used as a video src. In a more complex example shown below, the number of frames/buffers needed to avoid freezing, EVEN with wait-for-connection=0, is 6 or approx 38MB. Here is the example:<br> <br>/usr/local/bin/gst-launch-1.0 -v filesrc location=../test/LES_TDS_launch.mp4 do-timestamp=true ! decodebin name=decoder ! videoconvert ! videorate ! videoscale ! videoconvert ! 'video/x-raw, format=(string)BGRA, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, width=(int)1280, height=(int)720, framerate=(fraction)25/1' ! identity silent=false ! queue ! shmsink socket-path=/tmp/control-pipe shm-size=36864000 wait-for-connection=0 sync=true decoder. ! queue ! audioconvert ! audioresample ! 'audio/x-raw, rate=(int)44100, channels=(int)1' ! fdsink fd=3 sync=true<br> <br></div><div>If we remove the queue upstream to shmsink, the minimum required shm-size is 2 frames/buffers. If the consuming side keep one or more buffers, you for some cases have to add that number plus one to the buffer pool (shm-size).<br> <br></div><div>In this more advance example, using wait-for-connection=1 does not seem to increase the need for buffers.<br></div><div><div><div><div><div><br><div>Olivier, is this what you expect with the changes you made to shmsink?<br> <br></div><div>Kind regards<br></div><div>Peter MM<br></div><div><br></div></div></div></div></div></div></div></div></div></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Sep 20, 2013 at 11:53 AM, Peter Maersk-Moller <span dir="ltr"><<a href="mailto:pmaersk@gmail.com" target="_blank">pmaersk@gmail.com</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div>Hi Olivier<br><br></div>I'm afraid that's not exactly what I'm looking for. Also my pipeline example didn't detailed the problem clear enough, so please allow me to try again. Assuming you have the following pipeline:<br> <br></div> gst-launch udpsrc ! 'video/mpgets' !\<br> queue name=qts leaky=2 ! decodebin !\<br></div> 'video/x-raw' ! queue name=qvid leaky=2 ! shmsink name=shmvid \<br></div> 'audio/x-raw' ! queue name=qaud leaky=2 ! shmsink name=shmaud<br> <br></div>Also assume we have two other processes that process the buffers passed by the two shmsinks and acknowledge the buffers when read.<br><br>The leaky queue 'qts' is included to point out, that if decodebin stops or don't process buffers received from upstream fast enough, TS packets are lost. In fact that would happen anyway as an UDP connection in principle is a queue anyway and that queue is leaky in case data is not read fast enough. This is a hard fact and not something we can control. In fact I would prefer I could guarantee that decodebin always receives all packets.<br> <br>The queues 'qvid' and 'qaud' are included to illustrate that I would like raw video or raw audio to be dropped frame by frame if the processes that shmsink delivers buffers are not fast enough or decide it is desired that raw frames are dropped. However I believe decodebin may also include queues on its src pads (output), but they are harder to control individually.<br> <br></div>Now I think the pipeline works this way, that the shmsinks passes shmbufs upstream to decodebin for decodebin to fill with raw video and raw audio frames. The shmsinks receives these buffers and signal to the two external processes that shmbuffers are ready to be read.<br> <br></div>Lets now assume that the process reading the raw video frames in the pipeline would like to signal to the gstreamer pipeline, that the pipeline temporarily can halt sending decoded frames. Right now the only way to do that is by not sending Acks through the control pipe for buffers it has received. However, the shmsink will continue handing out shmbuffers upstream and signal received buffers ready to be read until it has no more buffers. Now when the shmsink 'shmvid' has no more buffers to hand out, I believe decodebin will stop decode video data. I fear this will also stop the decoder from sending audio data upstream which can be fatal. I also fear that if shmsink 'shmvid' stops, decodebin will stop sending buffers upstream leading to an unpredictable loss of packets.<br> <br></div>My questions are now these:<br><br></div><div>1) Am I correct that decodebin will stop decode video if it stops receiving ready shmbuffers from downstream?<br></div>2) If shmsink 'shmvid' stops sending shmbuffers upstream, will it also stop audio decoding ?<br> </div>3) If decodebin stops receiving shmbuf from downstream shmsink 'shmvid', will it eventually stop decoding TS packets?<br></div>4) If decodebin stops receiving shmbuf from downstream, shmsink 'shmvid', what is the state of decoding when it starts receive buffers again?<br> <br></div><div>Oberservations and opinions:<br><br></div><div>1) It can be fatal if stopping video also stops audio, so that is to be avoided.<br>2) It can be fatal for the quality of the decoded streams, if TS packets are dropped before decoded, so that is to be avoided.<br> </div>3) It can be fatal for the quality of the decoded video stream, if encoded video data is dropped before decoded because the the encoder otherwise will be in an unpredictable state and when started again (receiving buffers) we may get large pixelation, green or gray screen and weird moving blocks which is unacceptable for production of professional TV, though personal player may have this behaviour.<br> <br></div>So here is what I would like. I would like that the external process could send a message through the control pipe and instruct the shmsink that it temporarily is instructed not to signal more buffers ready for the process. That it self is not a very big difference, but the shmsink could use the information to signal upstream, that while it may be handing out shmbuffers upstream, it will temporarily not accept any more buffers downstream and a leaky queue upstream to shmsink can then leak(drop) buffers. Even better would it be if a decoder upstream can somehow use this information to understand, that it should not send decoded frames downstream, but it should continue to decode to keep an updated state. The latter may be unobtainable with the GStreamer (and decoder) design, but it would be nice to have to optimize resource usage. Even better would it be, if the decoder can stop decode while maintain a state and inspecting the encoeded stream so it when requested as soon as possible can start deliver a new decoded full frame.<br> <br></div><div>Sorry for the lengthy text, but it's a rather complicated subject.<br><br></div>Best regards<span class="HOEnZb"><font color="#888888"><br></font></span></div><span class="HOEnZb"><font color="#888888">Peter Maersk-Moller<br> <div><div><div><div><div><div><div><div><div><div><div><br></div></div></div> </div></div></div></div></div></div></div></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Sep 20, 2013 at 12:58 AM, Olivier Crête <span dir="ltr"><<a href="mailto:olivier.crete@collabora.com" target="_blank">olivier.crete@collabora.com</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On Thu, 2013-09-19 at 18:50 -0400, Olivier Crête wrote:<br> > One thing we could do is add properties to limit the size of "queue"<br> > that's inside shmsink by time or bytes or buffers (like the queue<br> > element). I'd certainly be open to something like that.<br> <br> </div>Actually, there is already a buffer-time property to limit the size of<br> the "queue", but it is only triggered when the buffer is free on the<br> receiving side, probably not exactly what you want, but that could be<br> fixed.<br> <div><div><br> --<br> Olivier Crête<br> <a href="mailto:olivier.crete@collabora.com" target="_blank">olivier.crete@collabora.com</a><br> <br> _______________________________________________<br> gstreamer-devel mailing list<br> <a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a><br> <a href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel" target="_blank">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a><br> </div></div></blockquote></div><br></div> </div></div></blockquote></div><br></div>