shmsink changes from 1.0.10 to 1.1.4

[Peter Maersk-Moller]

Experimenting a bit further with gstreamer-1.1.90 and shm, I have found
through trial and error the following.

1) If the the shmsink option 'wait-for-connection' is true (1), the shmsink
option shm-size must be a size equal to or greater than 6 frames. If the
shm-size is less than that, the pipeline with the shmsink may and quite
often will freeze whether or not another process connects in the other end.
If the shmsink does not have a queue element upstream, the shm-size must be
equal to or greater than 3 frames.

2) If the the shmsink option 'wait-for-connection' is false (0) and the
shmsink has a queue element upstream, the shmsink option shm-size must be a
size equal to or greater than 3 frames. If the shm-size is less than that,
the pipeline with the shmsink may and quite often will freeze whether or
not another process connects in the other end. If the shmsink does not have
a queue element upstream, the shm-size must be equal to or greater than 2
frames.

Is this what we want ? Especially the 6 frames can be a bit excessive.

Here is how it was tested.

Sender side with queue element:
gst-launch-1.0 -v videotestsrc is-live=true do-timestamp=false ! $RAWVIDEO
! queue ! identity silent=false ! shmsink socket-path=/tmp/control-pipe
shm-size=22118400 wait-for-connection=1 sync=true

Sender side without queue element
gst-launch-1.0 -v videotestsrc is-live=true do-timestamp=false ! $RAWVIDEO
! identity silent=false ! shmsink socket-path=/tmp/control-pipe
shm-size=22118400 wait-for-connection=1 sync=true

RAWVIDEO was defined as
$ RAWVIDEO='video/x-raw, format=(string)BGRA, width=(int)1280,
height=(int)720, framerate=(fraction)24/1,
pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false'

Receiver side was defined as
while true ; do gst-launch-1.0 -v shmsrc do-timestamp=true is-live=true
socket-path=/tmp/control-pipe ! identity silent=false ! fakesink sync=true
; sleep 2 ;done

With the defined frame type and size, the sizes tested was as follows
1 frame = 3686400
2 frames = 7372800
3 frames = 11059200
4 frames = 14745600
6 frames = 22118400
7 frames = 25804800

In the above example, the videotesrc element was used as a video src. In a
more complex example shown below, the number of frames/buffers needed to
avoid freezing, EVEN with wait-for-connection=0, is 6 or approx 38MB. Here
is the example:

/usr/local/bin/gst-launch-1.0 -v filesrc
location=../test/LES_TDS_launch.mp4 do-timestamp=true ! decodebin
name=decoder ! videoconvert ! videorate ! videoscale ! videoconvert !
'video/x-raw, format=(string)BGRA, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, width=(int)1280, height=(int)720,
framerate=(fraction)25/1' ! identity silent=false ! queue ! shmsink
socket-path=/tmp/control-pipe shm-size=36864000 wait-for-connection=0
sync=true decoder. ! queue ! audioconvert ! audioresample ! 'audio/x-raw,
rate=(int)44100, channels=(int)1' ! fdsink fd=3 sync=true

If we remove the queue upstream to shmsink, the minimum required shm-size
is 2 frames/buffers. If the consuming side keep one or more buffers, you
for some cases have to add that number plus one to the buffer pool
(shm-size).

In this more advance example, using wait-for-connection=1 does not seem to
increase the need for buffers.

Olivier, is this what you expect with the changes you made to shmsink?

Kind regards
Peter MM



On Fri, Sep 20, 2013 at 11:53 AM, Peter Maersk-Moller <pmaersk at gmail.com>wrote:

> Hi Olivier
>
> I'm afraid that's not exactly what I'm looking for. Also my pipeline
> example didn't detailed the problem clear enough, so please allow me to try
> again. Assuming you have the following pipeline:
>
>   gst-launch udpsrc ! 'video/mpgets' !\
>     queue name=qts leaky=2 ! decodebin !\
>     'video/x-raw' ! queue name=qvid leaky=2 ! shmsink name=shmvid \
>     'audio/x-raw' ! queue name=qaud leaky=2 ! shmsink name=shmaud
>
> Also assume we have two other processes that process the buffers passed by
> the two shmsinks and acknowledge the buffers when read.
>
> The leaky queue 'qts' is included to point out, that if decodebin stops or
> don't process buffers received from upstream fast enough, TS packets are
> lost. In fact that would happen anyway as an UDP connection in principle is
> a queue anyway and that queue is leaky in case data is not read fast
> enough. This is a hard fact and not something we can control. In fact I
> would prefer I could guarantee that decodebin always receives all packets.
>
> The queues 'qvid' and 'qaud' are included to illustrate that I would like
> raw video or raw audio to be dropped frame by frame if the processes that
> shmsink delivers buffers are not fast enough or decide it is desired that
> raw frames are dropped. However I believe decodebin may also include queues
> on its src pads (output), but they are harder to control individually.
>
> Now I think the pipeline works this way, that the shmsinks passes shmbufs
> upstream to decodebin for decodebin to fill with raw video and raw audio
> frames. The shmsinks receives these buffers and signal to the two external
> processes that shmbuffers are ready to be read.
>
> Lets now assume that the process reading the raw video frames in the
> pipeline would like to signal to the gstreamer pipeline, that the pipeline
> temporarily can halt sending decoded frames. Right now the only way to do
> that is by not sending Acks through the control pipe for buffers it has
> received. However, the shmsink will continue handing out shmbuffers
> upstream and signal received buffers ready to be read until it has no more
> buffers. Now when the shmsink 'shmvid' has no more buffers to hand out, I
> believe decodebin will stop decode video data. I fear this will also stop
> the decoder from sending audio data upstream which can be fatal. I also
> fear that if shmsink 'shmvid' stops, decodebin will stop sending buffers
> upstream leading to an unpredictable loss of packets.
>
> My questions are now these:
>
> 1) Am I correct that decodebin will stop decode video if it stops
> receiving ready shmbuffers from downstream?
> 2) If shmsink 'shmvid' stops sending shmbuffers upstream, will it also
> stop audio decoding ?
> 3) If decodebin stops receiving shmbuf from downstream shmsink 'shmvid',
> will it eventually stop decoding TS packets?
> 4) If decodebin stops receiving shmbuf from downstream, shmsink 'shmvid',
> what is the state of decoding when it starts receive buffers again?
>
> Oberservations and opinions:
>
> 1) It can be fatal if stopping video also stops audio, so that is to be
> avoided.
> 2) It can be fatal for the quality of the decoded streams, if TS packets
> are dropped before decoded, so that is to be avoided.
> 3) It can be fatal for the quality of the decoded video stream, if encoded
> video data is dropped before decoded because the the encoder otherwise will
> be in an unpredictable state and when started again (receiving buffers) we
> may get large pixelation, green or gray screen and weird moving blocks
> which is unacceptable for production of professional TV, though personal
> player may have this behaviour.
>
> So here is what I would like. I would like that the external process could
> send a message through the control pipe and instruct the shmsink that it
> temporarily is instructed not to signal more buffers ready for the process.
> That it self is not a very big difference, but the shmsink could use the
> information to signal upstream, that while it may be handing out shmbuffers
> upstream, it will temporarily not accept any more buffers downstream and a
> leaky queue upstream to shmsink can then leak(drop) buffers. Even better
> would it be if a decoder upstream can somehow use this information to
> understand, that it should not send decoded frames downstream, but it
> should continue to decode to keep an updated state. The latter may be
> unobtainable with the GStreamer (and decoder) design, but it would be nice
> to have to optimize resource usage. Even better would it be, if the decoder
> can stop decode while maintain a state and inspecting the encoeded stream
> so it when requested as soon as possible can start deliver a new decoded
> full frame.
>
> Sorry for the lengthy text, but it's a rather complicated subject.
>
> Best regards
> Peter Maersk-Moller
>
>
>
> On Fri, Sep 20, 2013 at 12:58 AM, Olivier Crête <
> olivier.crete at collabora.com> wrote:
>
>> On Thu, 2013-09-19 at 18:50 -0400, Olivier Crête wrote:
>> > One thing we could do is add properties to limit the size of "queue"
>> > that's inside shmsink by time or bytes or buffers (like the queue
>> > element). I'd certainly be open to something like that.
>>
>> Actually, there is already a buffer-time property to limit the size of
>> the "queue", but it is only triggered when the buffer is free on the
>> receiving side, probably not exactly what you want, but that could be
>> fixed.
>>
>> --
>> Olivier Crête
>> olivier.crete at collabora.com
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130922/4ac33d4c/attachment.html>