shmsink changes from 1.0.10 to 1.1.4

[Peter Maersk-Moller]

Hi Olivier

I'm afraid that's not exactly what I'm looking for. Also my pipeline
example didn't detailed the problem clear enough, so please allow me to try
again. Assuming you have the following pipeline:

  gst-launch udpsrc ! 'video/mpgets' !\
    queue name=qts leaky=2 ! decodebin !\
    'video/x-raw' ! queue name=qvid leaky=2 ! shmsink name=shmvid \
    'audio/x-raw' ! queue name=qaud leaky=2 ! shmsink name=shmaud

Also assume we have two other processes that process the buffers passed by
the two shmsinks and acknowledge the buffers when read.

The leaky queue 'qts' is included to point out, that if decodebin stops or
don't process buffers received from upstream fast enough, TS packets are
lost. In fact that would happen anyway as an UDP connection in principle is
a queue anyway and that queue is leaky in case data is not read fast
enough. This is a hard fact and not something we can control. In fact I
would prefer I could guarantee that decodebin always receives all packets.

The queues 'qvid' and 'qaud' are included to illustrate that I would like
raw video or raw audio to be dropped frame by frame if the processes that
shmsink delivers buffers are not fast enough or decide it is desired that
raw frames are dropped. However I believe decodebin may also include queues
on its src pads (output), but they are harder to control individually.

Now I think the pipeline works this way, that the shmsinks passes shmbufs
upstream to decodebin for decodebin to fill with raw video and raw audio
frames. The shmsinks receives these buffers and signal to the two external
processes that shmbuffers are ready to be read.

Lets now assume that the process reading the raw video frames in the
pipeline would like to signal to the gstreamer pipeline, that the pipeline
temporarily can halt sending decoded frames. Right now the only way to do
that is by not sending Acks through the control pipe for buffers it has
received. However, the shmsink will continue handing out shmbuffers
upstream and signal received buffers ready to be read until it has no more
buffers. Now when the shmsink 'shmvid' has no more buffers to hand out, I
believe decodebin will stop decode video data. I fear this will also stop
the decoder from sending audio data upstream which can be fatal. I also
fear that if shmsink 'shmvid' stops, decodebin will stop sending buffers
upstream leading to an unpredictable loss of packets.

My questions are now these:

1) Am I correct that decodebin will stop decode video if it stops receiving
ready shmbuffers from downstream?
2) If shmsink 'shmvid' stops sending shmbuffers upstream, will it also stop
audio decoding ?
3) If decodebin stops receiving shmbuf from downstream shmsink 'shmvid',
will it eventually stop decoding TS packets?
4) If decodebin stops receiving shmbuf from downstream, shmsink 'shmvid',
what is the state of decoding when it starts receive buffers again?

Oberservations and opinions:

1) It can be fatal if stopping video also stops audio, so that is to be
avoided.
2) It can be fatal for the quality of the decoded streams, if TS packets
are dropped before decoded, so that is to be avoided.
3) It can be fatal for the quality of the decoded video stream, if encoded
video data is dropped before decoded because the the encoder otherwise will
be in an unpredictable state and when started again (receiving buffers) we
may get large pixelation, green or gray screen and weird moving blocks
which is unacceptable for production of professional TV, though personal
player may have this behaviour.

So here is what I would like. I would like that the external process could
send a message through the control pipe and instruct the shmsink that it
temporarily is instructed not to signal more buffers ready for the process.
That it self is not a very big difference, but the shmsink could use the
information to signal upstream, that while it may be handing out shmbuffers
upstream, it will temporarily not accept any more buffers downstream and a
leaky queue upstream to shmsink can then leak(drop) buffers. Even better
would it be if a decoder upstream can somehow use this information to
understand, that it should not send decoded frames downstream, but it
should continue to decode to keep an updated state. The latter may be
unobtainable with the GStreamer (and decoder) design, but it would be nice
to have to optimize resource usage. Even better would it be, if the decoder
can stop decode while maintain a state and inspecting the encoeded stream
so it when requested as soon as possible can start deliver a new decoded
full frame.

Sorry for the lengthy text, but it's a rather complicated subject.

Best regards
Peter Maersk-Moller



On Fri, Sep 20, 2013 at 12:58 AM, Olivier Crête <olivier.crete at collabora.com
> wrote:

> On Thu, 2013-09-19 at 18:50 -0400, Olivier Crête wrote:
> > One thing we could do is add properties to limit the size of "queue"
> > that's inside shmsink by time or bytes or buffers (like the queue
> > element). I'd certainly be open to something like that.
>
> Actually, there is already a buffer-time property to limit the size of
> the "queue", but it is only triggered when the buffer is free on the
> receiving side, probably not exactly what you want, but that could be
> fixed.
>
> --
> Olivier Crête
> olivier.crete at collabora.com
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130920/885963ab/attachment-0001.html>