On-demand video encoding in different pipelines/bins

[Krutskikh Ivan]

Hi!

I want to achieve a scenario when a single video decoder upon client demand
would feed various encoders. And when demand is gone the encoders should be
stopped and recycled upon the next connection. I've tried 3 ways to achieve
my goal and all failed to some extent.

1) Multifdsink / raw video

    Main decoder :

    fdsrc ! tsdemux ! decodebin ! videoconvert ! videorate !  multifidsink

    Sample encoder (Separate pipeline):

    fdsrc ! rawvideoparse use-sink-caps=1 ! "caps from multifdsink inserted
here" ! videoconvert ! videorate ! videoscale ! x264enc tune=zerolatency
key-int-max=5 ! mpegtsmux ! multifdsink


    This way I can start/stop encoders at ease but I get a color mess of a
picture no mater what caps do multifdsink provide


2) Multiudpsink / raw video

    Main decoder:

    fdsrc ! tsdemux ! decodebin ! videoconvert ! videorate !  rtpvrawpay !
multiudpsink

    Sample encoder (Separate pipeline):

    udpsrc caps="caps from multiudpsink inserted here" ! rtpvrawdepay !
videoconvert ! videorate ! videoscale ! x264enc tune=zerolatency
key-int-max=5 ! mpegtsmux ! multifdsink


    This way I can start/stop encoders at ease. I even get a nice picture
when there's little motion on the scene, but when motion is introduced I
get a lot of artifacts like missing parts of walking man. Not to mention
that I get 50Mbit of network trafic on lo for a single 320X240 stream

3) Dynamic pipelines usage

    (Mostly inspired by
https://github.com/MaZderMind/dynamic-gstreamer-pipelines-cookbook/blob/master/05-add-and-remove-network-sink.py
)

    Main decoder:

    fdsrc ! tsdemux ! decodebin ! videoconvert ! videorate !  tee

    Sample encoder (Gst.Bin):

    queue ! videoconvert ! videorate ! videoscale !  x264enc
tune=zerolatency key-int-max=5 ! mpegtsmux ! multifdsink


    Here I start the main decoder and on demand add a Gst.Bin containing
sample encoder. First time (no encoder was requested yet) I can get a
stream with visible delay of 1-2 seconds. But when encoder get stopped and
detached from pipeline on the next demand I get a still frame with players
actually indicate time ticking (encoder pushes the same frame all over
again)


    Code to add encoder to main pipeline:

    self.motion_debug = self.create_encoder_pipeline('motion',
encoder='h264')
    self.pipeline.add(self.motion_debug)
    tee = self.pipeline.get_by_name('motion_tee')
    print(tee.link(self.motion_debug))
    self.motion_debug.set_state(Gst.State.PLAYING)


    That goes with Link_OK every time

    Code to remove encoder from main pipeline


    ghostpad = self.motion_debug.get_static_pad("sink")
    teepad = ghostpad.get_peer()
    teepad.add_probe(Gst.PadProbeType.BLOCK, self.blocking_pad_probe)

    def blocking_pad_probe(self,pad, info):
        print("Stopping Bin")
        self.motion_debug.set_state(Gst.State.NULL)


        tee = self.pipeline.get_by_name('motion_tee')
        ghostpad = self.motion_debug.get_static_pad("sink")
        teepad = ghostpad.get_peer()

        print('Removing encoder bin')
        print(self.pipeline.remove(self.motion_debug))

        print("Releasing Tee-Pad")
        tee.release_request_pad(teepad)

        self.motion_debug = None


        return Gst.PadProbeReturn.REMOVE


I believe that the most right way to solve my problem would be the 3rd
approach. But I welcome any solution or thought.

Thanks in advance!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230819/2b13ae3c/attachment.htm>