forwarded screen stutters every several seconds

[Peter Maersk-Moller]

Hi Henry.

One way to synchronize audio and video is to mux these two at the sender
end. You can mux H.264 and AAC using matroskamux or mpegtsmux. Matroskamux
require use of tcp while TS mux can use UDP. So no need to transport a
large amount of raw audio samples. Also using encoded audio, your decoder
can detect gaps in audio. Sending raw audio introduces challenges you may
not want to deal with.

Matroskamux can take Vorbis. Various muxes has various properties when it
comes to payload types and transport requirements. If the Pi gets too
loaded decoding the audio too, then use lower sample rate and/or mono or
even better, throw out the Pi and buy Pi2s.

Here is the pipeline I use on the RaspPis

        gst-launch-1.0 -v udpsrc port=$port caps="$caps" !\
                rtpjitterbuffer         !\
                decodebin name=decoder  !\
                autovideosink sync=true decoder. !\
                alsasink sync=true

Note that running ntpd on both sender and receiver helps get boths system
clock to drift very little compared, but you will need to run ntpd over
longer time so the ntpd detects the drift of the Pi's harware clock
correctly and if you place the Pi in an environment with large temperature
changes, ntpd has a delay picking up the changes due to temperature changes
in the drift of the internal clock

Best regards
Peter


On Mon, Jan 18, 2016 at 1:21 PM, henryco <fhenryco at yahoo.fr> wrote:

> OK, thank you, i eventually made it work with a huge udpsrc
> buffer-size=50000000 just to be sure this not anymore a limitation ... and
> i
> will come back on this later
>
> As for the sound i eventually decided to send it uncompressed (LAN is not
> bandwidth limited) to avoid charge the Pi too much.  i get both sound and
> image together almost perfect (one sound crack from time to time) ... dont
> know if i can do better , with simple following pipeline for the sound
>
> gst-launch-1.0 -v pulsesrc  ! audioconvert ! rtpL16pay ! udpsink
> host=192.168.1.19 port=5500
>
> gst-launch-1.0 -v udpsrc buffer-size=50000000 port=5500 \
> caps="application/x-rtp, media=(string)audio, clock-rate=(int)44100,
> encoding-name=(string)L16, encoding-params=(string)2, channels=(int)2,
> payload=(int)96" \
> ! rtpL16depay ! pulsesink
>
> and for the video
>
> gst-launch-1.0 -v ximagesrc  use-damage=true ! queue  ! videoflip method=5
> !
> videoconvert  ! videoscale \
> !
>
> "video/x-raw,framerate=(fraction)30/1,format=(string)I420,force-aspect-ratio=(gboolean)true"
> \
> ! queue ! x264enc tune=zerolatency  bitrate=5000 speed-preset=ultrafast  !
> queue ! rtph264pay  pt=96 \
> ! udpsink host=192.168.1.19 port=5000
>
> gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp,
> media=(string)video, clock-rate=(int)90000,
> encoding-name=(string)H264" ! rtpjitterbuffer ! rtph264depay ! queue !
> h264parse ! queue ! omxh264dec !
> queue ! eglglessink
>
> the next step is to try to solve the delay : sound is very late though not
> compressed , and to share usb mouse and keyboard with virtualhere to get
> the
> homemade thinclient...i'm beginning to believe in it... any advices welcome
>
>
>
> --
> View this message in context:
> http://gstreamer-devel.966125.n4.nabble.com/forwarded-screen-stutters-every-several-seconds-tp4675279p4675336.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160118/2e75d8e6/attachment-0001.html>