How to add audio to a h264 video stream using gstreamer

[Tim Müller]

On Fri, 2015-07-31 at 07:10 -0700, suspension wrote:

Hi,

> I can successfully stream HD video using following pipelines:
> 
> stream server:
> 
> gst-launch-1.0 filesrc location="Gravity.2013.720p.BluRay.x264.YIFY.mp4" !
> decodebin ! x264enc ! rtph264pay  pt=96 ssrc=0 timestamp-offset=0
> seqnum-offset=0 pt=96 !  gdppay ! tcpclientsink host=192.168.1.93 port=5000

Here you could just demux the stream and feed the H264 from the file to
the h264 payloader directly without re-encoding.

> client:
> gst-launch-1.0 tcpserversrc host=192.168.1.93 port=5000 ! gdpdepay !
> rtph264depay ! decodebin ! autovideosink
> 
> I want to add the audio stream too. I guess it is possible to use a
> different port and tcpserver/tcpclient combination to stream audio parallel
> to video. But I am not certain, how gstreamer would synchronize two streams
> properly to play the movie in the client end. Apart from this method, are
> there any other methods? such as muxing two streams before and demuxing it
> in client end?

The easiest method would be to just use RTSP instead, it can send the
data interleaved over the TCP connection if UDP is not an option.

For that you could use gst-rtsp-server. e.g. from
gst-rtsp-server/examples/:

$ ./test-uri file:///path/to/foo.mp4
stream ready at rtsp://127.0.0.1:8554/test

And then as client for example:

$ gst-play-1.0 rtsp://127.0.0.1:8554/test

or

$ gst-play-1.0 rtspt://127.0.0.1:8554/test

to force interleaved TCP transport.

This has the added benefit that it can be used with other
clients/players and seeking is also possible.

 Cheers
  -Tim

-- 
Tim Müller, Centricular Ltd - http://www.centricular.com