Encoding and decoding of RAW RTSP video frames

Nicolas Dufresne nicolas at ndufresne.ca
Thu May 23 17:43:39 UTC 2019


Le jeudi 23 mai 2019 à 22:04 +0530, Raghu Msrp a écrit :
> Dear List,
> 
> I was interested to know if gstreamer rtsp can handle transmission of pure RAW video frames to a client. To be more specific, my question here is, if I transfer pure raw video frames with no encoding to a client using rtsp protocol, do I still need to do decoding of frames? If yes, can someone in the list explain the reason?

We have support for raw video streaming in GStreamer RTP stack. In your
RTSP server, just use "... ! videoconvert ! rtpvrawpay name=pay0". You
don't have to "decode" the video, and there is no quality lost. Though
the frame is packetized in smaller chunk, the depayloader on the client
will collect and reassemble the image.

> 
> Secondly, in gstreamer, if I use raw encoding on video frames, what kind of encoding is performed on the frames? Does it compress the frames to a larger extent?

there is no encoding performed. The raw data is split into chunk and
prepended with an RTP header.

> 
> My understanding is if I encode the frame using h264 or vp8 or mpeg then only I have to decode. Please correct my understanding if I am wrong.

That's correct.

> 
> On the raw frames, can I use rtsp gstreamer protocol to visualize without decoding it. Is decoding mandatory?

That's an ambiguous question, did you mean to ask if you can display
h264/vp8 without decoding, or raw video without decoding ?

> 
> Request the list to kindly clarify the above queries.
> 
> 
> On Wed, May 22, 2019, 19:45 <gstreamer-devel-request at lists.freedesktop.org> wrote:
> > Send gstreamer-devel mailing list submissions to
> >         gstreamer-devel at lists.freedesktop.org
> > 
> > To subscribe or unsubscribe via the World Wide Web, visit
> >         https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > or, via email, send a message with subject or body 'help' to
> >         gstreamer-devel-request at lists.freedesktop.org
> > 
> > You can reach the person managing the list at
> >         gstreamer-devel-owner at lists.freedesktop.org
> > 
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of gstreamer-devel digest..."
> > 
> > 
> > Today's Topics:
> > 
> >    1. Re: Sound volume is very low when playing mkv file (longkas)
> >    2. Change RTSP Server source IP address (David Jaggard)
> >    3. AW: Change RTSP Server source IP address (Thornton, Keith)
> >    4. Re: Change RTSP Server source IP address (David Jaggard)
> >    5. Re: Sound volume is very low when playing mkv file (longkas)
> >    6. Re: Sound volume is very low when playing mkv file (Tim Müller)
> >    7. Re: MPEG-TS the ongoing story (Russel Winder)
> > 
> > 
> > ----------------------------------------------------------------------
> > 
> > Message: 1
> > Date: Wed, 22 May 2019 07:23:34 -0500 (CDT)
> > From: longkas <longkas at gmail.com>
> > To: gstreamer-devel at lists.freedesktop.org
> > Subject: Re: Sound volume is very low when playing mkv file
> > Message-ID: <1558527814965-0.post at n4.nabble.com>
> > Content-Type: text/plain; charset=us-ascii
> > 
> > I'm not sure if it is because the audio stream in the mkv has 6 channels, I'm
> > using decodebin to translate streams, maybe I could combine it into 2
> > channel audio streams, I need some help
> > 
> > 
> > BR
> > 
> > 
> > 
> > --
> > Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> > 
> > 
> > ------------------------------
> > 
> > Message: 2
> > Date: Wed, 22 May 2019 14:08:52 +0100
> > From: David Jaggard <davywj at gmail.com>
> > To: Discussion of the development of and with GStreamer
> >         <gstreamer-devel at lists.freedesktop.org>
> > Subject: Change RTSP Server source IP address
> > Message-ID:
> >         <CAJAmR4ZOSjBpSeOKRc0fZ6A71uwgVXU4C9W1c2qFPtGhT-6_Rg at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> > 
> > Hello
> > 
> > Using the example in the test-readme.c file of the RTSP server, how can I
> > change the source IP address of the RTP packets to specific address on a
> > multi-homed PC?
> > 
> > At present it is choosing the first address.
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190522/b90779fe/attachment-0001.html>
> > 
> > ------------------------------
> > 
> > Message: 3
> > Date: Wed, 22 May 2019 13:23:35 +0000
> > From: "Thornton, Keith" <keith.thornton at zeiss.com>
> > To: Discussion of the development of and with GStreamer
> >         <gstreamer-devel at lists.freedesktop.org>
> > Subject: AW: Change RTSP Server source IP address
> > Message-ID: <7153D1686E3590498D57D986E0F83BA60ADCB889 at adeerl01sms004>
> > Content-Type: text/plain; charset="utf-8"
> > 
> > Hi,
> > After calling gst_rtsp_server_new you can set the address with gst_rtsp_server_set_address otherwise it will use 127.0.0.1
> > 
> > Von: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> Im Auftrag von David Jaggard
> > Gesendet: Mittwoch, 22. Mai 2019 15:09
> > An: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
> > Betreff: Change RTSP Server source IP address
> > 
> > Hello
> > 
> > Using the example in the test-readme.c file of the RTSP server, how can I change the source IP address of the RTP packets to specific address on a multi-homed PC?
> > 
> > At present it is choosing the first address.
> > 
> > 
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190522/7a369e10/attachment-0001.html>
> > 
> > ------------------------------
> > 
> > Message: 4
> > Date: Wed, 22 May 2019 14:27:49 +0100
> > From: David Jaggard <davywj at gmail.com>
> > To: Discussion of the development of and with GStreamer
> >         <gstreamer-devel at lists.freedesktop.org>
> > Subject: Re: Change RTSP Server source IP address
> > Message-ID:
> >         <CAJAmR4aph0A0O2Q2ooFqX9+a7iUq7-qUTovKmJmWwm1kipiwug at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> > 
> > Hi Keith,
> > 
> > Thanks for your reply. That sets the listen address of the RTSP server but
> > doesn't change the src address of the RTP packets.
> > 
> > On Wed, 22 May 2019 at 14:23, Thornton, Keith <keith.thornton at zeiss.com>
> > wrote:
> > 
> > > Hi,
> > >
> > > After calling gst_rtsp_server_new you can set the address with
> > > gst_rtsp_server_set_address otherwise it will use 127.0.0.1
> > >
> > >
> > >
> > > *Von:* gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> *Im
> > > Auftrag von *David Jaggard
> > > *Gesendet:* Mittwoch, 22. Mai 2019 15:09
> > > *An:* Discussion of the development of and with GStreamer <
> > > gstreamer-devel at lists.freedesktop.org>
> > > *Betreff:* Change RTSP Server source IP address
> > >
> > >
> > >
> > > Hello
> > >
> > >
> > >
> > > Using the example in the test-readme.c file of the RTSP server, how can I
> > > change the source IP address of the RTP packets to specific address on a
> > > multi-homed PC?
> > >
> > >
> > >
> > > At present it is choosing the first address.
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > gstreamer-devel at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190522/7b8630e1/attachment-0001.html>
> > 
> > ------------------------------
> > 
> > Message: 5
> > Date: Wed, 22 May 2019 08:50:28 -0500 (CDT)
> > From: longkas <longkas at gmail.com>
> > To: gstreamer-devel at lists.freedesktop.org
> > Subject: Re: Sound volume is very low when playing mkv file
> > Message-ID: <1558533028272-0.post at n4.nabble.com>
> > Content-Type: text/plain; charset=us-ascii
> > 
> > Ok, my research is to use volume element to raise the volume, and use
> > audioconvert to downmix the 5.1 DTS audio to stereo, I dont know how to find
> > the current volume though:
> > 
> > gst-launch-1.0 -v filesrc location=g:/gtg.mkv ! decodebin ! 
> > audio/x-raw,channels=6 \
> >      ! audioconvert mix-matrix="<<(float)1.0, (float)0.0, (float)1.0,
> > (float)1.0, (float)1.0, (float)0.0>, <(float)0.0, (float)1.0, (float)1.0,
> > (float)1.0, (float)0.0,(float)1.0>>" !   audio/x-raw,channels=2  \
> >     ! volume volume=10.0  ! autoaudiosink
> > 
> > 
> > 
> > 
> > 
> > --
> > Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> > 
> > 
> > ------------------------------
> > 
> > Message: 6
> > Date: Wed, 22 May 2019 15:00:29 +0100
> > From: Tim Müller <tim at centricular.com>
> > To: Discussion of the development of and with GStreamer
> >         <gstreamer-devel at lists.freedesktop.org>
> > Subject: Re: Sound volume is very low when playing mkv file
> > Message-ID:
> >         <748661b0e32dc2178225fa7a9ff0f4ca5a518717.camel at centricular.com>
> > Content-Type: text/plain; charset="UTF-8"
> > 
> > On Wed, 2019-05-22 at 07:23 -0500, longkas wrote:
> > 
> > > I'm not sure if it is because the audio stream in the mkv has 6
> > > channels, I'm using decodebin to translate streams, maybe I could
> > > combine it into 2channel audio streams, I need some help
> > 
> > What codec/format is the audio stream (gst-discoverer-1.0 -v file.mkv)?
> > 
> > And what decoder is being used in your case? (see output of gst-play-
> > 1.0 -v or gst-launch-1.0 -v)
> > 
> > Is the decoder outputting 2 channels or 6 channels?
> > 
> > It might be that this is caused by e.g. AC-3 audio being downmixed to 2
> > channels improperly.
> > 
> > Cheers
> >  -Tim
> > -- 
> > Tim Müller, Centricular Ltd - http://www.centricular.com
> > 
> > 
> > 
> > ------------------------------
> > 
> > Message: 7
> > Date: Wed, 22 May 2019 15:15:06 +0100
> > From: Russel Winder <russel at winder.org.uk>
> > To: Discussion of the development of and with GStreamer
> >         <gstreamer-devel at lists.freedesktop.org>
> > Subject: Re: MPEG-TS the ongoing story
> > Message-ID:
> >         <702b56ef5099a0db13a551d981ebe810d1dab8fb.camel at winder.org.uk>
> > Content-Type: text/plain; charset="utf-8"
> > 
> > On Wed, 2019-05-22 at 13:20 +0300, Sebastian Dröge wrote:
> > > On Wed, 2019-05-22 at 10:32 +0100, Russel Winder wrote:
> > > > I am wondering if I am missing something other than "it's never been
> > > > needed before". 
> > > 
> > > That seems to be the case, yes. Please send a patch if you need it :)
> > 
> > I am guessing then that I need to do the C thing for entry into the MPEG-TS
> > library itself rather than just fill the gap in Rust in the MPEG-TS wrapper.
> > And I promised myself never to do any C programming again. Oh well.
> > 
> > Would changes get into GStreamer 1.16.1 or would it have to wait for 1.18.0? 
> > 
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190523/a3bc4880/attachment.sig>


More information about the gstreamer-devel mailing list