H.264 UDP recording improvements?
Gerard Toonstra
gtoonstra at gmail.com
Thu Jan 17 12:41:22 PST 2013
On Jan 17, 2013, at 5:20 PM, Tim-Philipp Müller <t.i.m at zen.co.uk> wrote:
> On Thu, 2013-01-17 at 17:07 -0300, Gerard Toonstra wrote:
>
> Hi,
>
>> Well, one step further. I'm not sure how well this is supposed to
>> work, but I'm getting consistent results.
>>
>>
>> I've added config-interval to the h264parse which on a failed run gave
>> me the codec_data buffer. I picked up this codec_data buffer and
>> inserted this into the caps before the mp4mux before the filesink.
>> The matroskamux allows it, but produces an incorrect file. The mp4mux
>> now produces a playable file with only some incorrect frames at the
>> start, until the next keyframe.
>>
>>
>> ./gst-launch-0.10 -e -vvv udpsrc multicast-group=239.255.12.12
>> port=5004 auto-multicast=true
>> caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264,payload=(int)96" typefind=true ! rtph264depay ! tee name=my_sink ! queue ! "video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal" ! ffdec_h264 ! videorate ! ffmpegcolorspace ! autovideosink sync=false my_sink. ! queue ! h264parse config-interval=50 ! "video/x-h264,width=(int)1280,height=(int)720,codec_data=(buffer)01428033ffe1000967428033e900a00b7201000468ce3c80" ! mp4mux ! filesink location=/Users/gt/movie.mp4
>>
>>
>>
>>
>> When I code this into a C application however, h264parse doesn't want
>> to be linked to mp4mux. The (buffer) type is a special type that
>> needs some special care. Here's what I'm doing:
>>
>>
>> GstBuffer *codec_data;
>> GValue value = { 0 };
>> g_value_init (&value, GST_TYPE_BUFFER);
>> gst_value_deserialize (&value,
>> "01428033ffe1000967428033e900a00b7201000468ce3c80");
>> codec_data = gst_value_get_buffer (&value);
>>
>>
>>
>> incaps = gst_caps_new_simple("video/x-h264",
>> "width", G_TYPE_INT, "1280",
>> "height", G_TYPE_INT, "720",
>> "codec_data", GST_TYPE_BUFFER,
>> codec_data,
>> NULL) ;
>> link_ok = gst_element_link_filtered(h264parse, mp4mux, incaps);
>> gst_caps_unref(incaps) ;
>> g_value_unset (&value);
>> if (!link_ok) {
>> g_print ("Failed to link h264parse to mp4mux!\n");
>> return;
>> }
>
> It sounds like https://bugzilla.gnome.org/show_bug.cgi?id=646327 might
> be related.
>
> Here's a question: if you pass -v to gst-launch, do you see the
> h264parse src caps changing at some point to contain codec_data?
>
yes (but because I'm specifying this). Without coded_data in the properties, both qtmux and mp4mux error out on "not negotiated".
Mac-mini:Commands gt$ ./gst-launch-0.10 -e -vvv udpsrc multicast-group=239.255.12.12 port=5004 auto-multicast=true caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264,payload=(int)96" typefind=true ! rtph264depay ! tee name=my_sink ! queue ! "video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal" ! ffdec_h264 ! videorate ! ffmpegcolorspace ! autovideosink sync=false my_sink. ! queue ! h264parse config-interval=50 ! "video/x-h264,width=(int)1280,height=(int)720,codec_data=(buffer)01428033ffe1000967428033e900a00b7201000468ce3c80" ! mp4mux ! filesink location=/Users/gt/movie.mp4
……...
/GstPipeline:pipeline0/ffdec_h264:ffdec_h2640.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, parsed=(boolean)true
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, parsed=(boolean)true, width=(int)1280, height=(int)720, codec_data=(buffer)01428033ffe1000967428033e900a00b7201000468ce3c80
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, parsed=(boolean)true
/GstPipeline:pipeline0/GstMP4Mux:mp4mux0.GstPad:video_00: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, parsed=(boolean)true, width=(int)1280, height=(int)720, codec_data=(buffer)01428033ffe1000967428033e900a00b7201000468ce3c80
/GstPipeline:pipeline0/GstMP4Mux:mp4mux0.GstPad:src: caps = video/quicktime, variant=(string)iso
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/quicktime, variant=(string)iso
/GstPipeline:pipeline0/ffdec_h264:ffdec_h2640.GstPad:src: caps = video/x-raw-yuv, width=(int)1280, height=(int)720, framerate=(fraction)25/1, format=(fourcc)I420, interlaced=(boolean)false
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps = video/x-raw-yuv, width=(int)1280, height=(int)720, framerate=(fraction)25/1, format=(fourcc)I420, interlaced=(boolean)false
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = video/x-raw-yuv, width=(int)1280, height=(int)720, framerate=(fraction)25/1, format=(fourcc)I420, interlaced=(boolean)false
/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = video/x-raw-yuv, framerate=(fraction)25/1, width=(int)1280, height=(int)720, format=(fourcc)UYVY, interlaced=(boolean)false
/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = video/x-raw-yuv, width=(int)1280, height=(int)720, framerate=(fraction)25/1, format=(fourcc)I420, interlaced=(boolean)false
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = video/x-raw-yuv, framerate=(fraction)25/1, width=(int)1280, height=(int)720, format=(fourcc)UYVY, interlaced=(boolean)false
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw-yuv, framerate=(fraction)25/1, width=(int)1280, height=(int)720, format=(fourcc)UYVY, interlaced=(boolean)false
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = video/x-raw-yuv, framerate=(fraction)25/1, width=(int)1280, height=(int)720, format=(fourcc)UYVY, interlaced=(boolean)false
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstOSXVideoSink:autovideosink0-actual-sink-osxvideo.GstPad:sink: caps = video/x-raw-yuv, framerate=(fraction)25/1, width=(int)1280, height=(int)720, format=(fourcc)UYVY, interlaced=(boolean)false
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, width=(int)1280, height=(int)720, parsed=(boolean)true, codec_data=(buffer)01428033ffe1000967428033e900a00b7201000468ce3c80
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, width=(int)1280, height=(int)720, parsed=(boolean)true, codec_data=(buffer)01428033ffe1000967428033e900a00b7201000468ce3c80
> Also, with h264 caps it's best to specify the stream-format and
> alignment as well: matroskamux and qtmux want
>
> video/x-h264, stream-format=avc, alignment=au
>
> the depayloader should/might be able to output that directly
> (byte-stream=false access-unit=true properties if it doesn't negotiate
> it correctly from the caps).
>
Matroskamux produces an unreadable file. qtmux and mp4mux both work, but the initial frames are a mess.
It cleans up quickly though and I don't really need 100% correct frames. would be good as indicated in the bug report to dump frames until
first keyframe + coded_data information.
The failure to link issue was caused here:
>> "width", G_TYPE_INT, "1280", <-- should be 1280
>> "height", G_TYPE_INT, "720", <-- should be 720
It's working now as far as I can tell, but I still need to generate an EOS message when the app terminates, then I'll know for sure.
Thanks for your help.
> Cheers
> -Tim
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
More information about the gstreamer-devel
mailing list