Decoding raw H.264 annex B stream

[4ernov]

Well, sorry for disturbing the list, that was my fault actually. In case of
someone would search for something similar here's a solution: the problem
is that the stream is actually not in Annex B format which can be played
successfully. It is AVC stream and I don't provide proper codec_data for
it, so it's impossible to decode such a stream. The solution is to set
"codec_data" field in the caps for appsrc:

GstBuffer* cd_buffer = gst_buffer_new_and_alloc(codec_data_size);
memcpy(GST_BUFFER_DATA(cd_buffer), codec_data, codec_data_size);
//...
GstCaps* caps = gst_caps_new_simple("video/x-h264", "codec_data",
GST_TYPE_BUFFER, cd_buffer, NULL);
//...
gst_app_src_set_caps(appsrc, caps);
// it's also necessary to set these caps to every input buffer which is
pushed to appsrc

Hope it helps someone.

Alexey Chernov


2012/11/10 4ernov <4ernov at gmail.com>

> Hello,
> I saw this case already discussed several times before but still quite
> unobvious one, so I decided to raise it once more.
> I've got a network client application which receives a stream of raw H.264
> video data encoded as annex b. The stream is depayed properly and video
> data packets go to avcodec_decode_video2() ffmpeg/libav function to be
> decoded like this:
>
> AVPacket pkt;
> pkt.dts = unit->pts();
> pkt.pts = unit->dts();
> pkt.data = unit->data();
> pkt.size = unit->size();
>
> if (unit->isRap()) {
>     pkt.flags |= AV_PKT_FLAG_KEY;
> }
>
> int isFrameFinished(0);
> if (0 >= avcodec_decode_video2(codecCtx_, frame_, &isFrameFinished, &pkt))
> {
>     printf("oops\n");
>     return;
> }
>
> if (isFrameFinished) {
>     *_frame = frame_;
>     *width = codecCtx_->width;
>     *height = codecCtx_->height;
>     *pixFmt = codecCtx_->pix_fmt;
> }
>
> The code above decodes the stream successfully. Now I need to switch the
> processing to GStreamer but got stuck with a problem. I use appsrc to
> inject data to the pipeline, the pipeline is like this:
>
> appsrc -> h264parse -> ffdec_h264 -> ffmpegcolorspace -> ximagesink
>
> I've set appsrc the following caps:
> video/x-h264,width=960,height=580,framerate=25/1 and push buffers with the
> same caps:
>
> GstBuffer* buffer =  gst_buffer_new();
> gst_buffer_set_data(buffer, const_cast<unsigned char*>(unit->data()),
> unit->size());
>
> if (unit->isRap())
>     GST_BUFFER_FLAGS(buffer) |= GST_BUFFER_FLAG_DELTA_UNIT;
>
> GstCaps* video_caps = gst_caps_new_simple ("video/x-h264",
>                                       "width", G_TYPE_INT, 960,
>                                       "height", G_TYPE_INT, 540,
>                                       "framerate", GST_TYPE_FRACTION, 25,
> 1,
>                          NULL);
>
> gst_buffer_set_caps(buffer, video_caps);
>     GstFlowReturn ret = gst_app_src_push_buffer(GST_APP_SRC(videosource),
> buffer);
>
> The problem is that ffdec_h264 element throws some error messages and
> provides no output, the pipeline is dry.  Here's some messages of interest
> from the LOG level output. ERROR level:
>
> 0:00:17.436071901 15944       0xfd8630 ERROR                 ffmpeg :0::
> non-existing PPS referenced
> 0:00:17.436199850 15944       0xfd8630 ERROR                 ffmpeg :0::
> slice type too large (0) at 0 0
> 0:00:17.436217450 15944       0xfd8630 ERROR                 ffmpeg :0::
> decode_slice_header error
> 0:00:17.436239240 15944       0xfd8630 ERROR                 ffmpeg :0::
> slice type too large (0) at 0 0
> 0:00:17.436255444 15944       0xfd8630 ERROR                 ffmpeg :0::
> decode_slice_header error
> 0:00:17.436335063 15944       0xfd8630 ERROR                 ffmpeg :0::
> no frame!
>
> and LOG-level (these messages seem to repeat for each buffer):
>
> 99:99:99.999999999, dur 99:99:99.999999999, size 109, offset
> 18446744073709551615, offset_end 18446744073709551615, caps: video/x-h264,
> width=(int)960, height=(int)540, framerate=(fraction)25/1,
> parsed=(boolean)true, stream-format=(string)byte-stream,
> alignment=(string)au
> 0:00:42.490123738 15944       0xfd8630 LOG                   ffmpeg
> gstffmpegdec.c:2622:gst_ffmpegdec_chain:<decoder> Received new data of size
> 109, offset:18446744073709551615, ts:99:99:99.999999999,
> dur:99:99:99.999999999, info 98
> 0:00:42.490146087 15944       0xfd8630 LOG                   ffmpeg
> gstffmpegdec.c:2660:gst_ffmpegdec_chain:<decoder> Calling av_parser_parse2
> with offset -1, ts:99:99:99.999999999 size 109
> 0:00:42.490170113 15944       0xfd8630 LOG                   ffmpeg
> gstffmpegdec.c:2669:gst_ffmpegdec_chain:<decoder> parser returned res 109
> and size 0, id 15
> 0:00:42.490186316 15944       0xfd8630 LOG                   ffmpeg
> gstffmpegdec.c:2683:gst_ffmpegdec_chain:<decoder> consuming 0 bytes. id 15
> 0:00:42.490222354 15944       0xfd8630 LOG           GST_SCHEDULING
> gstpad.c:4715:gst_pad_push:<parse:src> called chainfunction
> &gst_ffmpegdec_chain with buffer 0x7fd0bc171840, returned ok
> 0:00:42.528915821 15944       0xfd8630 LOG           GST_SCHEDULING
> gstpad.c:4708:gst_pad_push:<parse:src> calling chainfunction
> &gst_ffmpegdec_chain with buffer 0x7fd0bc35ec80, data 0x10f2220, malloc
> (nil), ts 99:99:99.999999999, dur 99:99:99.999999999, size 109, offset
> 18446744073709551615, offset_end 18446744073709551615, caps: video/x-h264,
> width=(int)960, height=(int)540, framerate=(fraction)25/1,
> parsed=(boolean)true, stream-format=(string)byte-stream,
> alignment=(string)au
>
> I've also tried to change a sink to filesink, but the output file is
> empty, too.
>
> Perhaps, someone could help me with the problem, maybe I should set more
> precise input caps or more buffer metadata. It is quite common use case
> when decoding raw H.264 stream, so any advice is particularly useful. My
> GStreamer version is 0.10.36.
>
> Thanks in advance!
>
> Alexey Chernov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20121117/fa89d6c6/attachment.html>