h264parse: nal.size of incoming frame is corrupted

Tue Mar 21 11:50:01 UTC 2017

Vivia,

Thanks for your attention.

I made a deep debug on this problem, however fall into difficult-to-understand situation.

First of all, I have to make a correction for the problematic pipeline.
------------------------------------------------------------------------------------
1. Problematic pipe line is as followings.

(NG)
        gst-launch-1.0 $VERBOSE filesrc location=$INPUT typefind=true ! qtdemux name=demux ! tee name=t \
        t. ! queue max-size-buffers=200000000 max-size-bytes=0 max-size-time=0 name=queue_byte_stream ! video/x-h264 ! h264parse  !  video/x-h264,stream-format=byte-stream ! mpegtsmux ! appsrc \
        t. ! queue max-size-buffers=0 max-size-bytes=0 max-size-time=0 name=queue_avc ! video/x-h264,stream-format=avc ! custom_dec ! custom_sink

(OK)
        gst-launch-1.0 $VERBOSE filesrc location=$INPUT typefind=true ! qtdemux name=demux ! tee name=t \
        t. ! queue max-size-buffers=200000000 max-size-bytes=0 max-size-time=0 name=queue_byte_stream ! video/x-h264 ! h264parse  !  video/x-h264,stream-format=byte-stream ! mpegtsmux ! appsrc \
        t. ! queue max-size-buffers=0 max-size-bytes=0 max-size-time=0 name=queue_avc ! video/x-h264,stream-format=avc ! fakesink

        gst-launch-1.0 $VERBOSE filesrc location=$INPUT typefind=true ! qtdemux name=demux ! tee name=t \
        t. ! queue max-size-buffers=200000000 max-size-bytes=0 max-size-time=0 name=queue_byte_stream ! video/x-h264 ! h264parse  !  video/x-h264,stream-format=byte-stream ! mpegtsmux ! appsrc

------------------------------------------------------------------------------------

2. Following description is valid for how to corrupt the data.
> I made debugging and it turned out that nalu.size is corrupted in NG case as shown below.
> Other fields are ok.
>
> (OK)
>    gst_h264_parse_handle_frame_packetized:<h264parse0>  processing packet buffer of size 32
>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 28           <= ok
>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 32
>    gst_h264_parse_process_nal:<h264parse0>processing nal of type 1 Slice, size 28
>
> (NG)
>    gst_h264_parse_handle_frame_packetized:<h264parse0> processing packet buffer of size 32
>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 1               <= nal size is corrupted to 1 !!
>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 5
>    gst_h264_parse_process_nal:<h264parse0>   not processing nal size 1
>
> nalu.size is got by the first 32 bit data of buffer in incoming frame to h264pasre plugin.

3. After digging into deeply,  it found out that deta corruption happens at queue_byte_stream  of "queue" element.
    - input data of queue_byte_stream of "queue" element is not corrupted.
    - output data of queue_byte_stream of "queue" element is corrupted.
        - data is fine at gst_queue_array_push_tail() in gst_queue_locked_enqueue_buffer()
         - data is corrupted at gst_queue_array_pop_head () in gst_queue_locked_dequeue ()

Another information regarding queue of "queue"
   - buffer underrun/overflow is not happened.
   - the output data of queue_avc  of "queue" element is not corrupted.

I can not understand at all why the data of queue is corrupted in spite of that the buffer underrun/overflow is not happened.
Even if underrun/overflow happened, the whole buffer should be dropped.

Any idea how to debug this issue furthermore ?

Kind Regards,
Makoto

On 2017/03/20 23:47, Vivia Nikolaidou wrote:
> Hi,
>
> This looks like a bug in h264parse. Please file a bug here:
> https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer (should be
> gst-plugins-bad)
>
> Make sure you include the debug output of one OK and one NG case (run
> with the env var GST_DEBUG=*:6 and GST_DEBUG_NO_COLOR=1)
>
> Thank you very much!
>
> Best regards,
>
> Vivia
>
> On 17 March 2017 at 08:57, Makoto Harada <makotouu at gmail.com> wrote:
>>> (NG)
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>>>
>> ops.
>> sorry appsrc should be appsink.
>>
>> Makoto
>>
>> On 2017/03/16 20:03, Makoto Harada wrote:
>>> Hello experts,
>>>
>>> I'm now investigating the problem that h264parse does not work as expected in some cases.
>>>
>>> My pipelines are as shown below.
>>>
>>> (OK)
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' filesink location=test.h264
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! filesink location=test.ts
>>>
>>> (NG)
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>>>
>>> In OK case, h264parse works as expected and the test.h264/test.ts has correct data. We can decode this data as expected.
>>> In NG case, h264parse outputs corrupted h264 data.
>>> I see lot's of the "not processing nal size 1" message, when GST_DEBUG="h264parse:5".
>>>
>>> I made debugging and it turned out that nalu.size is corrupted in NG case as shown below.
>>> Other fields are ok.
>>>
>>> (OK)
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0>  processing packet buffer of size 32
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 28           <= ok
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 32
>>>    gst_h264_parse_process_nal:<h264parse0>processing nal of type 1 Slice, size 28
>>>
>>> (NG)
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> processing packet buffer of size 32
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 1               <= nal size is corrupted to 1 !!
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 5
>>>    gst_h264_parse_process_nal:<h264parse0>   not processing nal size 1
>>>
>>> nalu.size is got by the first 32 bit data of buffer in incoming frame to h264pasre plugin.
>>> I confirmed this first 32 bit is broken at gst_h264_parse_handle_frame () while data size is expected.
>>>
>>> version of h264parse I'm using is 1.4.1.
>>>
>>> Question:
>>> 1. Is this known issue ? I could not find out the commit in git.
>>> 2. Who writes to this nalu.size field (first 32 bit of incoming frame to h264parse ) ?
>>>     parent class of h264parse or qtdemux ?
>>>     I could not figure it out the source code which does this yet.
>>>
>>> Any feedback is highly appreciated.
>>>
>>> Kind Regards,
>>> Makoto
>>>
>>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel