H264 and SEI messages

[Chuck Crisler]

Upon further investigate I found that the code I added in h264parse to
accumulate data in the adapter is causing the buffer pool SEI to be
processed twice for each I Frame. The time calculated is different for the
2 iterations. I guess that may be causing the problem.

Chuck


On Sat, Jan 25, 2014 at 3:44 PM, Chuck Crisler <ccrisler at mutualink.net>wrote:

> Sebastian, Thank You for your suggestion. However, it doesn't work.
> Remember, I am using 0.10.30 and bad plugins 0.10.20. Also, my pipeline is
> rather weird. It is basically
> udpsrc->mp2tdemux->h264parse->rtph264pay->udpsink (with some queues, etc.
> included). The mp2tdemux filter pushes random buffers. The size of those
> buffers ranges from 2 or 3 bytes up to 8,000 bytes or more. Those buffers
> do not correspond to AUs or NALs or frames or anything else that I can
> discern. Note: I don't know much about the MP2T spec. Early on I discovered
> that the rtph264pay filter was having problems with the buffers it was
> receiving, so I used the h264parse filter and modified it to collect data
> in the adapter until it identified the end of a NAL and then push that.
> RTPH264pay has an assert that the nal_queue->len must be 0 on entry, which
> implies to me that it really won't buffer data and expects entire frames,
> which is why I modified h264parse. (Please critique my actions if I have
> screwed up some aspect of h264parse due to my focusing on my specific
> problem set.)
>
> So I have modified h264parse to hold SEI NALs until the next video NAL and
> use the timestamp from that NAL. It pushes the SEI immediately before
> pushing the video buffer. That works and the SEI timestamp matches the
> video timestamp as verified in wireshark. That was very simple.
>
> The problem that I now have (and have probably had for some time) is that
> the timestamp on IFrames jumps by 59 frames (177000 ticks) from the
> previous PFrame. The next PFrame after the IFrame has a timestamp 3000
> after the PFrame before the IFrame. The timestamp sequence seems like the
> IFrame didn't exist. This *ONLY* happens with this one video source that is
> inserting SEI NALS. It is using buffer pool and picture timing SEI NALS.
> The buffer pool SEIs are grouped with the IFrames, so I suspect that is
> causing the timestamp error.
>
> I am slogging through this timestamp stuff which is pretty obtuse,
> especially with these SEIs. I believe that there is a bug here but probably
> fixed in the 1.0 series (aren't they all?). If anyone could please shed any
> insight on how these SEIs are supposed to contribute to timestamps I would
> really appreciate it.
>
> Chuck
>
>
> On Sat, Jan 25, 2014 at 5:44 AM, Sebastian Dröge <
> sebastian at centricular.com> wrote:
>
>> On Fr, 2014-01-24 at 11:00 -0500, Chuck Crisler wrote:
>> > I am using GStreamer 0.10.30 (sigh). I have an input source that is an
>> RTSP
>> > server that is using SEI messages. The stream is multiplexed into MP2T
>> and
>> > then converted back to RTP. The problem is that the SPS/PPS/SEI
>> messages on
>> > the output side have timestamps equal to the previous frame while on the
>> > input side the timestamps equal the next frame. When the output RTP is
>> sent
>> > to VLC on Windows, it doesn't display well. It pauses, then plays
>> several
>> > seconds very quickly, then pauses.
>> >
>> > 2 Questions.
>> >
>> > 1. Is it likely that the SEI timestamp change is responsible for this
>> issue?
>> > 2. Should I try to shift the SEI timestamps to the next frame in
>> rtph264pay?
>> >
>> > I don't have much experience with SEI messages. The RTP that I have
>> worked
>> > with in the past didn't use them.
>>
>> You might want to try if this is still like that with 1.0. In theory
>> (after h264parse if it outputs with alignment=AU) you will get
>> everything from the end of the previous frame until the end of the
>> current frame in a single buffer and thus with the timestamp.
>>
>> That means that any SPS/PPS/SEI that comes in front of a frame will be
>> bundled together with that frame in the same buffer with the same
>> timestamp.
>>
>> Everything else would seem like a bug. Not sure how it works in
>> alignment=NAL mode.
>>
>> --
>> Sebastian Dröge, Centricular Ltd - http://www.centricular.com
>> Expertise, Straight from the Source
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20140125/619ef608/attachment.html>