H264 and SEI messages

[Chuck Crisler]

Sebastian, Thank You for your suggestion. However, it doesn't work.
Remember, I am using 0.10.30 and bad plugins 0.10.20. Also, my pipeline is
rather weird. It is basically
udpsrc->mp2tdemux->h264parse->rtph264pay->udpsink (with some queues, etc.
included). The mp2tdemux filter pushes random buffers. The size of those
buffers ranges from 2 or 3 bytes up to 8,000 bytes or more. Those buffers
do not correspond to AUs or NALs or frames or anything else that I can
discern. Note: I don't know much about the MP2T spec. Early on I discovered
that the rtph264pay filter was having problems with the buffers it was
receiving, so I used the h264parse filter and modified it to collect data
in the adapter until it identified the end of a NAL and then push that.
RTPH264pay has an assert that the nal_queue->len must be 0 on entry, which
implies to me that it really won't buffer data and expects entire frames,
which is why I modified h264parse. (Please critique my actions if I have
screwed up some aspect of h264parse due to my focusing on my specific
problem set.)

So I have modified h264parse to hold SEI NALs until the next video NAL and
use the timestamp from that NAL. It pushes the SEI immediately before
pushing the video buffer. That works and the SEI timestamp matches the
video timestamp as verified in wireshark. That was very simple.

The problem that I now have (and have probably had for some time) is that
the timestamp on IFrames jumps by 59 frames (177000 ticks) from the
previous PFrame. The next PFrame after the IFrame has a timestamp 3000
after the PFrame before the IFrame. The timestamp sequence seems like the
IFrame didn't exist. This *ONLY* happens with this one video source that is
inserting SEI NALS. It is using buffer pool and picture timing SEI NALS.
The buffer pool SEIs are grouped with the IFrames, so I suspect that is
causing the timestamp error.

I am slogging through this timestamp stuff which is pretty obtuse,
especially with these SEIs. I believe that there is a bug here but probably
fixed in the 1.0 series (aren't they all?). If anyone could please shed any
insight on how these SEIs are supposed to contribute to timestamps I would
really appreciate it.

Chuck


On Sat, Jan 25, 2014 at 5:44 AM, Sebastian Dröge
<sebastian at centricular.com>wrote:

> On Fr, 2014-01-24 at 11:00 -0500, Chuck Crisler wrote:
> > I am using GStreamer 0.10.30 (sigh). I have an input source that is an
> RTSP
> > server that is using SEI messages. The stream is multiplexed into MP2T
> and
> > then converted back to RTP. The problem is that the SPS/PPS/SEI messages
> on
> > the output side have timestamps equal to the previous frame while on the
> > input side the timestamps equal the next frame. When the output RTP is
> sent
> > to VLC on Windows, it doesn't display well. It pauses, then plays several
> > seconds very quickly, then pauses.
> >
> > 2 Questions.
> >
> > 1. Is it likely that the SEI timestamp change is responsible for this
> issue?
> > 2. Should I try to shift the SEI timestamps to the next frame in
> rtph264pay?
> >
> > I don't have much experience with SEI messages. The RTP that I have
> worked
> > with in the past didn't use them.
>
> You might want to try if this is still like that with 1.0. In theory
> (after h264parse if it outputs with alignment=AU) you will get
> everything from the end of the previous frame until the end of the
> current frame in a single buffer and thus with the timestamp.
>
> That means that any SPS/PPS/SEI that comes in front of a frame will be
> bundled together with that frame in the same buffer with the same
> timestamp.
>
> Everything else would seem like a bug. Not sure how it works in
> alignment=NAL mode.
>
> --
> Sebastian Dröge, Centricular Ltd - http://www.centricular.com
> Expertise, Straight from the Source
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20140125/1fe2845f/attachment.html>