Yes, I have h264parse before rtph264pay. Yes, rtph264pay can/will merge FUs. The overall process is this: iPad/H264/RTP -> gstreamer RTP to multicast MP2T ->gstreamer mutlticast MP2T to RTP -> iPad/RTP Unfortunately, the MP2T portion is absolutely required due to legacy systems. :-((( I can't see the internal buffer data, but h264parse is finding a start code around 900 bytes into the incoming buffers. The original RTP stream had about 2200 bytes of payload data (over about 3 packets), so I know that this start code is artificial. The next buffer into h264parse has a first_mb_in_slice of around 200+. However, at that point h264parse has already pushed the previous buffer downstream (to rtph264pay). rtph264pay determines that the buffer it receives is complete (how would it know it isn't?) so it processes it as a complete packet that fits within the MTU and signals it as such. The only clue that there is more data for the frame is that the first_mb_in_slice on later buffers is greater than 0. I think that I need h264parse to accumulate multiple input buffers before pushing anything to rtph264pay until the slice number is 0. When it does, the buffer should be around 2k bytes in my current testing. Here is a brief log snippet to illustrate the problem. I have confirmed this with packet captures. h264parse gsth264parse.c:2569:gst_h264_parse_chain:<H264Parse>[00m received buffer of size 720 h264parse gsth264parse.c:1964:gst_h264_parse_chain_forward:[00m Next NAL Unit Pos: -1 <== I added this msg, it shows that there isn't a start code yet gsth264parse.c:2014:gst_h264_parse_chain_forward:<H264Parse>[00m NAL type: 5, ref_idc: 3 gsth264parse.c:2033:gst_h264_parse_chain_forward:<H264Parse>[00m first MB in slice: 0, slice type: 7 <== first part of IDR gsth264parse.c:2055:gst_h264_parse_chain_forward:<H264Parse>[00m we have an I slice h264parse gsth264parse.c:2569:gst_h264_parse_chain:<H264Parse>[00m received buffer of size 170 <== received another buffer segment h264parse gsth264parse.c:1964:gst_h264_parse_chain_forward:[00m Next NAL Unit Pos: 890 <== second segment had a start code - don't know where from but guess MP2T demux, still not the end of the H264 frame though h264parse gsth264parse.c:2014:gst_h264_parse_chain_forward:<H264Parse>[00m NAL type: 5, ref_idc: 3 h264parse gsth264parse.c:2033:gst_h264_parse_chain_forward:<H264Parse>[00m first MB in slice: 0, slice type: 7 <== starts back at the beginning, I will have to deal with this h264parse gsth264parse.c:2055:gst_h264_parse_chain_forward:<H264Parse>[00m we have an I slice h264parse gsth264parse.c:2223:gst_h264_parse_chain_forward:<H264Parse>[00m pushing buffer 0xf6f02de8, size 890, ts 0:00:00.066733333 <== NO! We really aren't done. Trust me! h264parse gsth264parse.c:1964:gst_h264_parse_chain_forward:[00m Next NAL Unit Pos: -1 h264parse gsth264parse.c:2014:gst_h264_parse_chain_forward:<H264Parse>[00m NAL type: 5, ref_idc: 3 h264parse gsth264parse.c:2033:gst_h264_parse_chain_forward:<H264Parse>[00m first MB in slice: 228, slice type: 7 <== I told you that we weren't finished! At this time rtph264pay has already sent the original buffer out as a completed RTP packet with the marker bit set. This causes the decoder (using iDoubs on an iPad) to fail. Gstreamer can handle this condition properly. However, this is effectively an error condition, even though the spec uses 'should' concerning the marker bit rather than 'shall'. Everyone relies on the marker bit. But that doesn't matter. The required ending (display on an iPad) doesn't work (yet). Chuck <div class="gmail_quote">On Thu, Mar 28, 2013 at 3:08 PM, Olivier Crête <<a href="mailto:olivier.crete@collabora.com" target="_blank">olivier.crete@collabora.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Thu, 2013-03-28 at 14:54 -0400, Chuck Crisler wrote: > Actually, as I have been working on this over time I have come to > realize that the current gstreamer (or any RTP code) cannot possibly > work. There is important information in the first RTP stream that is > lost in the conversion to MP2T. Specifically the fact that there are > multiple RTP packets required for the frame which is signaled via the > FU-A construct. However, that seems to be OK for the MP2T. When the > conversion back to RTP is made, it is only processing those MP2T > packets that were created from each single RTP packet in the original > stream. I don't think that there is any way in the normal scheme to > correct that. I think that to correct it I will need to modify > h264parse (my first element in the conversion back to RTP) to collect > input buffers until the first_mb_in_slice == zero. Then the previous > data can be processed and pushed out as a single buffer for rtph264pay > to process. I have seen other places in the code of both elements that > make that same assumption, so it seems safe to me. </div>I'm not sure I understand exactly what you mean, but rtph264depay will merge back FU-A and FU-B packets into a single GStreamer buffer (or drop them completely if any part is missing), if you control the encoding, you are strongly advised to make the NAL unit smaller than the MTU. I think you want to collect all of the buffers in an access unit (where first_mb_in_slice == zero), then rtph264depay can do that for you if the caps afterwards having "alignment=au". You should never need to have h264parse after rtph264depay. <div class="HOEnZb"><div class="h5"> -- Olivier Crête <a href="mailto:olivier.crete@collabora.com">olivier.crete@collabora.com</a> _______________________________________________ gstreamer-devel mailing list <a href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a> <a href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel" target="_blank">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a> </div></div></blockquote></div>