<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">I have built a gstreamer pipeline to play 1280x720 h264 encoded video, making the stream available through the GStreamer RTSP server. My video playback is choppy on all players I have tried (VLC, Milestone, Avigilon, etc).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">VLC complains about pictures maybe being too late to be displayed and picture IS to late to be displayed, which is consistent with choppy rendering.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My pipeline is as follows: appsrc is-live=true name=h264appsrc blocksize=8192 ! h264parse ! rtph264pay name=pay0 pt=96 config-interval=5 perfect-rtptime=false<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My code putting data into the app src is scanning the input video stream and breaking it into NAL units. Each NAL unit is then put into a GSTBUFFER, and the PTS timestamp assigned to the GSTBUFFER is the timestamp of the previous GSTBUFFER
+ incremental time per frame rate. So for 5FPS, the timestamps would go 0, 200ms, 400ms, 600ms,800ms, etc. This algorithm gives us the ability to clearly organize the video data into frames to be played back at a given framerate.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I have the pipeline using gst_app_src_set_callbacks() so the I get a callback when the pipeline needs data. When I get this callback I feed a NAL unit residing in a GSTBUFFER to the app src with the appropriate frame timestatmp, and repeat
until I get the enough_data callback. The feeding of the GSTBUFFERS to the appsrc occurs in a thread that is enabled and disabled by the needs data and enough data callbacks, the GStreamer modules calling my callbacks are not blocked.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In my encoding, each NAL unit (I and P frames only) is a frame, emitted by my camera at the defined frame rate.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When I look at the flow of my thread pushing data to the appsrc until the enough data/need data cycle repeats, it is pretty much on the edge of feeding in enough GSTBUFFERS to meet the frame rate requirement.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When monitoring the packet flow in Wireshark, I see packets stop flowing from my stream to the VLC player for 200-500 milliseconds. VLC is griping about packets being late almost non-stop, and my video is choppy.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When I point VLC at a competitors IP camera streaming 1280x720 H264, VLC does not complain at all. Wireshark shows packet flow stopping for 10’s of milliseconds vs my 100’s of milliseconds.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I feel like something is inefficient in GStreamer’s rtp packetizing or in its ability to push packets onto the network at the desired timing. I have the same problem whether I run my camera at 10fps or 5fps.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am looking for some hints for how to determine where this bottleneck is. I need to determine if the problem is indeed in GStreamer or is in the network layer.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks for your help.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:black">Kerry Calvert<o:p></o:p></span></p>
<p class="MsoNormal"><i><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:black">Manager Software Engineering<o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:black">MicroPower Technologies Inc.<br>
Office: 858-914-5189</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:black">Cell: 619-823-2198<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif";color:black"><a href="mailto:kerry.calvert@micropower.com"><span style="color:blue">kerry.calvert@micropower.com</span></a><br>
<a href="http://www.micropower.com/"><span style="color:blue">www.micropower.com</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><img border="0" width="210" height="158" id="_x0039_B3D462E-C365-49DE-A753-60C9750451C8" src="cid:image003.png@01D07210.F7CE4CC0" alt="cid:4D0D7556-A446-40F0-B4A5-A3A400DF0565@Home"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>