<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Lucida Console";
panose-1:2 11 6 9 4 5 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Sprechblasentext Zchn";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-US;}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.SprechblasentextZchn
{mso-style-name:"Sprechblasentext Zchn";
mso-style-priority:99;
mso-style-link:Sprechblasentext;
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">I try to get the vaapih264dec running for low latency streaming. I have good results if I use the avdec_h264 module but if I use the vaapih264dec the latency increases.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Pipeline (software):<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Lucida Console"">gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC ! rtpmp2tdepay ! tsdemux ! h264parse ! avdec_h264 ! glimagesink sync=false</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Lucida Console""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Pipeline (hardware):<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Lucida Console"">gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC ! rtpmp2tdepay ! tsdemux ! h264parse ! vaapih264dec ! glimagesink sync=false<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The encoder uses intra refresh and never sends a whole I-frame. So at the beginning vaapih264dec dropped all frames.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I modified gstvaapidecoder_h264.c:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">###################################################<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">priv->decoder_state |= sps_pi->state;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> if (!(priv->decoder_state & GST_H264_VIDEO_STATE_GOT_I_FRAME)) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">/* removed don't wait for a valid I-Frame (intra refresh problem)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> if (priv->decoder_state & GST_H264_VIDEO_STATE_GOT_P_SLICE)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> goto drop_frame;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">*/<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">###################################################<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">After that modification the stream started but the latency is still about 150ms higher as with avdec_h264. I guess this has something to do with frame reordering. Some decoders collect some frames and reorder them. In
this case the encoder does no reordering because for low latency but the decoder seem to collect the frames.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Does someone have an idea where the behavior can be changed e.g. reduce frame buffer to 0/1?<o:p></o:p></span></p>
</div>
</body>
</html>