[gst-devel] Latency regression on v4l2src ! x264enc ! rtp using most recent x264enc

[Mallard]

Semi newbie question here, I hope this is the correct place to post.

I am having trouble getting the most recent gstreamer H264 encoder to  
perform as well as the 0.10.12 release. I have been working with  
Ubuntu Lucid, and until a few days ago, I could capture and display  
locally from a V4L2 source (a Hauppauge WinTV HVR 950Q USB stick) with  
a pipeline I include below, and send a stream via RTP to another  
machine on my LAN. Latency as measured by comparing the image shown on  
the sending machine's xvimagesink tee to the receiving machine's  
display of the decoded image was about .25 seconds (that is, a quarter  
of a second - very low.) After I upgraded the latency has gone to  
about 2.5 seconds, ten times higher.

I then built a new machine with identical hardware to the first  
capture machine and installed Ubuntu Karmic AMD64 (these are all 64  
bit desktop installs) and was able to stream to the (same) receiving  
machine using the same pipeline with the expected quarter-second  
latency.

The Karmic install has these versions:

gstreamer0.10-x :: 0.10.25-2ubuntu1.2
gstreamer0.10-tools :: 0.10.25-2
gstreamer0.10-plugins-ugly :: 0.10.12-1
gstreamer0.10-plugins-ugly-multiverse :: 0.10.12-0ubuntu1
libx264-67 :: 1:0.svn20090621+git364d7d-0ubuntu2


This setup, using exactly the same WinTV950Q to capture and the same  
machine to receive
that I have been testing on gives very low latency - basically .25  
sec. If I reduce latency in
the receive script to 80ms (it jitters and gives me frame-drop error  
messages if I go
below that, like to 20ms)

If I go back to the Lucid machine, it has these versions:

gstreamer0.10-x :: 0.10.28-1
gstreamer0.10-tools :: 0.10.28-1
gstreamer0.10-plugins-ugly :: 0.10.14-1
gstreamer0.10-plugins-ugly-multiverse :: 0.10.14libx264-0ubuntu1
libx264-85 :: 2:0.85.1448+git1a6d32-2

Streaming from the Lucid machine I have almost 2.5 seconds of latency  
( about an order of magnitude more ) using exactly the same hardware  
capture device, the same gst-launch pipeline on capture and on  
receive.  However I do not get jitter. Also, the earlier gstreamer has  
a caps parameter that is
missing on the newer gstreamer -

  profile-level-id=(string)64001e

The receiver works with and without this parameter in the caps line.

The sender is running this:

------------------------
DEST=192.168.0.30
VOFFSET=0
AOFFSET=0

# H264 encode from the source
VELEM="v4l2src device=/dev/video0 always-copy=false"
VCAPS="video/x-raw-yuv, width=720,  
height=480,format=(fourcc)UYVY,framerate=(fraction)30000/1001"
VSOURCE="$VELEM ! $VCAPS ! queue ! tee name=t1 ! queue ! xvimagesink  
sync=false t1. ! queue ! videorate ! ffmpegcolorspace"
VENC="x264enc byte-stream=true bitrate=1500 dct8x8=true subme=6  
cabac=true threads=0 ! rtph264pay "

VRTPSINK="udpsink port=5000 host=$DEST ts-offset=$VOFFSET name=vrtpsink"
VRTCPSINK="udpsink port=5001 host=$DEST sync=false async=false name=vrtcpsink"
VRTCPSRC="udpsrc port=5005 name=vrtpsrc"
AELEM="autoaudiosrc"

ASOURCE="$AELEM ! queue ! audioresample ! audioconvert"
AENC="alawenc ! rtppcmapay"

ARTPSINK="udpsink port=5002 host=$DEST ts-offset=$AOFFSET name=artpsink"
ARTCPSINK="udpsink port=5003 host=$DEST sync=false async=false name=artcpsink"
ARTCPSRC="udpsrc port=5007 name=artpsrc"

gst-launch -v gstrtpbin name=rtpbin \
         $VSOURCE !  $VENC ! rtpbin.send_rtp_sink_0                     
                          \
         rtpbin.send_rtp_src_0 ! $VRTPSINK                              
                     \
         rtpbin.send_rtcp_src_0 ! $VRTCPSINK                            
                     \
       $VRTCPSRC ! rtpbin.recv_rtcp_sink_0                              
                     \
     $ASOURCE ! $AENC ! rtpbin.send_rtp_sink_1                          
                     \
         rtpbin.send_rtp_src_1 ! $ARTPSINK                              
                     \
         rtpbin.send_rtcp_src_1 ! $ARTCPSINK                            
                     \
       $ARTCPSRC ! rtpbin.recv_rtcp_sink_1
------------------------




The receiver is running this: (the caps line varies, of course, each  
time I start the sender, I replace the caps line in the receiver with  
the proper caps)

------------------------
DEST=192.168.0.162
LATENCY=100
VIDEO_CAPS="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264,sprop-parameter-sets=\"Z2QAHqwkqAtD2wFQgAAB9IAAdTAHixdQ,aO8yyLA=\",ssrc=(guint)204906444,payload=(int)96,clock-base=(guint)3249570813,seqnum-base=(guint)5734"
AUDIO_CAPS="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMA"
OUTSINK="xvimagesink"

gst-launch -v gstrtpbin name=rtpbin latency=$LATENCY                    
                 \
      udpsrc caps=$VIDEO_CAPS   port=5000 ! rtpbin.recv_rtp_sink_0      
                   \
        rtpbin. ! rtph264depay ! ffdec_h264 ! $OUTSINK  \
      udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                        
                 \
          rtpbin.send_rtcp_src_0 ! udpsink port=5005 host=$DEST  
sync=false async=false \
      udpsrc caps=$AUDIO_CAPS port=5002 ! rtpbin.recv_rtp_sink_1        
                 \
        rtpbin. ! rtppcmadepay ! alawdec ! audioconvert !  
audioresample ! autoaudiosink     \
      udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                        
                 \
          rtpbin.send_rtcp_src_1 ! udpsink port=5007 host=$DEST  
sync=false async=false

------------------------

There is no audio being sent, or rather, there is nothing connected to  
the audio input.

I want to use the newer releases, of course, but I don't see why th  
latency has increased, using exactly the same hardware and pipelines.

Any help or commentary is greatly appreciated!