How to get H264 & V4L2 pipeline work together?

[Tzu-Jung Lee]

On Tue, Apr 30, 2013 at 5:25 PM, Tim-Philipp Müller <t.i.m at zen.co.uk> wrote:
> On Tue, 2013-04-30 at 14:38 +0800, Tzu-Jung Lee wrote:
>
> Hi,
>>
>> I'm working on a device which can do mpeg2 to h264 transcoding.
>> I already have a preliminary V4L2 driver implementation, and can do
>> file to file transcoding with the v4l2-ctl utility.
>> And now I'd like to do the streaming over UDP with gstreamer.
>>
>> What are the best pipeline to do the streaming over UDP?
>> (I configure the device to do h264 to h264 transcoding, which takes
>> h264 ES feed, and output h264 ES)
>>
>> Current, I can get the transcode ES muxed and streamed over UDP with:
>>
>>     gst-launch-1.0 v4l2src ! h264parse ! mpegtsmux ! rndbuffersize
>> min=188 max=188 ! udpsink host=192.168.1.10 port=1234&
>
> I think MPEG-TS-over-UDP is usually streamed with 7 MPEG-TS packets per
> UDP packet (so 7*188 = 1316 bytes).

Yup, I tried it on PC first with min/max = 1316.
But on my board, it doesn't seem to work with size larger than 188.
So I currently work around it with this small size, and will get back
to find out what
happened once the whole pipeline work.

> Instead of wrapping it in MPEG-TS, you could also encapsulate it using
> RTP (rtph264pay ! udpsink + udpsrc ! rtpjitterbuffer ! rtph264depay)

One of the higher priority is to get the TS over UDP multicast work.
But I'll love to put RTP in the feature list :)

>> But I can't feed the bitstream to the device from udpsrc or filesrc yet.
>>
>>     gst-launch-1.0 filesrc location=test.264 ! h264parse ! v4l2sink
>> device=/dev/video0
>>
>> The following is the less verbose message. The full log is attached
>>
>> v4l2sink device=/dev/video0
>> Setting pipeline to PAUSED ...
>> Pipeline is PREROLLING ...
>> /GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps =
>> video/x-h264, width=(int)1920, height=(int)1088, parsed=(boolean)true,
>> stream-format=(string)avc, alignment=(string)au,
>> pixel-aspect-ratio=(frac0
>> ERROR: from element /GstPipeline:pipeline0/GstH264Parse:h264parse0:
>> GStreamer encountered a general stream error.
>> Additional debug info:
>> gstbaseparse.c(3038): gst_base_parse_loop ():
>> /GstPipeline:pipeline0/GstH264Parse:h264parse0:
>> streaming stopped, reason not-negotiated
>
> This looks like a bug in v4l2sink / the v4l2 plugin. It doesn't announce
> h264 caps properly, it should specify the stream-format supported (i.e.
> byte-stream). Since it doesn't specify anything, the parser outputs AVC
> (but V4L2_PIX_FMT_H264 == byte-stream if I'm not mistaken).
>
> Though that's not enough to explain why you're getting the
> not-negotiated error. Are you sure your output device supports this?

Chances are that I haven't got the driver right, but I'm not sure
which missing peace
failed gstreamer.

Here are the enumerated formats (only H264 is supported now):

    ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Output
        Pixel Format: 'H264' (compressed)
        Name        : H264 with start codes

        Index       : 1
        Type        : Video Output
        Pixel Format: 'MPG2' (compressed)
        Name        : MPEG-2 ES

        Index       : 2
        Type        : Video Output
        Pixel Format: 'MPEG' (compressed)
        Name        : MPEG-1/2/4 Multiplexed

All of them can be enumerated with frame size 1920x1080, 1080x720.

I haven't implemented interval enumeration. So I temporarily comment
out the following code, and
directly call gst_structure_set (s, "framerate",
GST_TYPE_FRACTION_RANGE, 0, 1, 100, 1, NULL);
Could it be a problem here?

------------------------------------
+++ b/sys/v4l2/gstv4l2object.c
@@ -881,7 +881,7 @@ static const GstV4L2FormatDesc gst_v4l2_formats[] = {
   {V4L2_PIX_FMT_MPEG, FALSE},

 #ifdef V4L2_PIX_FMT_H264
-  {V4L2_PIX_FMT_H264, TRUE},
+  {V4L2_PIX_FMT_H264, FALSE},
 #endif

   /*  Vendor-specific formats   */
@@ -1615,6 +1615,7 @@ gst_v4l2_object_probe_caps_for_format_and_size
(GstV4l2Object * v4l2object,

   /* keep in mind that v4l2 gives us frame intervals (durations); we invert the
    * fraction to get framerate */
+#if 0
   if (v4l2_ioctl (fd, VIDIOC_ENUM_FRAMEINTERVALS, &ival) < 0)
     goto enum_frameintervals_failed;

@@ -1755,6 +1756,7 @@ gst_v4l2_object_probe_caps_for_format_and_size
(GstV4l2Object * v4l2object,
   } else {
     goto unknown_type;
   }
+#endif

 return_data:
   s = gst_structure_copy (template);
@@ -1762,7 +1764,7 @@ return_data:
       "height", G_TYPE_INT, (gint) height,
       "interlace-mode", G_TYPE_STRING, (interlaced ? "mixed" : "progressive"),
       "pixel-aspect-ratio", GST_TYPE_FRACTION, 1, 1, NULL);
-
+#if 0
   if (G_IS_VALUE (&rates)) {
     /* only change the framerate on the template when we have a valid
probed new
      * value */
@@ -1772,6 +1774,8 @@ return_data:
     gst_structure_set (s, "framerate", GST_TYPE_FRACTION_RANGE, 0, 1, 100, 1,
         NULL);
   }
+#endif
+    gst_structure_set (s, "framerate", GST_TYPE_FRACTION_RANGE, 0, 1,
100, 1, NULL);
   return s;
------------------------------------

I'm still checking which V4L2 API is missing or failed for gstreamer
to work properly.
Here is an updated log which include v4l2sink:5, and disable colors.

Let me know if you spot anything strange.

Thanks.

> Cheers
>  -Tim
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fail.log
Type: application/octet-stream
Size: 27280 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130430/0c82cf08/attachment-0001.obj>