[gst-devel] playbin2 issue with multiple formats

[Jeremy Ramer]

Hi,

I am working on a video decoder plugin and am running into issues when
I try to add multiple output formats.
Originally I had one output format on the source pad:

static GstStaticPadTemplate src_factory =
  GST_STATIC_PAD_TEMPLATE ("src",
    GST_PAD_SRC,
    GST_PAD_ALWAYS,
    GST_STATIC_CAPS (
        "image/jpeg, "
        "width = (int) [ 720, 1920 ], "
        "height = (int) [ 480, 1080 ], "
        "framerate = (fraction) [ 15, 30 ]"
));

I start the playback with
gst-launch -v playbin2 uri=file:////foo.mp4

and the decoder plays properly. GstJpegDec is automatically put into the chain.

Now I changed the source pad factory as follows:

static GstStaticPadTemplate src_factory =
  GST_STATIC_PAD_TEMPLATE ("src",
    GST_PAD_SRC,
    GST_PAD_ALWAYS,
    GST_STATIC_CAPS (
        "image/jpeg, "
        "width = (int) [ 720, 1920 ], "
        "height = (int) [ 480, 1080 ], "
        "framerate = (fraction) [ 15, 30 ]"
	";"
        "video/x-raw-yuv, "
        "format = (fourcc) NV12, "
        "width = (int) [ 720, 1920 ], "
        "height = (int) [ 480, 1080 ], "
        "framerate = (fraction) [ 15, 30 ]"
    ));

I've also updated the sink pad set_caps callback to select the output
and set the source caps

switch (f->output_format) {
  case OUTPUT_JPEG:
    caps = gst_caps_new_simple (
         "image/jpeg",
         "width", G_TYPE_INT, f->config.w,
         "height", G_TYPE_INT, f->config.h,
         "framerate", GST_TYPE_FRACTION, f->config.n, f->key.d,
         NULL);
    break;

  case OUTPUT_NV12:
    caps = gst_caps_new_simple (
         "video/x-raw-yuv",
         "format", GST_TYPE_FOURCC, GST_MAKE_FOURCC ('N', 'V', '1', '2'),
         "width", G_TYPE_INT, f->config.w,
         "height", G_TYPE_INT, f->config.h,
         "framerate", GST_TYPE_FRACTION, f->config.n, f->key.d,
         NULL);
    break;
  }

  gst_pad_set_caps(f->src.pad, caps);

When I start playback I get no video decoding and see the following error:

0:00:00.431639137 10222       0xb28c20 ERROR               playbin2
gstplaybin2.c:2522:pad_added_cb:<playbin20> unknown type image/jpeg
for pad uridecodebin0:src1


I'm guessing this has something to do with caps negotiation. I've
played with different setcaps and getcaps methods, but always get the
same result. Chapter 10 of the Plugin Writer's Guide has not really
cleared up the process of negotiation. Does anyone have any feedback
on the above code or know of a good example for doing multiple output
formats?

Thanks!
Jeremy