vaapijpegdec fails to decode Logitech C930e jpeg image

[chingmlai at rcn.com]

Hi,

    In our system, the video data is captured in jpeg/image format by using v4l2 library functions.  The pipeline uses elements of appsrc, vaapijpegdec, vaapipostproc, and appsink.

    The appsrc is set with src_caps of 
    src_caps = gst_caps_new_simple ("image/jpeg",   "width", G_TYPE_INT, 1280 "height", G_TYPE_INT, 720,
  		                               "framerate", GST_TYPE_FRACTION, 30, 1, NULL);
    The appsink is set with sink_caps of
    sink_caps = gst_caps_new_simple ("video/x-raw",  "width", G_TYPE_INT, 1280,  "height", G_TYPE_INT, 720,
                        "format", G_TYPE_STRING, "NV12",   "framerate", GST_TYPE_FRACTION, 30, 1, NULL);

    They are added and linked to the pipeline by
    gst_bin_add_many (GST_BIN(pipeline), appsrc, vaapijpegdec, vaapipostproc, appsink, NULL);
    gst_element_link_many(appsrc, vaapijpegdec, vaapipostproc, appsink, NULL);

    For each captured frame,

  Step 1. The jpeg data is pushed into appsrc by calling
  g_signal_emit_by_name (appsrc, "push-buffer", gst_buffer, &ret);
  where the gst_buffer contains data copied from the captured jpeg data, as well as the time stamp and duration.

  Step 2. The decoded data is pulled from appsink by calling
  g_signal_emit_by_name (appsink, "pull-sample", &gst_sample, &ret);
  where the returning gst_sample contains the decoded NV12 data.

    In our test, the program is running under Linux OS with appsrc and appsink as global Gstreamer elements.  We experienced that in Step 1, the function g_signal_emit_by_name (appsrc, "push-buffer", gst_buffer, &ret) always returns immediately.  However, in Step 2, g_signal_emit_by_name (appsink, "pull-sample", &gst_sample, &ret) does not return immediately.  For C930e webcam, it may never return after one or two frames.  We described this indefinite waiting status as stall in the pipeline.

    We do not use formal debug logging.  But we tested the process by comparing it with a software decoding function, which produces correct results for all webcams.  When the software decoder option is selected, the captured jpeg data is decoded into YUYV format by calling

    m_pJpegDecoder->jpeg_decode((uint8_t *)yuyv_data, (uint8_t *)jpeg_data, jpeg_size);
    where m_pJpegDecode is created and initialized by
    m_pJpegDecoder = new CJpegDecoder();
    m_pJpegDecoder->jpeg_init_decoder(1280, 720);

   The yuyv_data is then converted to NV12 format.  The source code of this  CjpegDecoder class is attached here and uploaded to website of gstreamer-vaapi open issue 134.


    For your information, I received message from 'gstreamer-devel' following my last email -

    “Your email to 'gstreamer-devel' with the subject vaapijpegdec fails to decode Logitech C930e jpeg image
Is being held until the list moderator can review it for approval.  The reason it is being held:
    Post by non-member to a members-only list.”

    Best regards,

Ching Lai

----- Original Message -----
From: "Víctor Jáquez" <vjaquez at igalia.com>
To: "Discussion of the development of and with GStreamer" <gstreamer-devel at lists.freedesktop.org>
Cc: chingmlai at rcn.com, jan at centricular.com
Sent: Tuesday, February 19, 2019 5:49:52 AM
Subject: Re: vaapijpegdec fails to decode Logitech C930e jpeg image

On Fri, 15 Feb 2019 at 11:15, chingmlai--- via gstreamer-devel wrote:
> 

[..]

> 
>    I did create this issue 6 days ago on
>    gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/134.  I got a
>    comment from Olivier Crête with message of “Can you attach a couple
>    pictures from the C930e that fails?”.  I generated couple of jpeg files
>    from C930e.  But my email sent back to him was rejected.  I am not sure he
>    has received my response and whether he will work on it or not.  I then
>    attached these two jpeg files to the open issue 134.  Here is my response
>    to Olivier, and I hope some of you can look into this issue.

Hi,

Would you please upload the images in the created issue?
gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/134

And also attach, in gitlab's issue, the a debug log to know why the pipeline
stalls. Also describe how is your decoding pipeline.

vmjl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: JpegDecoder.cpp
Type: text/x-c++src
Size: 31906 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190219/d832fd9b/attachment-0001.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: JpegDecoder.h
Type: text/x-chdr
Size: 8392 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190219/d832fd9b/attachment-0001.h>