decreasing video playback start time

[Bryan Noll]

GStreamer version is 1.9.2 (downloaded and extracted via the universal
binaries).

I'm playing a 640x480 MJPEG stream being made available over http on a
local wireless network. The stream I'm consuming and playing on the Android
device is also being produced by GStreamer. I've been told that the mjpeg
stream is 30 frames per second, if that's meaningful at all.

The hardware I'm trying to play it on is primarily an Android Samsung
Tablet with model number SM-T550 (I assume you're asking about the device
I'm trying to play it on not the device where it's being produced).

> Is it using the hardware codecs?

GStreamer is very new to me, so forgive my ignorance, but the best answer I
can give you is basically I don't know. The code we've got is heavily
inspired by the tutorial-5 you've made available (thx for that btw). So
right now, it's just a playbin pipeline. Here's what my slightly modified
`gst_native_play` function looks like. The log line "GST_STATE is done
waiting for get state..." is consistently written ~5 seconds after the line
"GST_STATE: RESULT RETRIEVED" is written. It's at this instant that I can
see the video playback begin.

static void gst_native_play (JNIEnv* env, jobject thiz) {
  CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
  if (!data) return;
  GST_DEBUG ("Setting state to PLAYING");
  data->target_state = GST_STATE_PLAYING;
  GstStateChangeReturn result = gst_element_set_state (data->pipeline,
GST_STATE_PLAYING);
    GST_DEBUG ("GST_STATE: RESULT RETRIEVED %d", result);

    switch(result) {
      case GST_STATE_CHANGE_FAILURE :
          GST_DEBUG ("GST_STATE_CHANGE_FAILURE");
          break;
      case GST_STATE_CHANGE_SUCCESS:
          GST_DEBUG ("GST_STATE_CHANGE_SUCCESS");
          break;
      case GST_STATE_CHANGE_ASYNC:
          GST_DEBUG ("GST_STATE_CHANGE_ASYNC");
          GST_DEBUG ("GST_STATE is async, going to wait for it");
          GstState state;
          GstState pending;
          gst_element_get_state (data->pipeline,
                                 &state,
                                 &pending,
                                 GST_CLOCK_TIME_NONE);
          GST_DEBUG("GST_STATE is done waiting for get state...");
          break;
      case GST_STATE_CHANGE_NO_PREROLL:
          GST_DEBUG ("GST_STATE_CHANGE_NO_PREROLL");
          break;
      default:
          GST_DEBUG ("GST_STATE_default");

    }
  data->is_live = TRUE;
}

I'm not using a customized pipeline of any kind yet, but was wondering if
this was the direction I needed to start going. In terms of hardware
accelerated codecs, if you're talking about the kind of thing documented
here
http://docs.gstreamer.com/display/GstSDK/Playback+tutorial+8%3A+Hardware-accelerated+video+decoding
(not sure if that documentation is even legit at this point) then I can
definitely say that I am not using `gst_plugin_feature_set_rank` to set the
rank of particular plugins. If I'm not even answering your question
appropriately, I'm happy to check if you could let me know how.

The delay I'm interested in reducing is from the moment I call
`gst_element_set_state (data->pipeline, GST_STATE_PLAYING);` to the moment
that the underlying state actually changes to GST_STATE_PLAYING (which
correlates with the video visually playing on the screen).

The playback once the video starts is performing well, with very low
latency. As another data point, if we navigate to the same MJPEG url in
Chrome on the same Android device, the playback begins basically instantly
(sub 1 second).

I'm sure the difference is something I'm doing incorrectly or inefficiently
and not GStreamer itself. Again, appreciate any help.

--Bryan


On Fri, Sep 30, 2016 at 2:10 AM, Sebastian Dröge <sebastian at centricular.com>
wrote:

> On Thu, 2016-09-29 at 15:23 -0500, Bryan Noll wrote:
> > Hello folks. I'm building an Android app that needs to play mjpeg
> > video. So far I've been able to successfully get the video playing on
> > the Android device following the tutorial here https://cgit.freedeskt
> > op.org/~slomo/gst-sdk-tutorials/tree/gst-sdk/tutorials/android-
> > tutorial-5/ . That said, right now I'm seeing it take roughly 5
> > seconds to get the video playback started. To be very specific, in
> > the `gst_native_play` function in that tutorial there is a call to
> > `gst_element_set_state`. I've modified the code, splitting the lines
> > apart so I capture the GstStateChangeReturn value being returned from
> > that call. This result is a GST_STATE_CHANGE_ASYNC. Then, I call
> > `gst_element_get_state` on the pipeline (which blocks until the
> > actual result is available). By the time that function call returns,
> > roughly 5 seconds have gone by since we originally called
> > `gst_element_set_state`. The specific line that I'm talking about
> > (the one I broke apart into separate lines) is this one:
> > https://cgit.freedesktop.org/~slomo/gst-sdk-tutorials/tree/gst-
> > sdk/tutorials/android-tutorial-5/jni/tutorial-5.c#n457
> >
> > More details about the stream. It's a live mjpeg stream playing off
> > of a local network (not public internet). I'd like to decrease this
> > 5s figure as much as I can so that video playback begins sooner. Can
> > anyone point me in the right direction on how I'd go about doing
> > that?
>
> Which GStreamer version are you using, and what kind of file or stream
> are you trying to play? Also on which hardware is this, and is it using
> the hardware codecs?
>
> --
> Sebastian Dröge, Centricular Ltd · http://www.centricular.com
> _______________________________________________
> gstreamer-android mailing list
> gstreamer-android at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-android
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-android/attachments/20160930/12d6bcf2/attachment.html>