Bad performance of H264 stream playback on android

[diexsie]

Hello together,

i face the problem that a h264 video stream works fine on my LG G4, but very
poor on my HTC u11 and Samsung Galaxy S7.

I used tutorial-3 from the android examples and modified it to show a H264
rtp stream. I used the following pipeline on android side:
data->pipeline = gst_parse_launch("udpsrc port=5000 ! application/x-rtp,
media=video, clock-rate=90000, encoding-name=H264 ! decodebin !
autovideosink", &error);

For sending i use the following command on a Windows 7 machine:
gst-launch-1.0 -v ksvideosrc do-timestamp=true ! videoconvert ! x264enc
tune=zerolatency !
"video/x-h264,profile=baseline,width=1280,height=720,framerate=30/1,bitrate=300"
! rtph264pay config-interval=1 ! udpsink host=192.168.8.106 port=5000
sync=false

The only difference is where the android application is running. It's the
same network setup etc.

I realized that the amc hardware decoders are not used in the case of HTC
u11 and Galaxy S7. It fails with some message
"amcvideodec-omxqcomvideodecoderavc0:sink> pad peer query failed". Instead
avdec_h264 was suggested by decodebin. To force gstreamer using amc hardware
decoders I modified the decoder ranks(1) and set some environment variable
(2) suppressing "unknown color stuff".

(1)
static void enable_factory (const gchar *name, gboolean enable) {
    GstRegistry *registry = NULL;
    GstElementFactory *factory = NULL;
    GstRegistry* reg = gst_registry_get();
    if (!reg)
    {
        GST_DEBUG("No registry found for %s", name);
        return;
    }
    GstPluginFeature* feat = gst_registry_lookup_feature(reg, name);
    if(feat == NULL) {
        GST_DEBUG("No feat found for %s", name);
        return;
    }
    if (enable) {
        GST_DEBUG("RAISE rank %s", name);
        gst_plugin_feature_set_rank(feat, GST_RANK_PRIMARY + 1);
    }
    else {
        GST_DEBUG("Disable plugin %s", name);
        gst_plugin_feature_set_rank(feat, GST_RANK_NONE);
    }
    gst_object_unref(feat);
    return;
}

(2):
 static {

        try {
            Os.setenv("GST_AMC_IGNORE_UNKNOWN_COLOR_FORMATS", "yes", true);
            //Os.setenv("GST_DEBUG", "5", true);
            Os.setenv("GST_DEBUG_DUMP_DOT_DIR",
"/data/data/org.freedesktop.gstreamer.tutorials.tutorial_3/files", true);
        } catch (ErrnoException e) {
            e.printStackTrace();
        }

        System.loadLibrary("gstreamer_android");
        System.loadLibrary("tutorial-3");
        nativeClassInit();
    } 

In the end I could achieve that HTC and Samsung are building up the same
pipeline like the LG (only with decoder ranks modification and env variable.
pipeline itself is still determined by decodebin). I verified this by
comparing both pipelines by generating the corresponding visual
representations:

HTC:
<http://gstreamer-devel.966125.n4.nabble.com/file/t378782/dotfile_htcu11_playing.png> 

LG:
<http://gstreamer-devel.966125.n4.nabble.com/file/t378782/dotfileplaying_lgg4.png> 

Nevertheless the HTC and Samsung show very poor performance. I can observe a
delay of 1-2 seconds and a lots of artifacts during playback and shuttering.
Playback is not smooth at all.

I am using GStreamer 1.15.1. First I though it is due to the reason that the
Samsung is based on an Exynos SOC and some amcviddec-omxexynosavcdec
decoder, while the LG G4 is based on Snapdragon using some
amcviddec-omxqcomvideodecoderavc. Then I bought a HTC u11, which also has  a
Snapdragon like the G4 and it's decoder is named the same way
amcviddec-omxqcomvideodecoderavc. Nevertheless there were errors like:

W/VideoCapabilities: Unrecognized profile 2130706434 for video/avc
E/GStreamer+amc: 0:00:00.100972812 0x7fa1fdb000
../sys/androidmedia/gstamc.c:2069:accepted_color_formats Unknown color
format 0x7fa30c06, ignoring
W/GStreamer+amc: 0:00:00.100993906 0x7fa1fdb000
../sys/androidmedia/gstamc.c:1732:scan_codecs video/avc decoder has unknown
color formats, only direct rendering will be supported

I checked with Android Media API that the HTC supports exactly the same
color formats than the LG G4. Except that the HTC is supporting some more
color formats, which are not known to GStreamer. But with setting this env
variable I could overcome this issue. 

I can attach logs of the tutorial 3 android app without modification of my
rank and env variable. I thought that the error might be more obvious there,
because gstreamer decides in this case on its own to prefer software
decoding:
https://www.dropbox.com/s/daq4x13zyhi2vtt/htc_u11full.txt?dl=0

Thank you already very much in advance. As you might see I tried already a
lot, but sadly without having success...








--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/