OpenGL on i.MX6 : bad performance

[Jean-Michel Hautbois]

Hi there,

I am using openGL elements from gstreamer in particular glimagesink
and I also tried glvideomixer.

Here is a simple pipeline example :
export GST_GL_PLATFORM=egl
export GST_GL_API=gles2
export DISPLAY=:0
gst-launch-1.0 -vvv videotestsrc !
'video/x-raw,width=640,height=480,format=RGBA' ! imagefreeze !
glvideomixer ! 'video/x-raw,framerate=(fraction)30/1,width=1920,height=1080'
! glimagesink

This works but the CPU usage is very high, and I did some profiling in
order to understand how it works.

Here is the result (CPU consumption is ~75% in top).

First, the global picture :
:~# opreport
Using /var/lib/oprofile/samples/ for samples directory.
CPU: CPU with timer interrupt, speed 996000 MHz (estimated)
Profiling through timer interrupt
          TIMER:0|
  samples|      %|
------------------
     4672 73.0000 vmlinux
      932 14.5625 libc-2.20.so
      375  5.8594 libGAL.so
      139  2.1719 libglib-2.0.so.0.4000.0
      116  1.8125 libpython2.7.so.1.0
       29  0.4531 libgstreamer-1.0.so.0.401.0
       21  0.3281 libpthread-2.20.so
       15  0.2344 libGLESv2.so.2.0.0
       15  0.2344 libX11.so.6.3.0
       13  0.2031 Xorg

Then, the kernel (Freescale 3.10.17) :
:~# opreport -l /boot/vmlinux
Using /var/lib/oprofile/samples/ for samples directory.
CPU: CPU with timer interrupt, speed 996000 MHz (estimated)
Profiling through timer interrupt
samples  %        symbol name
1596     34.1610  _HandleOuterCache
844      18.0651  v7_dma_flush_range
737      15.7748  cpuidle_enter_state
161       3.4461  get_page_from_freelist
139       2.9752  __memzero
120       2.5685  free_hot_cold_page

For the libc I don't have symbols, but using nm I can tell this is memcpy()

~# opreport -l /usr/lib/libGAL.so
Using /var/lib/oprofile/samples/ for samples directory.
CPU: CPU with timer interrupt, speed 996000 MHz (estimated)
Profiling through timer interrupt
samples  %        symbol name
293      80.2740  gcoHARDWARE_UploadTexture
6         1.6438  gcoSURF_QueryFormat
3         0.8219  _AllocateSurface
3         0.8219  gcoHARDWARE_GetPatchID

This seems ok, but if the pipeline is a little bit more complex, the
problems start.
Let's add a freezed image in the background, and this is not usuable.

gst-launch-1.0 -vv \
videotestsrc pattern=19 is-live=true !
'video/x-raw,width=1920,height=1080,format=RGBA' ! imagefreeze ! mix.
\
glvideomixer name=mix ! glimagesink \
videotestsrc is-live=true !
'video/x-raw,width=320,height=240,format=RGBA' ! imagefreeze ! mix.

CPU is ~91%
But now there are some messages like :
gst_base_sink_is_too_late ():
/GstPipeline:pipeline0/GstGLImageSink:glimagesink0:There may be a
timestamping problem, or this computer is too slow.

Here is what the profiler tells :
~# opreport
Using /var/lib/oprofile/samples/ for samples directory.
CPU: CPU with timer interrupt, speed 996000 MHz (estimated)
Profiling through timer interrupt
          TIMER:0|
  samples|      %|
------------------
     1338 49.0829 vmlinux
     1229 45.0844 libc-2.20.so
       83  3.0448 libglib-2.0.so.0.4000.0
       34  1.2472 libpython2.7.so.1.0
       10  0.3668 libGAL.so

And this would mean that uploading of texture is not the issue now...

These are static images, but in a real case, it could be a webcam, or
something else like a decoded video, how could these elements be more
efficient ? Maybe is it a pipeline problem, I may have forgotten some
things ?

Thanks in advance,
JM