<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Hi Nicolas,</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Note, I am not asking for a fix just
trying to get some help with gstreamer in general to try and see
if anyone else has seen a similar issue and to get some pointers
of where to look, as gstreamer/v4l2/NXP drivers/kernel is quite
complicated and I don't have much experience with the gstreamer
code base. I am digging into the code and will report back to NXP
and/or gstreamer if I find out what is causing this issue for me.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">As I'm getting nowhere with
investigation, I thought I would try the gstreamer1.0-plugins-imx
plugin as suggested to see how that fares. Unfortunately it didn't
build using the NXP hardknott release I was using so I have built
the latest NXP Yokto kirkstone LF5.15.52_2.1.0
for our board and have tried to build the gstreamer1.0-plugins-imx
but there are issues.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">I am using the NXP Yocto repo install
with all of the versions of things that its repo pulls in. As
suggested I have "disabled the NXP gstreamer fork entirely" using
then "PREFERRED_VERSION_gstreamer1.0*" settings in my local.conf.</div>
<div class="moz-cite-prefix">But when I build "bitbake
gstreamer1.0-plugins-imx" I get an error when it tries to build
libimxvpuapi2. It looks like this is looking for -lhantro_h1 which
does not exist.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Is there any info on how to build the
gstreamer1.0-plugins-imx over the NXP Yokto LF5.15.52_2.1.0 BSP ?<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">
<style type="text/css">
</style></div>
<div class="moz-cite-prefix">
<style type="text/css">td p { orphans: 0; widows: 0; background: transparent }p { line-height: 115%; margin-bottom: 0.25cm; background: transparent }a:link { color: #000080; so-language: zxx; text-decoration: underline }</style></div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Terry<br>
</div>
<div class="moz-cite-prefix">On 12/11/2022 01:34, Nicolas Dufresne
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAKQmDh-LE8fVACg_gFEHdgGhAWY5S7jLRQzF+kFa+DPic_j2Cw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="auto">
<div><br>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Le jeu. 10 nov. 2022, 03 h
17, Terry Barnaby via gstreamer-devel <<a
href="mailto:gstreamer-devel@lists.freedesktop.org"
moz-do-not-send="true" class="moz-txt-link-freetext">gstreamer-devel@lists.freedesktop.org</a>>
a écrit :<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
Nirbheek,<br>
<br>
Many thanks for the reply and info. When we started the
project quite a <br>
few of the IMX8MP hardware features were not supported by
the FSL <br>
Freescale code and so we have based our system on the NXP
BSP. I'm not <br>
sure that everything is supported now and I really don't
want to be <br>
mixing packages and getting into horrible package version
issues at this <br>
moment in time. However I might try that on a debug test
release to see <br>
if it affects things.<br>
<br>
We have to ship our first production systems in three
weeks and <br>
everything has been tested in our system and is working
well apart from <br>
this one issue and so we would like to try and debug and
fix this item <br>
for now and can look at moving the whole Linux platform
stack away from <br>
the NXP BSP for a later release.<br>
</blockquote>
</div>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">Gstreamer-imx is meant to run over the BSP. Its
a drop-in replacement usually. That being said, we do have
folks helping with gstreamer-imx in GStreamer community, but
the NXP plugins though are solely maintained by NXP. So for
continuing to use these and asking for a fix, please use NXP
support and/or user forum.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Nicolas</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Terry<br>
On 09/11/2022 18:50, Nirbheek Chauhan wrote:<br>
> Hi Terry,<br>
><br>
> You seem to be using the NXP gstreamer fork, and the
i.MX elements in<br>
> that aren't that great. The recommended elements are
in the Freescale<br>
> gstreamer-imx repository:<br>
><br>
> <a
href="https://github.com/Freescale/gstreamer-imx/"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/Freescale/gstreamer-imx/</a><br>
><br>
> The compositor there is called "imxg2dcompositor",
and there are<br>
> equivalents for all other elements too.
meta-freescale ships bitbake<br>
> files for it:<br>
><br>
> <a
href="https://github.com/Freescale/meta-freescale/blob/kirkstone/recipes-multimedia/gstreamer/gstreamer1.0-plugins-imx_2.1.0.bb"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/Freescale/meta-freescale/blob/kirkstone/recipes-multimedia/gstreamer/gstreamer1.0-plugins-imx_2.1.0.bb</a><br>
><br>
> You will need to put this in your config
(conf/local.conf or the OE<br>
> distro config) to disable the NXP gstreamer fork
entirely:<br>
><br>
> PREFERRED_VERSION_gstreamer1.0:mx6-nxp-bsp = ""<br>
>
PREFERRED_VERSION_gstreamer1.0-plugins-base:mx6-nxp-bsp =
""<br>
>
PREFERRED_VERSION_gstreamer1.0-plugins-good:mx6-nxp-bsp =
""<br>
>
PREFERRED_VERSION_gstreamer1.0-plugins-bad:mx6-nxp-bsp =
""<br>
> PREFERRED_VERSION_gstreamer1.0-libav:mx6-nxp-bsp = ""<br>
> PREFERRED_VERSION_gstreamer1.0:mx8-nxp-bsp = ""<br>
>
PREFERRED_VERSION_gstreamer1.0-plugins-base:mx8-nxp-bsp =
""<br>
>
PREFERRED_VERSION_gstreamer1.0-plugins-good:mx8-nxp-bsp =
""<br>
>
PREFERRED_VERSION_gstreamer1.0-plugins-bad:mx8-nxp-bsp =
""<br>
> PREFERRED_VERSION_gstreamer1.0-libav:mx8-nxp-bsp = ""<br>
> PREFERRED_VERSION_gstreamer1.0 = ""<br>
> PREFERRED_VERSION_gstreamer1.0-plugins-base = ""<br>
> PREFERRED_VERSION_gstreamer1.0-plugins-good = ""<br>
> PREFERRED_VERSION_gstreamer1.0-plugins-bad = ""<br>
> PREFERRED_VERSION_gstreamer1.0-libav = ""<br>
><br>
> If your OE-core layer has a very old gstreamer
version, you will also<br>
> need this layer: <a
href="https://github.com/OSSystems/meta-gstreamer1.0"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/OSSystems/meta-gstreamer1.0</a>
--<br>
> pick the right branch in that repo for your yocto
version.<br>
><br>
> Basically, if you see a gstreamer package version
with "imx" in it,<br>
> like x.y.z.imx, that's the NXP fork and you need to
fix up your<br>
> config.<br>
><br>
> Cheers,<br>
> Nirbheek<br>
><br>
> On Wed, Nov 9, 2022 at 8:16 PM Terry Barnaby via
gstreamer-devel<br>
> <<a
href="mailto:gstreamer-devel@lists.freedesktop.org"
target="_blank" rel="noreferrer" moz-do-not-send="true"
class="moz-txt-link-freetext">gstreamer-devel@lists.freedesktop.org</a>>
wrote:<br>
>> We are working on a video inspection system that
is using an NXP IMX8MP<br>
>> CPU based system that this using gstreamer for
its image processing<br>
>> pipeline. Generally the system is working fine
but occasionally when you<br>
>> pause and unpause the pipeline we get large video
stuttering (frames<br>
>> could take 20 seconds to update etc.).<br>
>><br>
>> This is using NXP gstreamer Elements as well as
NXP hardware blocks and<br>
>> also one of our own gstreamer elements. I know
this is not the standard<br>
>> gstreamer code, but I am just after some debug
advice as I have now<br>
>> spent a week trying to get to the bottom of the
issue and maybe someone<br>
>> might have some ideas/pointers.<br>
>><br>
>> I have got it down to a simple 'C' program
example that shows the issue<br>
>> after a few minutes. The basic pipeline for this
is:<br>
>><br>
>> imxcompositor_g2d name=c sink_0::alpha=1.0
sink_1::alpha=0.5 ! queue !<br>
>> waylandsink v4l2src device=/dev/video3 !
beamdeinterlace !<br>
>> video/x-raw,width=720,height=576,framerate=25/1 !
c.sink_0 appsrc<br>
>> name=appsrc ! c.sink_1<br>
>><br>
>> There is a 720x576@25 video stream from a CSI
camera and a 720x576@25<br>
>> overlay stream generated by an appsrc. The
beamdeinterlace Element is a<br>
>> very simple GstVideoFilterClass based Element
that just uses a<br>
>> transform_frame() function.<br>
>><br>
>> If this stream is paused and unpaused, sometimes
the camera video stream<br>
>> will have this severe stuttering while the
overlay stream is unaffected<br>
>> when looking at the video output from the
compositor.<br>
>><br>
>> Adding probes to the imxcompositor_g2d sinks show
that the buffer times<br>
>> on both sinks are updating fine. Indeed if I tee
of the stream after the<br>
>> beamdeinterlace this looks fine whilst the issue
is happening.<br>
>><br>
>> Digging down into the imxcompositor_g2d it is
using gstvideoaggregator<br>
>> and gstaggregator. When I add printf's to its<br>
>> gst_imxcompositor_aggregate_frames() function I
see that its<br>
>> gst_video_aggregator_pad_get_current_buffer()
call is returning the same<br>
>> frame continuously when the problem occurs. So it
looks like something<br>
>> in the GStreamer gstvideoaggregator or
gstaggregator is dropping the<br>
>> camera's video frames for some reason.<br>
>><br>
>> Setting GST_DEBUG=imxcompositor:7 shows no
obvious issues (no dropping<br>
>> frame like messages).<br>
>><br>
>> The GStreamer being used is 1.18.0 although I
suspect with a few NXP mods.<br>
>><br>
>> Has anyone seen this sort of behaviour with a
compositor before ?<br>
>><br>
>> Any ideas on what may be happening and where/how
to look ?<br>
>><br>
<br>
</blockquote>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>