Handling of QoS in a BaseTransform subclass
Jimmy De Pauw
jimmy at dcode.eu
Wed Jan 3 12:48:48 UTC 2018
I have been trying many many things but as soon as i use my own plugin in the pipeline the behavior remains the same.
QoS is not applied and all but the first buffer is flagged as DISCONT.
So if i do this :
gst-launch-1.0 filesrc location=... ! queue ! decodebin ! videoconvert ! jpegenc ! avimux ! filesink location=... sync=false|true
It indeed keep the processing in sync or not depending on the sink property i set.
I can also control buffering with the queue which works really well.
A custom plugin using python works as excpected.
But now i want to add my own plugin in C with processing.
gst-launch-1.0 filesrc location=... ! queue ! decodebin ! videoconvert ! processing_plugin ! jpegenc ! avimux ! filesink location=... sync=true|false
I also tried without a queue or adding "identity sync=true" after videoconvert but the behavior never change.
I am really at loss for what is happening but i am certain i am not doing something right in my plugin code.
For my plugin code here is the relevant parts, i did not include the processing code nor the setup of properties.
Transform_ip is the only method i override.
/* initialize the tfobjectdetection's class */
static void gst_tfobjectdetection_class_init (GsttfobjectdetectionClass * klass) {
GObjectClass *gobject_class;
GstElementClass *gstelement_class;
gobject_class = (GObjectClass *) klass;
gstelement_class = (GstElementClass *) klass;
g_type_class_add_private (klass, sizeof (GstTFObjectDetectionPrivate));
parent_class = g_type_class_peek_parent(klass);
gobject_class->set_property = gst_tfobjectdetection_set_property;
gobject_class->get_property = gst_tfobjectdetection_get_property;
/* Register properties used by tensorflow */
gst_element_class_add_pad_template (gstelement_class, gst_static_pad_template_get (&src_template));
gst_element_class_add_pad_template (gstelement_class, gst_static_pad_template_get (&sink_template));
GST_BASE_TRANSFORM_CLASS(klass)->transform_ip = GST_DEBUG_FUNCPTR(gst_tfobjectdetection_transform_ip);
/* debug category for filtering log messages
*/
GST_DEBUG_CATEGORY_INIT(gst_tfobjectdetection_debug, "gsttfobjectdetection", 0, "Tensorflow Debug");
}
/* initialize the new element
* initialize instance structure
*/
static void gst_tfobjectdetection_init (Gsttfobjectdetection *filter) {
GstTFObjectDetectionPrivate *priv;
GST_DEBUG("gst_tfobjectdetection_init");
/* Always enable qos */
gst_base_transform_set_qos_enabled(GST_BASE_TRANSFORM(filter), TRUE);
}
/* this function does the actual processing */
static GstFlowReturn gst_tfobjectdetection_transform_ip(GstBaseTransform * base, GstBuffer * outbuf) {
gpointer bytes;
gsize copied;
Gsttfobjectdetection *filter = GST_TFOBJECTDETECTION(base);
if (GST_CLOCK_TIME_IS_VALID (GST_BUFFER_TIMESTAMP(outbuf))) {
gst_object_sync_values (GST_OBJECT(filter), GST_BUFFER_TIMESTAMP(outbuf));
}
if (GST_BUFFER_IS_DISCONT(outbuf)) {
GST_WARNING("The buffer is DISCONT OMG!!!!");
}
... processing stuff ...
}
> On 20 Dec 2017, at 15:42, Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
>
>
>
> Le 20 déc. 2017 7:31 AM, "Jimmy De Pauw" <jimmy at dcode.eu <mailto:jimmy at dcode.eu>> a écrit :
> I am almost certain that BaseTransform ignore QoS because it does not have any segment timing to work with.
> Therefore i wonder if it is possible to have QoS when the main sink is filesrc with a test file.
>
> Considering the following pipeline :
>
> filesrc ! decodebin ! queue ! videoconvert ! <my plugin> ! jpegenc ! multifilesink
>
> In the debug log i have only one « gst_base_transform_chain ... we have a pending DISCONT» but never twice.
> I also never have a buffer dropped, looking at the code of GstBaseTransform i think this is because i don’t receive and segment information.
>
> Is it possible to simulate a live feed from a video file with full QoS support so i can test my processing speed?
>
> As soon as you have time segment and timestamp (e.g. after demuxer/parsing/decoding) you can add identity sync=1 to simulate a live feed.
>
>
>
>> On 18 Dec 2017, at 15:24, Nicolas Dufresne <nicolas at ndufresne.ca <mailto:nicolas at ndufresne.ca>> wrote:
>>
>>
>>
>> Le 18 déc. 2017 5:10 AM, "Jimmy De Pauw" <jimmy at dcode.eu <mailto:jimmy at dcode.eu>> a écrit :
>> Hello,
>>
>> I have been creating my own plugin using GstBaseTransform as the parent class.
>> It works find but now i want to handle the QoS in my tranform method to know if i have to drop the buffer or not.
>>
>> I can see that BaseTransform already listen to QoS event on its sink and set some properties in its priv structure.
>> I would like to have access to the earliest_time property from my own transform method but i don’t know how to do it, if it is possible at all.
>> It sounds like i have no access to it at all.
>>
>> Looking at the videoconvert source code i don’t see how it handles QoS eventl.
>>
>> It's entirely handled by the base class. But you can catch the event on your srcpad, there is virtual functions for that.
>>
>>
>> What is the proper way of using what is already there in the BaseTransform class to handle QoS on my own subclass?
>>
>> Thanks
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel <https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel <https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel <https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel <https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180103/f30eeaa2/attachment.html>
More information about the gstreamer-devel
mailing list