Thread safety in GStreamer plugins

[Alexander Hedges]

Mon, Mar 7, 2022 at 3:46 PM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
>
> Le lundi 07 mars 2022 à 13:23 +0100, Alexander Hedges via gstreamer-devel a
> écrit :
> > Hi,
> >
> > I'm currently quite confused about thread safety guarantees provided
> > by gstreamer.
> >
> > My mental model of threading in gstreamer is that gstreamer manages
> > its own thread pool
> > in which the elements are run. Each queue creates a thread boundary
> > that allows the user
> > to specify which elements are run on the same thread. The main loop
> > that does all the
> > scheduling is run synchronously when `g_main_loop_run()` is called.
>
> No, all the GMainLoop can optionally do is pull the messages from the bus,
> GStreamer does not have a dependency on GMainLoop.
>
> >
> > So far so good.
> >
> > In the gstreamer docs it says that "GStreamer is inherently
> > multi-threaded, and is fully
> > thread-safe."[1](https://gstreamer.freedesktop.org/documentation/application-development/advanced/threads.html?gi-language=c#threads)
> > which makes me think that it is fully supported to have two more
> > application threads that
> > both have references to an element and they can change properties
> > without having to do any
> > kind of locking to coordinate access to the element.
>
> Properties have flags (see gst-inspect-1.0). In theory, only flagged properties
> can be modified while the pipeline is running. The flags will indicate which
> states the property is valid to be modified. Arguably all the property change
> should have been protected with a lock, though over time, some non-protected
> property have been introduced and remain. If you hit any issue, file a bug for
> the specific property (or make an MR).

Thanks, I long wondered why some properties could be set at runtime
and some didn't have any effect. Now it makes sense.

>
> >
> > With that in mind, I now go ahead implementing a gstreamer plugin
> > based on some sample code
> > and looking at existing plugins.
> >
> > When implementing the `set_property` method of the class, I directly
> > set the instance variables
> > from the GValues that are passed in like so `object->integer_property
> > = g_value_get_int(value);`
> > without doing any synchronization. In my understanding, running the
> > above line doesn't write the
> > value atomically, so concurrent writes could lead to write tearing
> > such that in the end a different
> > value is set than requested by either calls. With the guarantees
> > mentioned above, I would assume
> > that to do this properly there would have to be some locking involved,
> > but looking at well-known
> > plugins like videotestsrc it only acquires its own lock for the the
> > `info` property which is a
> > GstVideoInfo.
> >
> > Given that there is not much gstreamer-specific about setting
> > properties of GObjects I wondered if
> > maybe GLib 2.0 provides some multithreading safety guarantees. Looking
> > at the GLib docs[2](https://docs.gtk.org/glib/threads.html),
> > they say that all functions concerning internally held state (stuff
> > like IO I guess) are thread safe,
> > but data structures are not. They also say that ref counting is thread safe.
> > I'm assuming a GObject is a data structure, so that would mean that
> > accessing a GObject via its methods
> > from multiple threads is not guaranteed to be thread safe. Also
> > looking at some code paths for
> > getting properties it doesn't seem like those are protected by locks
> > or scheduled on other threads.
> >
> > Which makes me wonder whether the client is not supposed to protect
> > against concurrent accesses.
> >
> > Looking at the thread safety design document
> > [3](https://gstreamer.freedesktop.org/documentation/additional/design/MT-refcounting.html?gi-language=c#properties),
> > it is also not super clear about the guarantees provided. It mentions that
> > Objects require the use of `GST_OBJECT_LOCK()` and
> > `GST_OBJECT_UNLOCK()`, but then again, this is
> > for the implementations, not the clients of the library, and in
> > existing plugins the object lock is
> > not acquired for every property.
> >
> > So I guess my question is twofold:
> >
> > 1. What multi-threading guarantees does gstreamer provide to users of gstreamer?
> >
> > 2. What do plugins need to do in order to make sure the gstreamer
> > guarantees are actually valid?
> >
> > Best regards,
> > Alexander Hedges
>