Reading and Modifying Buffers in a Pipeline

[Hassan Muhammad]

Thank you for your prompt response. It has been really helpful. I have
taken a look over the plugin writing tutorial that you have referenced and
it seems to me that this may be the most appropriate solution to modifying
buffers in a non-intrusive manner i.e. within an element between source and
sink pads. One thing I'd like to ask is, would mapping buffer read/writable
and performing some operation on the buffer (i.e. like you did converting
BGRx colorspace to GRAY8) impose any significant performance penalty on the
pipeline when specifically rendering HD or FHD videos?
Sorry if this seems like a silly question, I'm actually new to gstreamer
and only starting to get the hang of how it works. Thanks again for your
help.

On Tue, Nov 10, 2020 at 6:00 PM Sebastian Dröge <sebastian at centricular.com>
wrote:

> On Tue, 2020-11-10 at 12:24 +0500, Hassan Muhammad wrote:
>
> Hi. I am new to gstreamer and trying to figure out a way to read buffers
> from a video source, modifying the buffers in some way (e.g. draw overlay
> information) and output the buffers to a video or filesink. I am using the
> gstreamer-rust bindings and have not been able to find a solution to this
> problem. Reading on the forums, I tried to add a pad probe at the source
> element of the following pipeline as a test by getting the static pad:
>
> "videotestsrc ! autovideosink"
>
> I am able to read the buffer from "probe_info.data" :
>
> src_pad.add_probe(gst::PadProbeType::BUFFER, |_, probe_info| {
> ___READ BUFFER___
> }
>
> However, if I try to map the buffer as writable by using
> gst::buffer::BufferRef::map_writeable()
>
>
> it doesn't allow me to do so with the error "cannot borrow as mutable". At
> this point I am completely lost and unable to find an alternative to read
> and modify a buffer element through a pad probe. Any help or
> suggestion would be appreciated to help me overcome this.
>
>
> The following should work
>
>     src_pad.add_probe(gst::PadProbeType::BUFFER, |_, probe_info| {
>
>         if let Some(gst::PadProbeData::Buffer(ref mut buffer)) = probe_info.data {
>
>             let buffer = buffer.make_mut();
>
>
>             let map = buffer.map_writable().unwrap();
>
>             [do things with `map`]
>
>         }
>
>         gst::PadProbeReturn::Ok
>
>     });
>
>
> You need to get a mutable reference to the `gst::Buffer` from the
> `probe_info`, and then make sure that its content is actually writable and
> get a `&mut gst::BufferRef` to modify it via `make_mut()`.
> Then you can map it writable, change the PTS/DTS etc.
>
>
> However for modifying buffers (e.g. to add an overlay), it would be better
> to write a new element for that or make use of the `overlaycomposition`
> element.
>
> For the former you can find various examples in gst-plugins-rs, for the
> latter you can find an example here:
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/examples/src/bin/overlay-composition.rs
>
> --
>
> Sebastian Dröge, Centricular Ltd · https://www.centricular.com
> <http://www.centricular.com>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201112/d1914ce5/attachment-0001.htm>