Reading and Modifying Buffers in a Pipeline

[Sebastian Dröge]

On Tue, 2020-11-10 at 12:24 +0500, Hassan Muhammad wrote:
> Hi. I am new to gstreamer and trying to figure out a way to read
> buffers from a video source, modifying the buffers in some way (e.g.
> draw overlay information) and output the buffers to a video or
> filesink. I am using the gstreamer-rust bindings and have not been
> able to find a solution to this problem. Reading on the forums, I
> tried to add a pad probe at the source element of the following
> pipeline as a test by getting the static pad: 
> 
> "videotestsrc ! autovideosink"
> 
> I am able to read the buffer from "probe_info.data" :
> 
> src_pad.add_probe(gst::PadProbeType::BUFFER, |_, probe_info| {
> ___READ BUFFER___
> }
> 
> However, if I try to map the buffer as writable by using
> gst::buffer::BufferRef::map_writeable()
> 
> 
> it doesn't allow me to do so with the error "cannot borrow as
> mutable". At this point I am completely lost and unable to find an
> alternative to read and modify a buffer element through a pad probe.
> Any help or suggestion would be appreciated to help me overcome this.

The following should work

    src_pad.add_probe(gst::PadProbeType::BUFFER, |_, probe_info| {     
  if let Some(gst::PadProbeData::Buffer(ref mut buffer)) =
probe_info.data {            let buffer = buffer.make_mut();
            let map = buffer.map_writable().unwrap();            [do
things with `map`]         }        gst::PadProbeReturn::Ok    });
You need to get a mutable reference to the `gst::Buffer` from the
`probe_info`, and then make sure that its content is actually writable
and get a `&mut gst::BufferRef` to modify it via `make_mut()`.
Then you can map it writable, change the PTS/DTS etc.


However for modifying buffers (e.g. to add an overlay), it would be
better to write a new element for that or make use of the
`overlaycomposition` element.

For the former you can find various examples in gst-plugins-rs, for the
latter you can find an example here:
    
https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/blob/master/examples/src/bin/overlay-composition.rs


-- 
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201110/daef0ba9/attachment.htm>