Trying to understand gst_video_frame_map

[Ben Rush]

I believe I have figured out my issue. There is a "pack_func" that is
attached to the GstVideoFrame's GstVideoInfo structure that all the bytes
must pass through in order to be sent down the chain. Upon executing this
function against my YUV data, I was able to get it to save a well-formatted
and good-looking MP4 file.



On Mon, Apr 22, 2019 at 4:54 PM Ben Rush <ben at ben-rush.net> wrote:

> The following is more of an academic question, though I do have a
> practical reason for wanting to understand a bit more about this function.
> Bottom line is that I'm mostly exploring.
>
> First, I'll admit that I'm not an expert in media formats (all the various
> ways of encoding YUV, etc), most of the time I let the underlying library
> handle that part for me (converting between color spaces, creating packed
> vs. unpacked byte arrays for the frame data, etc). However, I'm trying to
> write my own simple "videotestsrc" in order to understand the basics of
> creating a GstPushSrc. I've got the thing written from the ground up (and
> it works in a gstreamer pipeline which dumps the video to an MP4) with the
> exception of the actual creation of the video frames. All I've got thus far
> is a green video being written out.
>
> So here's my question: say I've got a simple frame in YUV444 (
> https://gstreamer.freedesktop.org/documentation/design/mediatype-video-raw.html see
> AYUV) and I want to send this down to the next element as a source. How do
> I do it? The videotestsrc code is confusing with its creation of moving
> patterns and use of some helper class called "paintinfo"; I feel like it's
> making it hard for me to understand where I insert the data, and in
> precisely what format. According to the documentation on the link above,
> the format I'm playing with is YUV444
>
> +-- + -- + -- + -- + +-- + -- + -- + -- +
> | A0 | Y0 | U0 | V0| |A1 | Y1 | U1 | V1 | ...
> + -- + -- + -- + -- + +-- + -- + -- + -- +
>
> In videotestsrc.c, line 1156 in
> https://github.com/GStreamer/gst-plugins-base/blob/master/gst/videotestsrc/gstvideotestsrc.c the
> function "make_image" is surrounded by the gst_video_frame_map function.
> And according to its documentation (
> https://thiblahute.github.io/GStreamer-doc/gst-plugins-base-video-1.0/video-frame.html?gi-language=c#gst_video_frame_map )
> I should be able to call it, use the macro GST_VIDEO_FRAME_PLANE_DATA to
> get a pointer to an underlying data buffer, and then write to it. However,
> if that macro returns a variable called "pixels" as a gchar*, the following
> code will cause exceptions (likely buffer overruns):
>
> for (int y = 0, i=0; y < frame.info.height; y++)
> {
> for (int x = 0; x < frame.info.width; x++, i++)
> {
> pixels[i * 4] = 0;
> pixels[i * 4 + 1] = Y;
> pixels[i * 4 + 2] = U;
> pixels[i * 4 + 3] = V;
> }
> }
>
> So I feel like I'm not really accessing the data right. The videotestsrc
> code uses some ORC library and appears to do something similar by creating
> an int32 and writing it atop the buffer (see line 148 of
> https://github.com/GStreamer/gst-plugins-base/blob/master/gst/videotestsrc/gstvideotestsrcorc-dist.c
>  ).
>
> Any input? I feel like I'm very, very close. I just need to figure out how
> to packetize the data  into the GstVideoFrame structure, I feel.
>
> Thanks in advance.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190423/d220628e/attachment-0001.html>