Trying to understand gst_video_frame_map

[Ben Rush]

The following is more of an academic question, though I do have a practical
reason for wanting to understand a bit more about this function. Bottom
line is that I'm mostly exploring.

First, I'll admit that I'm not an expert in media formats (all the various
ways of encoding YUV, etc), most of the time I let the underlying library
handle that part for me (converting between color spaces, creating packed
vs. unpacked byte arrays for the frame data, etc). However, I'm trying to
write my own simple "videotestsrc" in order to understand the basics of
creating a GstPushSrc. I've got the thing written from the ground up (and
it works in a gstreamer pipeline which dumps the video to an MP4) with the
exception of the actual creation of the video frames. All I've got thus far
is a green video being written out.

So here's my question: say I've got a simple frame in YUV444 (
https://gstreamer.freedesktop.org/documentation/design/mediatype-video-raw.html
see
AYUV) and I want to send this down to the next element as a source. How do
I do it? The videotestsrc code is confusing with its creation of moving
patterns and use of some helper class called "paintinfo"; I feel like it's
making it hard for me to understand where I insert the data, and in
precisely what format. According to the documentation on the link above,
the format I'm playing with is YUV444

+-- + -- + -- + -- + +-- + -- + -- + -- +
| A0 | Y0 | U0 | V0| |A1 | Y1 | U1 | V1 | ...
+ -- + -- + -- + -- + +-- + -- + -- + -- +

In videotestsrc.c, line 1156 in
https://github.com/GStreamer/gst-plugins-base/blob/master/gst/videotestsrc/gstvideotestsrc.c
the
function "make_image" is surrounded by the gst_video_frame_map function.
And according to its documentation (
https://thiblahute.github.io/GStreamer-doc/gst-plugins-base-video-1.0/video-frame.html?gi-language=c#gst_video_frame_map
)
I should be able to call it, use the macro GST_VIDEO_FRAME_PLANE_DATA to
get a pointer to an underlying data buffer, and then write to it. However,
if that macro returns a variable called "pixels" as a gchar*, the following
code will cause exceptions (likely buffer overruns):

for (int y = 0, i=0; y < frame.info.height; y++)
{
for (int x = 0; x < frame.info.width; x++, i++)
{
pixels[i * 4] = 0;
pixels[i * 4 + 1] = Y;
pixels[i * 4 + 2] = U;
pixels[i * 4 + 3] = V;
}
}

So I feel like I'm not really accessing the data right. The videotestsrc
code uses some ORC library and appears to do something similar by creating
an int32 and writing it atop the buffer (see line 148 of
https://github.com/GStreamer/gst-plugins-base/blob/master/gst/videotestsrc/gstvideotestsrcorc-dist.c
 ).

Any input? I feel like I'm very, very close. I just need to figure out how
to packetize the data  into the GstVideoFrame structure, I feel.

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190422/e0be2630/attachment.html>