Trying to understand videorate - Slideshow with PNG images to be produced
Luis Mariano Luporini
lluporini at gmail.com
Sat Jun 29 08:57:13 PDT 2013
Hi Tim!
I do really appreciate you taking to time to answer. Many thanks!
Sorry, some newbie stuff here as you can see.
Writing a new element to change my original buffers was the way I first
discovered reading through the docs. Maybe it's a better way to do it
but I'm not seeing it yet.
You say that in this case there's only a 0 timestamp at start and
nothing else. In my test case there was also another second buffer after
the first one. Are you telling me that it is some information missing
for the videorate plugin to work? Maybe I can complete it some way to
avoid the need to assume it if that's the case.
I changed my plugin code to use your suggestions below. My plugin_chain
function now look like this (please note I've changed fps_num from 5 to
2 to match my previous testcontext):
------
static GstFlowReturn
gst_timed_image_filter_chain (GstPad * pad, GstObject * parent,
GstBuffer * buf)
{
GstTimedImageFilter *filter;
static int frame_count = 0;
static GstClockTime ts, next_ts;
static GstClockTime duration;
int fps_num = 2;
int fps_denom = 1;
filter = GST_TIMEDIMAGEFILTER (parent);
frame_count++;
ts = gst_util_uint64_scale (frame_count, fps_denom * GST_SECOND, fps_num);
next_ts = gst_util_uint64_scale (frame_count + 1, fps_denom *
GST_SECOND, fps_num);
duration = next_ts - ts;
GST_BUFFER_PTS(buf) = ts;
GST_BUFFER_DURATION(buf) = duration;
return gst_pad_push (filter->srcpad, buf);
}
--------
Running the same original pipeline results in what follows:
gst-launch-1.0 --gst-debug=timedimagefilter2:6 multifilesrc
location=%02d.png index=1 caps="image/png,framerate=\(fraction\)1/1" !
pngdec ! timedimagefilter ! videoconvert ! videorate !
video/x-raw,framerate=\(fraction\)2/1 ! timedimagefilter2 ! timeoverlay
text="Stream time:" shaded-background=true ! autovideosink
Estableciendo el conducto a PAUSA …
El conducto está PREPARÁNDOSE …
0:00:00.209356439 1984 0x214f8a0 DEBUG timedimagefilter2
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [1] Buffer PTS
0 - Buffer duration: 500000000 - Offsets 0 - 1
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading
configurations from ~/.fonts.conf is deprecated.
El conducto está PREPARADO …
Estableciendo el conducto a REPRODUCIENDO …
New clock: GstSystemClock
0:00:00.264201501 1984 0x214f8a0 DEBUG timedimagefilter2
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [2] Buffer PTS
500000000 - Buffer duration: 500000000 - Offsets 1 - 2
0:00:00.736199990 1984 0x214f8a0 DEBUG timedimagefilter2
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [3] Buffer PTS
1000000000 - Buffer duration: 500000000 - Offsets 2 - 3
Se recibió un EOS del elemento «pipeline0».
Execution ended after 1500091958 ns.
Estableciendo el conducto a PAUSA …
Estableciando el conducto a PREPARADO …
Estableciendo el conducto a NULL …
Liberando la tubería…
--------
You can note that 3 buffers was produced:
Number 1: At timestamp 0 with 0.5s duration
Number 2: At timestamp 0.5 with 0.5s duration
Number 3: At timestamp 1 with 0.5s duration
Total duration = 1.5s
Then I changed the code again to start with a frame_count = 0 like this:
------
static GstFlowReturn
gst_timed_image_filter_chain (GstPad * pad, GstObject * parent,
GstBuffer * buf)
{
GstTimedImageFilter *filter;
static int frame_count = 0;
static GstClockTime ts, next_ts;
static GstClockTime duration;
int fps_num = 2;
int fps_denom = 1;
filter = GST_TIMEDIMAGEFILTER (parent);
//frame_count++; // <-------- CHANGED THIS AND PUT IT AFTER TS AND
DURATION ASSIGMENTS
ts = gst_util_uint64_scale (frame_count, fps_denom * GST_SECOND, fps_num);
next_ts = gst_util_uint64_scale (frame_count + 1, fps_denom *
GST_SECOND, fps_num);
duration = next_ts - ts;
GST_BUFFER_PTS(buf) = ts;
GST_BUFFER_DURATION(buf) = duration;
frame_count++; // <-------- THIS IS THE NEW LINE
return gst_pad_push (filter->srcpad, buf);
}
-------
And then pipe produced this output:
-----
gst-launch-1.0 --gst-debug=timedimagefilter2:6 multifilesrc
location=%02d.png index=1 caps="image/png,framerate=\(fraction\)1/1" !
pngdec ! timedimagefilter ! videoconvert ! videorate !
video/x-raw,framerate=\(fraction\)2/1 ! timedimagefilter2 ! timeoverlay
text="Stream time:" shaded-background=true ! autovideosink
Estableciendo el conducto a PAUSA …
El conducto está PREPARÁNDOSE …
0:00:00.236610397 2131 0x17500a0 DEBUG timedimagefilter2
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [1] Buffer PTS
0 - Buffer duration: 500000000 - Offsets 0 - 1
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading
configurations from ~/.fonts.conf is deprecated.
El conducto está PREPARADO …
Estableciendo el conducto a REPRODUCIENDO …
New clock: GstSystemClock
0:00:00.268205353 2131 0x17500a0 DEBUG timedimagefilter2
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [2] Buffer PTS
500000000 - Buffer duration: 500000000 - Offsets 1 - 2
Se recibió un EOS del elemento «pipeline0».
Execution ended after 1000379462 ns.
Estableciendo el conducto a PAUSA …
Estableciando el conducto a PREPARADO …
Estableciendo el conducto a NULL …
Liberando la tubería…
---------------
You can note that 2 buffers was produced:
Number 1: At timestamp 0 with 0.5s duration
Number 2: At timestamp 0.5 with 0.5s duration
Total duration = 1.0s
But I'm expecting the following behaviour:
1. I feed two frames to my plugin that produce the following (values in
secs):
Buffer 1: ts 0.0, duration 2.0
Buffer 2: ts 2.0, duration 2.0
2. Tell videorate I want this at 2 FPS so it can produce (values in secs):
Buffer 1: ts 0.0, duration 0.5 (dup of orignal buffer 1 contents)
Buffer 2: ts 0.5, duration 0.5 (dup of orginal buffer 1 contents)
Buffer 3: ts 1.0, duration 0.5 (dup of orginal buffer 1 contents)
Buffer 4: ts 1.5, duration 0.5 (dup of orginal buffer 1 contents)
Buffer 5: ts 2.0, duration 0.5 (dup of orginal buffer 2 contents)
Buffer 6: ts 2.5, duration 0.5 (dup of orginal buffer 2 contents)
Buffer 7: ts 3.0, duration 0.5 (dup of orginal buffer 2 contents)
Buffer 8: ts 3.5, duration 0.5 (dup of orginal buffer 2 contents)
Total time 4.0s that equals to 2 original PNG images to be shown during
2 secs each one at 2 FPS.
What I'm missing and not understanding?
Thanks about fakesink and identity tip I'll take a look at it. Noob
learning things here :)
Many thanks Tim!
Regards,
Luis
On 29/06/13 10:32, Tim-Philipp Müller wrote:
> On Sat, 2013-06-29 at 03:32 -0300, Luis Mariano Luporini wrote:
>
> Hi Luis,
>
>> I'm trying to understand gstreamer 1.0 (1.0.7 linux to be more specific)
>> concepts and I'm noob at this.
>>
>> I have the following requirement.
>>
>> 1. Take a list of PNG images
>> 2. Convert them to video at a desired FPS, let's say in range 20-30.
>> 3. Get to display each image at a desired duration. It will be an
>> slideshow where every image should be display during N secs (where N
>> will be an integer and can differ between each image)
> So first of all, you shouldn't really have to write your own element for
> this, but it seems there are some issues in videodecoder and videorate
> that don't handle this case properly where there's only a 0 timestamp at
> the start and then nothing else. Both should extrapolate based on the
> framerate set in the caps.
>
>
>> To simplify the first implementacion I decided to make N constant for
>> every image.
>> Then I took the following route to tackle the problem but I finding a
>> hard time understanding output and effects of videorate on this.
>>
>> The context is:
>>
>> * I have to images named: 01.png and 02.png
>> * I will try to read them using multifilesrc and then use my own custom
>> plugin to timestamp the two buffers I get from multifilesrc to instruct
>> when and for how long I want these images displayed.
>> * I decided that N will be 4 for now (4 secs).
>> * I decided to test using a FPS of just 2 for this attempt (but
>> experimenting with other values -5, 10, 15, 20, 24, 30- give me similar
>> not understandable results, at least, for me by now)
>> * My pipeline is:
>>
>> $ gst-launch-1.0 --gst-debug=timedimagefilter2:6 multifilesrc
>> location=%02d.png index=1 caps="image/png,framerate=\(fraction\)1/1" !
>> pngdec ! timedimagefilter ! videoconvert ! videorate !
>> video/x-raw,framerate=\(fraction\)2/1 ! timedimagefilter2 ! timeoverlay
>> text="Stream time:" shaded-background=true ! autovideosink
>>
>> I do the buffers timestamping in "timedimagefilter". this pluging was
>> created using the make_element tool presented at the "Guide for Plugin
>> Writers", and I only implemented the plugin_chain function that follows:
>>
>> ----
>>
>> static GstFlowReturn
>> gst_timed_image_filter_chain (GstPad * pad, GstObject * parent,
>> GstBuffer * buf)
>> {
>> GstTimedImageFilter *filter;
>> static int buffer_count = 0;
>> static GstClockTime last_ts = 0;
>> static GstClockTime freeze_const = 0;
>>
>> filter = GST_TIMEDIMAGEFILTER (parent);
>>
>> buffer_count++;
>>
>> if (last_ts == 0) {
>> //last_ts = gst_util_uint64_scale_int(4, GST_SECOND, 1);
>> }
>> if (freeze_const == 0) {
>> freeze_const = gst_util_uint64_scale_int(4, GST_SECOND, 1);
>> }
>> GST_BUFFER_PTS(buf) = last_ts;
>> GST_BUFFER_DURATION(buf) = freeze_const;
>> last_ts += GST_BUFFER_DURATION(buf);
>>
>> if (filter->silent == FALSE) {
>> GST_DEBUG("[%d] Buffer PTS %" G_GUINT64_FORMAT " - Buffer duration: %"
>> G_GUINT64_FORMAT " - Offsets %d - %d", buffer_count,
>> GST_BUFFER_PTS(buf), GST_BUFFER_DURATION(buf), GST_BUFFER_OFFSET(buf),
>> GST_BUFFER_OFFSET_END(buf));
>> }
>>
>> return gst_pad_push (filter->srcpad, buf);
>> }
>
> I think it should be something like:
>
> int fps_num = 5;
> int fps_denom = 1;
>
> ts = gst_util_uint64_scale (frame_count,
> fps_denom * GST_SECOND, fps_num);
>
> next_ts = gst_util_uint64_scale (frame_count + 1,
> fps_denom * GST_SECOND, fps_num);
>
> duration = next_ts - ts;
>
>
>> ----
>>
>> Then I just created another plugin to simply print out what buffers it
>> gets to better understand whats results the pipeline was producing. The
>> plugin name is "timedimagefilter2". It was created following the same
>> procedure as the first one and its chain functions follows:
> For what it's worth, you can also use fakesink silent=false and identity
> silent=false in combination with the -v option of gst-launch-1.0 to make
> it print the timestamps of the buffers passing through a pipeline.
>
> Cheers
> -Tim
>
> (Haven't really looked at the code or debugged your actual problem, just
> commenting from the sidelines)
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
More information about the gstreamer-devel
mailing list