Trying to understand videorate - Slideshow with PNG images to be produced

Fri Jun 28 23:32:50 PDT 2013

Hi guys!

I'm trying to understand gstreamer 1.0 (1.0.7 linux to be more specific) 
concepts and I'm noob at this.

I have the following requirement.

1. Take a list of PNG images
2. Convert them to video at a desired FPS, let's say in range 20-30.
3. Get to display each image at a desired duration. It will be an 
slideshow where every image should be display during N secs (where N 
will be an integer and can differ between each image)

To simplify the first implementacion I decided to make N constant for 
every image.
Then I took the following route to tackle the problem but I finding a 
hard time understanding output and effects of videorate on this.

The context is:

* I have to images named: 01.png and 02.png
* I will try to read them using multifilesrc and then use my own custom 
plugin to timestamp the two buffers I get from multifilesrc to instruct 
when and for how long I want these images displayed.
* I decided that N will be 4 for now (4 secs).
* I decided to test using a FPS of just 2 for this attempt (but 
experimenting with other values -5, 10, 15, 20, 24, 30- give me similar 
not understandable results, at least, for me by now)
* My pipeline is:

$ gst-launch-1.0 --gst-debug=timedimagefilter2:6 multifilesrc 
location=%02d.png index=1 caps="image/png,framerate=\(fraction\)1/1" ! 
pngdec ! timedimagefilter ! videoconvert ! videorate ! 
video/x-raw,framerate=\(fraction\)2/1 ! timedimagefilter2 ! timeoverlay 
text="Stream time:" shaded-background=true ! autovideosink

I do the buffers timestamping in "timedimagefilter". this pluging was 
created using the make_element tool presented at the "Guide for Plugin 
Writers", and I only implemented the plugin_chain function that follows:

----

static GstFlowReturn
gst_timed_image_filter_chain (GstPad * pad, GstObject * parent, 
GstBuffer * buf)
{
GstTimedImageFilter *filter;
static int buffer_count = 0;
static GstClockTime last_ts = 0;
static GstClockTime freeze_const = 0;

filter = GST_TIMEDIMAGEFILTER (parent);

buffer_count++;

if (last_ts == 0) {
//last_ts = gst_util_uint64_scale_int(4, GST_SECOND, 1);
}
if (freeze_const == 0) {
freeze_const = gst_util_uint64_scale_int(4, GST_SECOND, 1);
}
GST_BUFFER_PTS(buf) = last_ts;
GST_BUFFER_DURATION(buf) = freeze_const;
last_ts += GST_BUFFER_DURATION(buf);

if (filter->silent == FALSE) {
GST_DEBUG("[%d] Buffer PTS %" G_GUINT64_FORMAT " - Buffer duration: %" 
G_GUINT64_FORMAT " - Offsets %d - %d", buffer_count, 
GST_BUFFER_PTS(buf), GST_BUFFER_DURATION(buf), GST_BUFFER_OFFSET(buf), 
GST_BUFFER_OFFSET_END(buf));
}

return gst_pad_push (filter->srcpad, buf);
}

----

Then I just created another plugin to simply print out what buffers it 
gets to better understand whats results the pipeline was producing. The 
plugin name is "timedimagefilter2". It was created following the same 
procedure as the first one and its chain functions follows:

----
static GstFlowReturn
gst_timed_image_filter2_chain (GstPad * pad, GstObject * parent, 
GstBuffer * buf)
{
GstTimedImageFilter2 *filter;
static int buffer_count = 0;
static GstClockTime last_ts = 0;
static GstClockTime freeze_const = 0;

filter = GST_TIMEDIMAGEFILTER2 (parent);

buffer_count++;

if (filter->silent == FALSE) {
GST_DEBUG ("[%d] Buffer PTS %" G_GUINT64_FORMAT " - Buffer duration: %" 
G_GUINT64_FORMAT " - Offsets %d - %d", buffer_count, 
GST_BUFFER_PTS(buf), GST_BUFFER_DURATION(buf), GST_BUFFER_OFFSET(buf), 
GST_BUFFER_OFFSET_END(buf));
}

return gst_pad_push (filter->srcpad, buf);
}

-----

Then if I run the pipeline (presented before) I think it should:

1. Read the PNG files and produce two buffers without timestamp nor 
duration info with a 1 FPS caps
2. Then, when timedimagefilter takes these two buffer, it should 
timestamp the first one with: pts = 0, duracion = 40000000000 (4 secs) 
and the second one with: pts = 4000000000, duracion = 40000000000 (4 secs)
3. videorate comes into play, I think it shoould proceduce a perfect 
stream at 2 FPS (with this pipeline), so it should come out with: (fps) 
* (duration secs) * (number of buffers after step 2) = 2 * 4 * 2 = 16 
buffers with a duration of 500000000 (0.5 secs) totaling 8 secs of 
video, that is my intention at least. Two images shown 4 secs each one.

Well, problem is, step 3 is producing 13 buffers totalling 6.5 secs.

Can you please tell me what I'm missing? I getting nuts at this point. 
Ouput of the pipeline run througout gst-launch follows:

gst-launch-1.0 --gst-debug=timedimagefilter2:6 multifilesrc 
location=%02d.png index=1 caps="image/png,framerate=\(fraction\)1/1" ! 
pngdec ! timedimagefilter ! videoconvert ! videorate ! 
video/x-raw,framerate=\(fraction\)2/1 ! timedimagefilter2 ! timeoverlay 
text="Stream time:" shaded-background=true ! autovideosink
Estableciendo el conducto a PAUSA …
El conducto está PREPARÁNDOSE …
0:00:00.129041407 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [1] Buffer PTS 
0 - Buffer duration: 500000000 - Offsets 0 - 1
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading 
configurations from ~/.fonts.conf is deprecated.
El conducto está PREPARADO …
Estableciendo el conducto a REPRODUCIENDO …
New clock: GstSystemClock
0:00:00.140751676 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [2] Buffer PTS 
500000000 - Buffer duration: 500000000 - Offsets 1 - 2
0:00:00.641115902 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [3] Buffer PTS 
1000000000 - Buffer duration: 500000000 - Offsets 2 - 3
0:00:01.141342501 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [4] Buffer PTS 
1500000000 - Buffer duration: 500000000 - Offsets 3 - 4
0:00:01.641005323 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [5] Buffer PTS 
2000000000 - Buffer duration: 500000000 - Offsets 4 - 5
0:00:02.141468509 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [6] Buffer PTS 
2500000000 - Buffer duration: 500000000 - Offsets 5 - 6
0:00:02.641065625 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [7] Buffer PTS 
3000000000 - Buffer duration: 500000000 - Offsets 6 - 7
0:00:03.141271028 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [8] Buffer PTS 
3500000000 - Buffer duration: 500000000 - Offsets 7 - 8
0:00:03.640990739 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [9] Buffer PTS 
4000000000 - Buffer duration: 500000000 - Offsets 8 - 9
0:00:04.141329374 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [10] Buffer 
PTS 4500000000 - Buffer duration: 500000000 - Offsets 9 - 10
0:00:04.641044586 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [11] Buffer 
PTS 5000000000 - Buffer duration: 500000000 - Offsets 10 - 11
0:00:05.141288924 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [12] Buffer 
PTS 5500000000 - Buffer duration: 500000000 - Offsets 11 - 12
0:00:05.641109001 32143 0x1a294a0 DEBUG timedimagefilter2 
gsttimedimagefilter2.c:242:gst_timed_image_filter2_chain: [13] Buffer 
PTS 6000000000 - Buffer duration: 500000000 - Offsets 12 - 13
Se recibió un EOS del elemento «pipeline0».
Execution ended after 6500440485 ns.
Estableciendo el conducto a PAUSA …
Estableciando el conducto a PREPARADO …
Estableciendo el conducto a NULL …
Liberando la tubería…

Thanks for your time. Any hints would be greatly appreciated.

Best regards,
Luis