Counting video frames and audio samples on GstBuffer

[Duzy Chan]

On Thu, Feb 21, 2013 at 1:21 AM, Ralph <ralph.gucwa at racelogic.co.uk> wrote:

> Have you considered using QoS messages on the pipeline bus?
>
> http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-qos.txt


QoS is using QOS upstream event to tell the status at the sync point. I'm
looking forward a more advanced method, which will enable me to inspect
into the pipeline status at a specific time. For example, at time 12345678,
what's the states of the pipeline before an encoding element, after an
mixing element, etc. Thus I'm going to be able to build a visualized graphs
to show the running pipeline's performance, and some thing else to help to
analyze the system (depending on what kind of variable can we know from the
system).

In order to archive this goal. I'm making a special element which I named
"assess" to help to inspect the pipelines real time. The aim of the
*assess*element is to inspect into pipelines (many pipelines) and
collect
*interesting information* about the pipeline. By using these collected
information, we're going to be able to build such a real time analyze graph.

For an example of the *assess* element, here we have:

    videotestsrc ! *assess* name=source ! XXXenc ! *assess* name=encoded !
.... ! *assess* name=assessX ! ...
    ! *assess* name=assessY ! .... ! *assess* name=sink ! fakesink

For this goal, I have no idea how QoS could achieve this.

And now I'm have the assess element working well. But I'm now have some
difficulties on deciding which is the *interesting information*, and how to
get those *interesting information*. And by now, the most significant
things to know would be like:

   - number of frames dropped at each *assess point*
   - time spent from one point to any downstream point of a specific frame.

I sincerely hope that some gurus could give me some tips on this. I'm now
using variables like

   - *ats*: (*assess time stamp*) a custom timestamp based on a global
   system clock for all *assess point*s
   - *pts*: presentation time stamp
   - *dts*: decode time stamp
   - *duration*: presentation time duration
   - *time*: total presentation time (sums all duration)
   - *buffer_count*: total buffer counts per *assess point*
   - *offset*: the buffer offset in the stream (the meaning varies basing
   on the encoding/type of stream, etc)

As what I can see from my current development is that:

   - the *pts* which is the timestamp of the buffer, could some times
   be GST_CLOCK_TIME_NONE, for example at the point after elements like
   tcpserversrc, some muexer
   - the *offset* some times could be -1, for example after the videomixer
   element.

And I can't find a variable to help me to *identify* a GstBuffer (or a
frame/sample). And my idea about *identification* of a GstBuffer is that,
every time when a buffer was created, it could have a "id", and the buffer
may be resized, encoded, or changed anyhow, and I'm think that the "id" is
always identical.


>

> --
> View this message in context:
> http://gstreamer-devel.966125.n4.nabble.com/Counting-video-frames-and-audio-samples-on-GstBuffer-tp4658661p4658672.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130224/8f873f72/attachment.html>