<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> Hi, I have made some python wrappers for easier building gstreamer applications. And in there I have added some debug/performance measure tools. I have done that using buffer probes. I think your problem is that the buffers you get from 'rtspsrc' are not necessarily frames (I guess depending on format, packet sizes, ...). So I would recommend you measure after 'decodebin', there you should get frames. This is how I measured "performance" using buffer probes on specific elements: <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div>def measure_elem_stats(self, elem: Gst.Element) -> None:</div><div> """Measure bitrate, fps and latency of gstreamer element"""</div><div> stats = GstPipeline.GstElementStats(elem)</div><div> add_buffer_probe(elem, "src", self._calc_stats, stats)</div><div> add_buffer_probe(elem, "sink", self._probe_in, stats)</div><div> </div><div> <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div>def add_buffer_probe(</div><div> elem: Gst.Element,</div><div> pad_name: str,</div><div> func,</div><div> data: Optional[object] = None,</div><div>) -> bool:</div><div> """Add a buffer probe to pad of element</div> <div> :param elem: element to probe</div><div> :param pad_name: name of the pad to probe, normally 'sink' or 'src'</div><div> :param func: callback function called when a buffer is received on the pad</div><div> :param data: optional data to pass to probe function, defaults to None</div><div> :return: True if the probe was sucessfully installed,</div><div> False otherwise.</div><div> """</div><div> if not elem:</div><div> return False</div><div> pad = elem.get_static_pad(pad_name)</div><div> if not pad:</div><div> # print(f"Couldn't find {elem.name}::{pad_name}")</div><div> return False</div> <div> pad.add_probe(Gst.PadProbeType.BUFFER, func, data)</div> <div> return True</div><div> </div><div> <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div>def _calc_stats(</div><div> self, pad: Gst.Pad, info: Gst.PadProbeInfo, stats: GstElementStats</div><div> ) -> Gst.PadProbeReturn:</div><div> """Callback of buffer probe for measuring bitrate, fps and latency"""</div><div> buf = info.get_buffer()</div><div> time = self.get_pipe_timestamp()</div><div> diff = time - buf.pts # total latency from src</div><div> elapsed = time - stats.perf_sink.prev_time</div> <div> if not is_clock_valid(stats.perf_sink.prev_time):</div><div> stats.perf_sink.prev_time = time</div><div> elif is_clock_valid(time) and elapsed > Gst.SECOND:</div><div> # frames per second</div><div> factor_n = elapsed / Gst.MSECOND</div><div> factor_d = Gst.SECOND / Gst.MSECOND</div> <div> # bytes per second</div><div> bps = stats.perf_sink.bps * factor_d / factor_n</div><div> stats.bitrate = bps / (1024 * 1024 / 8) # Mbps</div> <div> stats.fps = stats.perf_sink.frame_count * factor_d / factor_n</div> <div> latency = stats.perf_sink.latency / stats.perf_sink.frame_count</div><div> stats.elem_latency = latency - stats.src_latency</div> <div> self.__logger.debug(</div><div> f"{stats.elem_name}: "</div><div> f"bitrate={stats.bitrate} Mbps "</div><div> f"fps={stats.fps} "</div><div> f"latency elem={timestamp_msecs(stats.elem_latency)} ms "</div><div> f"total={timestamp_msecs(latency)} ms"</div><div> )</div><div> self._plot_stats(time, stats)</div> <div> stats.perf_sink.reset(time)</div> <div> stats.perf_sink.frame_count += 1</div><div> stats.perf_sink.bps += buf.get_size()</div><div> stats.perf_sink.latency += diff</div> <div> return Gst.PadProbeReturn.OK</div><div> </div><div> <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div> def _probe_in(</div><div> self, pad: Gst.Pad, info: Gst.PadProbeInfo, stats: GstElementStats</div><div> ) -> Gst.PadProbeReturn:</div><div> """Callback of buffer probe for measuring processing time"""</div><div> buf = info.get_buffer()</div><div> time = self.get_pipe_timestamp()</div><div> diff = time - buf.pts # latency from src</div><div> elapsed = time - stats.perf_src.prev_time</div> <div> if not is_clock_valid(stats.perf_src.prev_time):</div><div> stats.perf_src.prev_time = time</div><div> elif is_clock_valid(time) and elapsed > Gst.SECOND:</div><div> stats.src_latency = stats.perf_src.latency / stats.perf_src.frame_count</div> <div> # self.__logger.debug(</div><div> # f"{stats.elem_name}: src_latency={latency / Gst.MSECOND} ms"</div><div> # )</div> <div> stats.perf_src.reset(time)</div> <div> stats.perf_src.frame_count += 1</div><div> stats.perf_src.latency += diff</div> <div> return Gst.PadProbeReturn.OK</div><div> </div><div> <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div>class GstElementStats:</div><div> """Statistic values for Gst.Element</div> <div> :param elem: gstreamer element to measure statistics on</div><div> """</div> <div> class PerfData:</div><div> """Used for measuring bitrate, fps and latencies in buffer probes"""</div> <div> def __init__(self):</div><div> """Constructor"""</div><div> self.prev_time = Gst.CLOCK_TIME_NONE</div><div> self.frame_count = 0</div><div> self.bps = 0</div><div> self.latency = 0</div> <div> def reset(self, time: int) -> None:</div><div> """Reset values"""</div><div> self.frame_count = 0</div><div> self.bps = 0</div><div> self.latency = 0</div><div> self.prev_time = time</div> <div> def __init__(self, elem: Gst.Element):</div><div> """Constructor"""</div><div> self.bitrate = 0 # in Mbps</div><div> self.fps = 0</div><div> self.src_latency = 0 # in ms</div><div> self.elem_latency = 0 # in ms</div><div> self.elem_name = elem.name</div><div> self.perf_src = GstPipeline.GstElementStats.PerfData()</div><div> self.perf_sink = GstPipeline.GstElementStats.PerfData()</div></div></div></div></div></div></div></div></div></div> Otherwise it is also possible to use GST_TRACERS (either the standard gstreamer ones or possible also the ones from gst-shark from ridgeRun) and get the values inside your application using a custom log function: <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div>Gstreamer Tracers can be enabled by setting them through the correct<div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div> environment variable (`GST_TRACERS`).</div><div> See <a class="moz-txt-link-freetext" href="https://gstreamer.freedesktop.org/documentation/additional/design/">https://gstreamer.freedesktop.org/documentation/additional/design/</a></div><div> tracing.html?gi-language=python#latency</div> <div> Latency tracer:</div> <div> GST_TRACERS="latency(flags=pipeline+element+reported)" python3 pipeline.py</div> <div> CPU usage tracer:</div><div> </div><div> GST_TRACERS="stats;rusage" python3 pipeline.py</div></div></div><div> </div><div> <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div> # tracer: must be set before 'Gst.init()': no runtime control</div><div> # Example tracers</div><div> LATENCY_TRACER = "latency(flags=pipeline+element+reported)"</div><div> CPU_USAGE_TRACER = "stats;rusage"</div><div> DESIRED_TRACERS = None</div><div> if not os.getenv("GST_TRACERS") and DESIRED_TRACERS:</div><div> os.environ["GST_TRACERS"] = DESIRED_TRACERS</div></div></div><div> </div><div> </div><div>def _on_debug(</div><div> self,</div><div> category: Gst.DebugCategory,</div><div> level: Gst.DebugLevel,</div><div> file: str,</div><div> function: str,</div><div> line: int,</div><div> source: GObject.Object,</div><div> message: Gst.DebugMessage,</div><div> user_data: Optional[object] = None,</div><div> ):</div><div> """Custom gstreamer log function"""</div><div> level_name = Gst.DebugLevel.get_name(level)</div><div> text = f"Gstreamer {level_name} {category.get_name()} "</div><div> if source:</div><div> file_name = os.path.basename(file)</div><div> text += f"{file_name}:{line}:{function}<{source.name}>"</div><div> text += f": {message.get()}"</div> <div> # currently ignoring gst debug log level below DEBUG</div><div> if level == Gst.DebugLevel.ERROR:</div><div> self.__logger.error(text)</div><div> elif level == Gst.DebugLevel.WARNING:</div><div> self.__logger.warning(text)</div><div> elif level == Gst.DebugLevel.INFO:</div><div> self.__logger.info(text)</div><div> elif level == Gst.DebugLevel.DEBUG:</div><div> self.__logger.debug(text)</div><div> # handle GST_TRACERS</div><div> elif level == Gst.DebugLevel.TRACE:</div><div> # self.__logger.debug(text)</div><div> self._handle_tracer_output(category, message)</div><div> </div><div> <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div>def _handle_tracer_output(</div><div> self, category: Gst.DebugCategory, message: Gst.DebugMessage</div><div> ) -> None:</div><div> """Handle the output generated by Gstreamer Tracers</div> <div> Detect the type of tracer and generate the relevant internal Event</div><div> """</div><div> if category.get_name() == "GST_TRACER":</div><div> tracer = self._parse_trace_msg(message.get())</div> <div> # 'Gst.Structure.from_string' fails for some types of tracer messages</div><div> if tracer == "message" or tracer == "query":</div><div> # self.__logger.debug(f"{category.get_name()}: {message.get()}")</div><div> return</div> <div> try:</div><div> s, _ = Gst.Structure.from_string(message.get())</div><div> # tracer = s.get_name()</div><div> except Exception:</div><div> self.__logger.warning("Failed parsing structure")</div><div> self.__logger.debug(f"{category.get_name()}: {message.get()}")</div><div> return</div> <div> if tracer == "element-latency":</div><div> type = GstPipeline.GstEvent.GstTracerEventType.ELEM_LATENCY</div><div> self._plot_elem_latency(s)</div><div> elif tracer == "latency":</div><div> type = GstPipeline.GstEvent.GstTracerEventType.LATENCY</div><div> elif tracer == "thread-rusage":</div><div> # self._plot_rusage(s)</div><div> type = GstPipeline.GstEvent.GstTracerEventType.THREAD_RUSAGE</div><div> elif tracer == "proc-rusage":</div><div> # self._plot_rusage(s)</div><div> type = GstPipeline.GstEvent.GstTracerEventType.PROC_RUSAGE</div><div> else:</div><div> type = GstPipeline.GstEvent.GstTracerEventType.TRACER</div><div> self.queue_event(</div><div> GstPipeline.GstEvent(</div><div> type,</div><div> f"{s.to_string()}",</div><div> )</div><div> )</div><div> </div><div> <div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace;font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div>def _config_gst_log(self) -> None:</div><div> """Configure custom gstreamer logging function</div> <div> If 'GST_DEBUG' is set though the command line the custom 'on_debug()'</div><div> will be ignored and the default gstreamer logging will be used instead.</div> <div> Also enable the gstreamer tracing system (when required).</div><div> """</div><div> if not Gst.debug_is_active():</div><div> Gst.debug_remove_log_function(None)</div><div> Gst.debug_add_log_function(self._on_debug, None)</div><div> level = Gst.debug_get_default_threshold()</div><div> if level < Gst.DebugLevel.ERROR:</div><div> Gst.debug_set_default_threshold(Gst.DebugLevel.WARNING)</div><div> # enable tracer</div><div> if self.DESIRED_TRACERS:</div><div> Gst.debug_set_threshold_for_name("GST_TRACER", Gst.DebugLevel.TRACE)</div><div> Gst.debug_set_active(True)</div></div></div></div></div></div> Best Regards Marianna <div class="moz-cite-prefix">On 17.03.2023 05.04, <a class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel-request@lists.freedesktop.org">gstreamer-devel-request@lists.freedesktop.org</a> wrote: </div> <blockquote type="cite" cite="mid:mailman.8825.1679025872.929.gstreamer-devel@lists.freedesktop.org"> <pre class="moz-quote-pre" wrap="">hello all, i am attempting to measure FPS of a source stream (e.g. rtsp or hls). I am currently using an identity element immediately after the source element (e.g. rtspsrc -> identity -> decodebin -> fakesink). Using the handoff signal on identity, I am able to get a good calculation of buffers per second. However, it seems that this value is not always 1:1 with frames per second. It very frequently is on stable streams (e.g. I often see 29-31 buffers/sec on a 30fps stream). However, on streams with intermittent connectivity, it doesn't seem to tell the whole story. I've seen fpsdisplaysink which actually works really well (I am able to see "real" fps values that are dynamically changing as the pipeline plays). However, a) I don't want to measure the fps at the sink (i have a tee'd pipeline with multiple sinks and more interesting in measuring source fps than that of my pipeline) and b) this element seems to just overlay it ? ideally i would be able to get the average "recent" fps as a property. I've seen ridgerun's gst-perf element, but that only seems to log values. Is there a recommended approach to monitoring "real" FPS of an incoming live stream? Ideally using identity, but open to other options. Thanks, Matt</pre> </blockquote> <pre class="moz-signature" cols="72">-- Best regards / Med venlig hilsen “Marianna Smidth Buschle”</pre> </body> </html>