Video Low-latency plugin development

[Lijia (George Lee, Euler)]

It is always interest of topic drawing public eyeball to develop low-latency video system, even though GST provides all means and utility to inspect element's latency property as well as offer a good example to demonstrate design consideration for low-latency real-time use-case. But all these design spots is confine to GStreamer framework itself, for designing a well-suited plugin we also need to understand the system-wide technique that is very diversified and in nature the core points do not go beyond the following framework,

A. For either encoder or decoder, change processing-granularity from single frame into line level or slice level that is so called "sub-frame" video codec, in other words split a frame into a series of pieces then feed these pieces into pipeline, thus by means of intra-frame paralleling reduce overall latency

B. During bitstream encoding no B frame configured with I-frame and P-frame encoded into GOP

C. Remove any buffering between any consecutive pipeline processing stage so as to guarantee real-time bitstream pass-through with memory zero-copy implicited

D. Frame reorder/lipsync/FRC feature that are adopted under regular situation MUST de disabled for saving processing time

E. In response to addressing worst networking condition frame dropping sometimes performed and always process the most recent frame

Roll back to GST plugin development for low-latency application, how to design LL-friendly pipeline and what are compact element suits forming pipeline? I hope to get community's idea exchange and smart point inspiring me to reach to right destination.



George Lee



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170803/0f40ab01/attachment.html>