How to send a series of zipped bitmaps files / raw video frames by tcp with gstreamer?

[F32]

Here's more detailed background information:


I have a 3D (OpenGL) rendering software running on server A, and a media server based on gstreamer running on server B.


Server A is also the start point of the gstreamer pipeline. It fetches raw images from OpenGL, and puts images into gstreamer (e.g., the tcpclientsink element)



H264 / H265 is not used because it consumes a lot of cpu / gpu memory. However, sending raw images over network also consumes a lot of cpu / bandwidth, because raw images are just too large.


A trade-off is to use a lightweight compression algorithm (e.g., lz4 -1) which can compress 1GB bitmaps / raw images per second, per core. In my case, file size can be reduced by 90%.


Too achieve this, I'm thinking about:
 - creating a lz4enc element, which puts "zipped content size" + "zipped content" into a gstbuffer, and then sends it by tcp
 - creating a lz4dec element, which keeps reading bytes from tcpserversrc, and outputs a decompressed gstbuffer when both "zipped content size" and "zipped content" are received
Is it the correct way to achieve this?


(Also, I think that lightweight compression / decompression like lz4 is an interesting feature that might be officially added to gstreamer in the long term)


At 2020-06-15 21:46:40, "Nicolas Dufresne" <nicolas at ndufresne.ca> wrote:
>Le lundi 15 juin 2020 à 17:28 +0800, F32 a écrit :
>> Hi,
>> 
>> I'm trying to send a series of zipped bitmap files / raw frames over
>> network (because it consumes much less cpu than h264).
>> The following command works and all files are copied correctly:
>> 
>> > gst-launch-1.0 multifilesrc location="/dev/shm/1/%05d.bmp" index=1
>> ! multifilesink location="%05d.bmp"
>> 
>> However, when I tried to transfer these files over tcp:
>> 
>>  > gst-launch-1.0 multifilesrc location="%05d.bmp" index=1 !
>> tcpclientsink port=3000
>>  > gst-launch-1.0 tcpserversrc port=3000 ! multifilesink
>> location="%05d.bmp"
>> 
>> The result is incorrect. What I got is numerous 4KB files instead of
>> large bitmap files.
>> 
>> I have learned that for raw video over tcp, we need to add a
>> "rawvideoparse" element after tcpserversrc, but it does not seem to
>> support generic compression (zip, gzip, etc.)
>> 
>> What is the simplest way to support transferring a series of zipped
>> bitmap files / raw video frames by tcp with gstreamer? Do I need to
>> extend the rawvideoaprse, write a special zipdec element, or
>> something else?
>
>The answer will depend on what you really intend to do. If generic file
>transfer is your end goal, this is simply the wrong software ;-P But if
>you do have some streaming use cases around that, we can likely help,
>assuming we have more details. Of course, when you chose a streaming
>method that is not following any existing standard, you might endup
>having to implement some elements for it.
>
>> 
>> Regards,
>> Windy
>> 
>> 
>>  
>> 
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>_______________________________________________
>gstreamer-devel mailing list
>gstreamer-devel at lists.freedesktop.org
>https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200616/f774ebd7/attachment-0001.htm>