How to send a series of zipped bitmaps files / raw video frames by tcp with gstreamer?
nicolas at ndufresne.ca
Tue Jun 16 17:43:04 UTC 2020
Le mardi 16 juin 2020 à 11:07 +0800, F32 a écrit :
> Here's more detailed background information:
> I have a 3D (OpenGL) rendering software running on server A, and a
> media server based on gstreamer running on server B.
> Server A is also the start point of the gstreamer pipeline. It
> fetches raw images from OpenGL, and puts images into gstreamer (e.g.,
> the tcpclientsink element)
> H264 / H265 is not used because it consumes a lot of cpu / gpu
> memory. However, sending raw images over network also consumes a lot
> of cpu / bandwidth, because raw images are just too large.
> A trade-off is to use a lightweight compression algorithm (e.g., lz4
> -1) which can compress 1GB bitmaps / raw images per second, per core.
> In my case, file size can be reduced by 90%.
> Too achieve this, I'm thinking about:
> - creating a lz4enc element, which puts "zipped content size" +
> "zipped content" into a gstbuffer, and then sends it by tcp
> - creating a lz4dec element, which keeps reading bytes from
> tcpserversrc, and outputs a decompressed gstbuffer when both "zipped
> content size" and "zipped content" are received
> Is it the correct way to achieve this?
Perhaps you'd like to seek inspiration in PNG (or just use that). It's
a simple libz based RGB tile compressor, but to support the P
(portable) they introduce a magic cookie and pixel layout / image size
information. With this design, the receiver can:
- Figure-out the image size and format (signalling free)
- Decompress as data flows (tiles) or on multiple cores
- Use low CPU (low complexity)
- It's truly loss-less
Its bad for lossy network and do not have RTP integration (though
GStreamer offer a GST specific generaic payloader). It also does not
have lossy mechanism to be able to guaranty a compression budget. Of
course, there is other solution to like old JPEG 2K, Apple ProRES, RED
encoding, NDI (warning, bad reputation ;-P), etc. They are all about
visually loss-less/low-complexity and fixed budget networks/storage
> (Also, I think that lightweight compression / decompression like lz4
> is an interesting feature that might be officially added to gstreamer
> in the long term)
Adding niche and unique compression format, rather then using widely
adopted standard is not that interesting for maintainers. But if you
host your own project with a library and documentation, and provide a
plugin using that, this would be acceptable for upstream GStreamer
inclusion. Hosting your own plugins as part of your project is of
course totally fine ;-P
Instead of your own project, you could add to FFMPEG CODEC collection,
which unlike GStreamer is meant to offer compression algorithm directly
(clean room implementation, patent encumbered is apparently fine
> At 2020-06-15 21:46:40, "Nicolas Dufresne" <nicolas at ndufresne.ca>
> >Le lundi 15 juin 2020 à 17:28 +0800, F32 a écrit :
> >> Hi,
> >> I'm trying to send a series of zipped bitmap files / raw frames
> >> network (because it consumes much less cpu than h264).
> >> The following command works and all files are copied correctly:
> >> > gst-launch-1.0 multifilesrc location="/dev/shm/1/%05d.bmp"
> >> ! multifilesink location="%05d.bmp"
> >> However, when I tried to transfer these files over tcp:
> >> > gst-launch-1.0 multifilesrc location="%05d.bmp" index=1 !
> >> tcpclientsink port=3000
> >> > gst-launch-1.0 tcpserversrc port=3000 ! multifilesink
> >> location="%05d.bmp"
> >> The result is incorrect. What I got is numerous 4KB files instead
> >> large bitmap files.
> >> I have learned that for raw video over tcp, we need to add a
> >> "rawvideoparse" element after tcpserversrc, but it does not seem
> >> support generic compression (zip, gzip, etc.)
> >> What is the simplest way to support transferring a series of
> >> bitmap files / raw video frames by tcp with gstreamer? Do I need
> >> extend the rawvideoaprse, write a special zipdec element, or
> >> something else?
> >The answer will depend on what you really intend to do. If generic
> >transfer is your end goal, this is simply the wrong software ;-P But
> >you do have some streaming use cases around that, we can likely
> >assuming we have more details. Of course, when you chose a streaming
> >method that is not following any existing standard, you might endup
> >having to implement some elements for it.
> >> Regards,
> >> Windy
> >> _______________________________________________
> >> gstreamer-devel mailing list
> >> gstreamer-devel at lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >gstreamer-devel mailing list
> >gstreamer-devel at lists.freedesktop.org
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
More information about the gstreamer-devel