Records, mix and sync several RTP audio and video streams

[pablo platt]

Hi,

I have a RTP MCU that broadcasts audio and video streams from several users
in a conference.
Users turn on and off their mics and cameras randomly during the conference.

I'm looking for a file format to record the streams without processing and
a way to later mix audio and video using gstreamer's audiomixer and
compositor elements.
Multiple videos will be displayed on a grid or a single video will be
chosen based on the active speaker.

The file formats I've found are:
1. pcap - I'll might be able to save all streams in a single file and demux
with the udp port. The pcap frame timestamp might be useful. Some of the
pcap frame data might be redundant or unneeded.
2. rfc4571 (rtpstreampay/rtpstreamdepay) - the format is simple and
efficient. I'll probably need a separate file per stream/session. Might be
able to demux based on SSRC.
3. gdp - seems like gstreamer specific. Not sure how hard it will be to
create a compatible file with a separate program.

Do I need a separate file per SSRC/RTP session/User (audio+video)?
Do I need to save both RTP and RTCP packets or only RTP packets?
How do I synchronize audio and video from the same user?
How can I handle streams starting and stopping at different times duration
the conference?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150728/69294ca6/attachment.html>