[Bug 642595] New: [camerabin2] audio recording support (sync audio and video)

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=642595
  GStreamer | gst-plugins-bad | git

           Summary: [camerabin2] audio recording support (sync audio and
                    video)
    Classification: Desktop
           Product: GStreamer
           Version: git
        OS/Version: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: gst-plugins-bad
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: thiago.sousa.santos at collabora.co.uk
         QAContact: gstreamer-bugs at lists.freedesktop.org
      GNOME target: ---
     GNOME version: ---


It's been some days since I've started adding audio recording support for
camerabin2, and after some discussions and tries I decided to file a bug to
gather ideas on how to solve this.

Camerabin2 has a videosrc that is running from the start and feeding the
viewfindersink. When the user decides to start a video capture, the audiosrc
is set to playing and starts pushing buffers into encodebin, as does the
videosrc.

The problem is that the timestamps of the audiosrc starts at a non-zero value
and encodebin plugs an audiorate element that fills the gap from 0 to the first
timestamp with empty gap buffers and the muxer uses those.

If audiorate wouldn't do this, both video and audio would have the same
timestamps getting to the muxer (in terms of running_time, at least) and we'd
all be happy I think, but this gap filling is nice for other situations.

So the problem is how to make the muxer drop/ignore this initial GAP data or
how to deal with this situation in general? Remember that some muxers (qt) can
support segments.

Do we need some kind of 'stream/segment synchronization element'?
Should audiorate have a skip-to-first property just like videorate and how to
set it from outside of encodebin?

Making the muxers drop buffers is dangerous because it is dropping encoded
data.

So I ran out of more ideas for today, any help is appreciated here.

Btw, thanks to Win and Jan who already helped me a lot on understanding this
issue :)

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.