[gst-devel] Muxing subtitles to matroska [Was: Re: dvdsrc status update, sundry questions]

[Mark Nauwelaerts]

Michal Benes wrote:
>>> Well, just hoping that these hacks around collectpads can shed their "hacking"
>>> existence some day, and that the functionality they provide can make it cleanly
>>> into upstream, as much re-usable as possible ... :-)
>>>
>> Probably by considering CollectPads to have been an experimenting and
>> creating a CollectPads2 to replace it, as we've done with decodebin2
> 
> I have been thinking about this last night and I think that the real
> problem here is that CollectPads is used in very different situations
> (muxer ver. adder), therefore it is very difficult to implement correct
> NEWSEGMENT handling (I do not know what is correct anyway).
> 
> So it would be probably better to implement new BaseMuxer class (that
> uses CollectPads internally). This class would automatically find the
> oldest buffer handling all the NEWSEGMENT stuff. I have quick checked
> oggmux, avimux ffmux and matroskamux. It seems that they are interested
> only in the oldest buffer so that they they could all benefit from the
> BaseMuxer class (I am not quite sure about oggmux as I do not understand
> it completely).
> 
> BaseMuxer would probably need some api addition to CollectPads to
> disable/enable waiting for a pad (alternatively BaseMuxer could use
> filler buffer hack internally).
> 
> Most important BaseMuxer API would be render function that looks like
> this
> 
> /**
>  * GstBaseMuxerRenderFunction:
>  * @muxer: the #GstBaseMuxer that trigered the callback
>  * @data: the #GstBaseMuxerData that received the oldest buffer
>  * @buffer: the oldest buffer
>  *
>  * A function that will be called when #GstBaseMuxer found the oldest
> buffer.
>  *
>  * Returns: GST_FLOW_OK for success
>  */
> typedef GstFlowReturn (*GstBaseMuxerRenderFunction) (GstBaseMuxer
> *muxer, GstBaseMuxerData *data, GstBuffer *buffer);
> 
> Please let me know your thoughts.

I believe the most important (and tricky) of the above lies in the "some api
additions to CollectPads" part.  This is currently at the core of
things/problems regarding e.g. muxers not having enough say in what is going on.
 E.g. though CollectPads does publicly expose a segment for inspection, but this
does not provide enough "thread" responsivity for your filler trick.
This part of it brings us back to CollectPads2 etc

Beyond that, avimux, matroskamux etc are typically interested in the "oldest
buffer", but there may be intricate differences in a muxer's taste (e.g. how to
deal with GST_CLOCK_TIME_NONE buffers, typically carrying headers ?).  Other's
taste may be even more exotic, e.g. oggmux.  This type of "finer control" would
be lost versus the gain of factorizing out a "typical/simple" _collected
function.  So the tripping point seems to be how much other trickery this
class/BaseMuxer would have to (and can) take care of beyond _collected, and
therefore, again, how easy (or how hacking) CollectPads(2) makes this (and it
not being a point of hiding the hacking in a BaseMuxer ;-) ).
Also, if BaseMuxer is to be truly a base class rather than a component to use
like CollectPads (re-use by inheritance versus composition), then it raises
other questions such as "who" will handle _request_pad, _release_pad, _setcaps
etc.  If that can be adressed, it would require quite a re-write for existing
muxers, but may have the benefit that they are more aligned.

As for "some api additions to CollectPads", I think the following would be a
start, at least based on current (unaddressed) use-cases:
- give some more (direct interactive) access to events; not necessarily all of
them, depends on how much wanting to prevent the mux element to "shoot itself in
the foot"
- (as you say) request CollectPads to not wait around for some pad; the element
would then typically have a _peek to see if there is something interesting
(would have to be careful about detecting eos in these cases).  To keep
buffers/streams in increasing time order, it should also be able to request
CollectPads to start waiting around again for some activity on the pad in
question (when nearing announced NEWSEGMENT time), so that other pads do not
overshoot.  So, a sort of gst_collect_pads_wait (collect, pad, TRUE/FALSE), to
be called from within event handling or _collected function.

However, the plans/direction for CollectPads are not clear to me,
considering that a suggestion like the first item above was already given in a
patch in http://bugzilla.gnome.org/show_bug.cgi?id=340060, but apparently to no
avail ... (maybe the problem there is in some details ?)

Regards,
Mark.