Variable frame rate stream in Matroska container

[Roland Elek]

Hi,

We are developing an application that includes muxing variable frame
rate video streams into Matroska files, and playing them back at
different speeds, including single frame stepping, forwards and
backwards. Variable frame rate here means switching between fixed
frame rates, but any use case that involves frame durations different
from the track default is affected by these issues..

Reverse playback and frame stepping, as we understand, relies on
correct seeking, for which it is necessary to always end up at an
I-frame. The problem is that matroskamux generates an incorrect (or at
least confusing) output if the frame rate (more precisely, the
duration of a frame) is different from the track default, that is,
after switching frame rate. All frames look like I-frames in the
Matroska stream, because when a full Block element is assembled for
the frame (because of the non-default duration), it does not include a
ReferenceBlock element. This breaks seeking, and as a result, reverse
playback. The issue has already been reported:
https://bugzilla.gnome.org/show_bug.cgi?id=140783 SimpleBlock with
default duration works fine. (I guess it's because there is no
referencing involved, only 2 flags.)

Another idea for fixing seeking was, on caps renegotiation, closing
the current Segment in the Matroska stream, and opening a new one, and
in that, a new track with the new default duration. Multi-segment
Matroska support is, however, not yet implemented:
https://bugzilla.gnome.org/show_bug.cgi?id=334082

Do you have any suggestions on the future of these issues, or on how
to work around them, and get a Matroska file with variable frame rate
video suitable for trick mode playback and frame stepping?

Thank you in advance for your help.

Best regards,
Roland Elek