[Bug 679466] [2.0] support negative DTS
GStreamer (bugzilla.gnome.org)
bugzilla at gnome.org
Wed Apr 24 03:31:24 PDT 2013
https://bugzilla.gnome.org/show_bug.cgi?id=679466
GStreamer | gstreamer (core) | 2.x
--- Comment #2 from Tim-Philipp Müller <t.i.m at zen.co.uk> 2013-04-24 10:26:10 UTC ---
#gstreamer, 23-04-2013 20:50:05-23:50:49 UK time:
matej_k: this non negative timestamps only thing really sucks
matej_k: big time
matej_k: also getting DTS set to -1 just because it's before segment start is
pure evil
matej_k: I'm really hoping gstreamer 2 will allow negative running/stream time
for DTS
matej_k: ds, ping
ds: matej_k: yo
matej_k: there's still a problem with qtmux timestamps
matej_k: well, actually, it's not really qtmux
matej_k: it's that first couple of frames have dts = -1
matej_k: because gstreamer can't have negative running time for DTS
ds: yeah, that's a known problem.
ds: it depends if we want to use the MP4 concept of DTS, or MPEG-TS
concept of DTS
ds: in either case, they're incompatible with each other, which will make
remuxing... interesting
matej_k: what do you mean incompatible?
matej_k: well, actually, what do you mean by different concept?
ds: MP4 uses DTS==PTS for keyframes. MPEG-TS uses DTS=PTS-1 or 2
matej_k: uh?
matej_k: that's really depending on the encoder, isn't it?
ds: no
matej_k: how is it not?
matej_k: you can't have PTS = DTS for H264 with b-pyramid
matej_k: ever
ds: IF you are assuming the MPEG-TS idea of DTS
matej_k: but you can have same thing in MP4
matej_k: you have monotonous DTS and offset to PTS
matej_k: where do you get the PTS == DTS from?
matej_k: for keyframes
ds: because that's how it works. It's a different definition of DTS
ds: offset by one or two frames from MPEG-TS DTS values
matej_k: is it in MP4 specification?
ds: it's in the AVC mapping spec
matej_k: that's really weird and doesn't make much sense to me. if it really
is like this than maybe this should be handled at muxer / demuxed level
ds: you're right, it makes no sense.
matej_k: and the rest of gstreamer should use mpeg ts DTS
ds: I generally agree, aside from the current failings of
MP4/qtmux/qtdemux
ds: Since MP4's idea of DTS is fundamentally useless
matej_k: well, you still need to have the PTS offset right
matej_k: anyway, mp4 doesn't bother me as much as the fact that you can't have
negative running / stream time for DTS
ds: indeed
matej_k: ds, i added a patch https://bugzilla.gnome.org/show_bug.cgi?id=696333
ds: we probably need to do something hackish like defining -GST_SECOND to
-1 as valid DTS values and go with it
matej_k: ds, what's wrong with GstClockTime being guint64?
ds: matej_k: GstClockTime is guint64
matej_k: ds, I meant gint
matej_k: :)
ds: matej_k: among other things, it would be an ABI break
matej_k: the way I understand it, this won't be addressed until gst 2.0
ocrete: problem is that we're using -1 as GST_CLOCK_TIME_NONE
matej_k: I'd have to be redefined
ocrete: : not likely to happen.. can't you just add an offset the DTS in the
demuxer ?
ds: ocrete: the problem is that negative DTS values need to be carried
around generally in gstreamer
matej_k: ocrete, it's not just a demuxed thing
matej_k: x264enc generates negative dts for example
ocrete: : ds: why?
ocrete: : who is -4..4 different from 0..8 ?
ds: because DTS is usually negative for the first 2 frames
ocrete: : matej_k: add an offset there too ?
matej_k: ocrete https://bugzilla.gnome.org/show_bug.cgi?id=679466
ds: and those negative values represent a real negative value
ds: er, that was circular, but you know what I mean
ds: brb
ocrete: : I understand that, but I expect that 2.0 is pretty far away, so it
may make sense to just shift the timestamps for the forseable future
ocrete: : (and manipulate the segments, etc to match)
matej_k: ocrete, it still won't help you with
gst_segment_to_running/stream_time
ds: exactly.
ds: because you still have the out-of-segment problem
ocrete: : but having encoded frames out of segment are fine I think
ocrete: : no?
ocrete: : they'll be clipped after decoding
ds: again, this is DTS, not PTS
ocrete: : sure, but you can have dts timestamps outside of the segment, no ?
ds: not if you expect calculations to work correctly. like converting to
running time
matej_k: ocrete, muxers get running time
ocrete: : hmm
matej_k: so the PTS that muxer gets start from 0; thus the DTS have to start
from -something
ds: __tim: ^
__tim: ds, matej_k: I think we de facto standardised on the mp4 variant btw
matej_k: __tim, why?
ds: __tim: not that I know of
matej_k: why would you do that?
__tim: there's quite a bit of code that assumes dts == pts for keyframes
matej_k: is there?
matej_k: because it's not true for most of my videos and things work
__tim: that's esp. how it worked in 0.10 when you couldn't even signal
dts/pts separately
matej_k: also, __tim, if you say that dts == pts for keyframes, does it mean
there are frames where dts > pts?
matej_k: because it is bound to happen
__tim: I somehow doubt that code has just gone away
__tim: I don't know what it means, it's just my impression from discussions
I have seen over the years
matej_k: and I'm pretty sure that there is code in gstreamer that assumes that
dts <= pts always
ds: __tim: the code in qtmux has gone away, at least
matej_k: __tim, I know base parse used to assume that dts = pts, but that's
gone now
__tim: ah, ok
__tim: I guess that only leaves encoders+decoders then
__tim: (to check)
matej_k: x264enc outputs negative timestamps (mpeg ts style)
matej_k: don't know about other encodes
matej_k: encoders
matej_k: haven't tried any, but they have same base class
__tim: really? didn't it try to fix those up?
matej_k: sorry, maybe you have
__tim: the x264enc element
matej_k: last time I looked it was few months ago i think
matej_k: how did you fix it?
matej_k: did you compress the timestamps ?
matej_k: because it wasn't really broken, negative timestamps are a feature :)
matej_k: __tim, so you added the dts_offset to x264enc?
matej_k: I see now what you mean
matej_k: so x264 really outputs mp4 like DTS
__tim: I did not add that
matej_k: well, someone must have :)
matej_k: but this is not very good approach
matej_k: you need to know how much to shift back when mixing mpeg ts
matej_k: muxing
matej_k:
http://cgit.freedesktop.org/gstreamer/gst-plugins-ugly/commit/ext/x264/gstx264enc.c?id=839e75d5d145d486a5c114f8f9f7ea362d12866f
matej_k: this is not good, now I have parts of pipeline that produce mp4 dts
and parts that expect mpeg ts dts
matej_k: __tim, would it make sense to have a sticky dts_shift event?
__tim: something like that
matej_k: well, I'd be nice to have something when muxing mpeg ts, 2.0 probably
won't happen for a while :)
MikeS: It took about 15 years for 1.0 to happen!
ds: I am strongly against any method that does not put the correct DTS
value in the DTS field.
matej_k: ds, that's happening now
matej_k: well,
matej_k: depends on how you define correct DTS value I guess :)
ds: The MP4 method is not correct.
__tim: ds, so a) how can you do that for negative dts if you can't express
negative dts directly? (other than via an additional dts offset)
__tim: and I forgot what b) was ;)
matej_k: well, DTS_OFFSET event would probably solve things for me right now,
but it's not a pretty solution
__tim: ds, I don't really care, someone just go and decide for one or the
other and write a design document how it should work
__tim: ds, ah yes and b) what's the "correct" dts?
ds: The correct dts is the time when the decoder should start decoding a
frame. Decoding is assumed to be instantaneous, so you can have dts==pts for
some frames
ds: but never dts > pts.
__tim: right
ds: it has to be exactly right to mux correct MPEG-TS streams, otherwise
hardware decoders will fail
matej_k: well, yeah, but maybe you can say that you can never have dts +
dts_offset > pts
matej_k: where dts_offset would be a sticky event
ds: or you could just do it correctly from the start
matej_k: ds, it's kinda late for that, isn't it
ds: no
matej_k: given that Gstclocktime is unsigned
ds: who cares
ds: it's just 64 bits.
ds: we get to use them as we wish
__tim: we don't really get to change the special NONE value
matej_k: yeah
__tim: though we can of course add a CORRECT_DTS field
__tim: ds, can you always create correct-for MPEG-TS from mp4?
ds: __tim: no
ds: because mp4 is fundamentally broken
__tim: if so, are we just talking about which is the better way of
representating the same thing?
matej_k: ds, isn't there way to save the offset in edit list atom?
ds: matej_k: not that I know of, but I haven't read all the updates to
the AVC file format spec
matej_k: ds, it's not really relevant anyway. my problem now is that x264enc
generates DTS and there is no way to get the exact same DTS to mpeg ts muxer
__tim: matej_k, just posting "I don't think everyone agrees on this" isn't
terribly helpful imho, esp. not without the background of the discussion here
:)
__tim: matej_k, someone needs to propose a scheme that makes sense and
document it ;)
matej_k: __tim, ok, I'll quote ds then :)
GregDR: Does it make sense that the first buffers to hit mpegtsmux sink pads
always have DTS and PTS == 0? I'm seeing that consistently with my pipeline
and it is causing a bug in mpegtsmux and I just want to know if it is specific
to my stream
GregDR: the first video buffer has DTS == 0. After that, all video buffers
only have valid PTS. This mpegtsmux prefers the last valid DTS over PTS to
stamp the outgoing buffer. Therefore, my outgoing timestamps are always 0.
GregDR: FYI -- this only happens on master. Not on 1.0
Also: "DTS being broken in GStreamer" post on mailing list:
http://lists.freedesktop.org/archives/gstreamer-devel/2013-April/040603.html
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
More information about the gstreamer-bugs
mailing list