[gst-devel] fractions

[Thomas Vander Stichele]

On Thu, 2004-07-15 at 05:22, David Schleef wrote:
> On Thu, Jul 15, 2004 at 02:23:31AM +0200, Thomas Vander Stichele wrote:
> > Nice skid around the previous example for framerate :)  I must admit it
> > threw me off as well when I checked what the original fraction was.
> 
> Actually, I was just not being petty by pointing out that the
> different in rate turns out to be 1 frame per millenia.

It's ok to be petty.  I gave you the example because you told me that
the correct fraction can be calculated from the float without error. 
Now you're telling me that there's an error but the error is not
significant.  But why get it wrong, even slightly, when we can get it
right ?


> > Not sure how you would want to implement that, and not sure it's a good
> > thing to do right now.  Maybe it makes sense for a 0.9 project, but it
> > seems to invasive for this series, and I don't yet see the gain in it. 
> 
> Funny, I'm saying the same thing about fractions.

How are they as invasive as a general number type ? My patch only adds
one GValue type, together with the functions needed to support it. 
Porting over GValue's double and int types together with this new
fraction type to a general Number type sounds a lot more complicated and
invasive to me.


> > But it sounds to me like this means you have agreed on having a
> > "fraction" concept in GStreamer ?
> 
> One of the drafts of the new caps system had fractions.  I dropped
> it because there wasn't a need at the time, and I didn't care to
> write the extra code.  I'm merely making sure that adding them now
> is the right decision.

I agree that it should be made sure if it is.  I'm convinced that it is,
because people like Ronald and Wim told me before that they need it for
framerate, and I now figured out myself that I need it for aspect ratio.

> > - I've given examples that you got wrong for stuff you said could be
> > done correctly
> 
> Keep in mind that the given framerate is probably a mangled form
> of 30000/1001.  There aren't very many interesting non-integer
> framerates.

For editing purposes, the fact that they are mangled form of something
else is irrelevant.  You still need to be able to edit them, and maybe
even provide the option of fixing them again. Let's not add to the body
of media out there with rounded-off framerates because we thought floats
were good enough.

> > - Based on the examples, I don't see how we can feed encoders correct
> > frame rate fractions without rounding errors
> 
> Apparently, based on the 29.969999 framerate, rounding doesn't matter.

... but it does.  A snippet from a conversation on this subject:

<derf_> thomasvs: The fun I had with them, is that I'd cut a segment out
of a video, edit it, and then try to splice it back in.
<derf_> And it would refuse to splice it in, because the segments had
different frame rates.
<derf_> Because a float does not retain enough precision to restore the
original framerate when writing the file.
<derf_> My favorite bit, though, was that if you ran the whole stream
through mencoder, so that it mucked up the framerate of the whole clip,
and THEN tried to splice a segment out, edit it, and splice it back in,
it still didn't work.
<derf_> Because it changed the framerate AGAIN... it couldn't even get
it back to the same inaccurate float value it had before.

So the rounding errors made it impossible for mencoder to do a common
operation we eventually want to support.

> > - Every codec project out there seems to think fractions are the right
> > way to handle both aspect ratio and framerate - what makes you think all
> > of them conbined are wrong ?
> 
> My speculation would be that pairs of integers are easier to handle
> on CPUs that don't have floating point units.  Since the oldest video
> formats predate FPUs being common and newer video formats generally
> just copy old ones, it stuck.

... or maybe it was the correct solution in the first place.  I don't
think either Matroska or Theora chose fractions "because MPEG did it" :)
I've checked the patch that changed from float to fraction in FFMPEG,
giving me a hint of the code you think would solve this using floats,
and the patch they did cleaned up a bunch of dubious guessing code.  I
don't think FFMPEG made this chance last year to satisfy non-FPU
machines.


> > I don't think I need to produce any more factual evidence for the
> > usefulness of GstFraction than I already have.
> 
> I'm calling into question your "factual evidence".  You claim that
> floating point numbers are somehow grossly inaccurate and incapable
> of expressing fractions.  This is patently not true.  The set of
> fractions useful for our purposes (say, denominators less than 1e6)
> are many orders of magnitude less dense on the range [0,1] as are
> unique double-precision floating point numbers.  For that set of
> fractions, there is an exact 1-1 mapping to doubles.

I'd love to reason about the *theory* of it some other day.  But the
example I gave you is a *reference* file from a *codec* project and I
want GStreamer to be able to support it correctly, not
sort-of-correctly.  If your code or mechanism cannot convert the
fraction to a float and back again, then it is clear that for our
purposes floats are inadequate.


> I'm also concerned with what I percieve as roadmapping by feature.
> That is, 1) find a problem, 2) add something to the core to fix
> the problem.  I don't think of this as being 100% bad, but I
> would prefer to have the steps 2) think of similar problems,
> 3) decide on a long range direction, and 4) pick a solution that
> solves the current problem but also makes it easy to transition
> later.

I agree to a certain extent.  First of all, I think roadmapping by
feature in itself is not bad at all.  People work differently.  I'm the
kind of person that needs a real use case or a practical problem to
think about a theoretical solution.  Such a real use case is "please
handle aspect ratios correctly".  Some of you are gifted with the
ability to theorize endlessly without needing practical examples, but
I'm not.  It doesn't mean I can't contribute to the solution.

So, before reading up and thinking about it I couldn't care less if it
was fractions or floats.  I wanted it to work.  So I do the work, talk
to people, read code, read references, try out stuff, and everyone is
telling me that "floats are bad, you should use fractions."  Everyone
except you, whose opinion I obviously respect a lot.  I'm trying to get
you to convince me why floats are good enough given that every codec
project or person that has an opinion about it (excluding me) is saying
it isn't.

I have a maths background, and an engineering background, and I can
think in both fields.  The reason we're having a discussion is because
it's a subjective issue.  We both know fractions are the mathematically
correct solution.  We both know that floats have errors, but in some
cases the error doesn't matter.  We just disagree about what error we
tolerate.

The idea of fractions in GStreamer is not something that fell out of the
sky.  Wim and Ronald have said a long time ago they'd like to have it,
and you have given it thought as well but then decided we didn't really
need it yet.

Now, in practice.  I did "2) think of similar problems" - framerate is
something both Ronald and Wim have complained about and both said
fractions would be a lot better.  I've done 3) too.  I think fractions
are a necessity, and we should find a good way to support them.  I think
it would be nice to have a general range type instead of inventing a new
specific range type.  Also, I've said that I think framerate should be
moved over eventually as well.

And as for 4), I think that's what we should have in the future.  To fix
the current problem, adding a GstFraction type is enough, and will allow
us to easily transition to a GstNumber type if you think that is the
right end term solution (I have no idea what you think it would solve)
by just renaming functions.  It doesn't conflict with anything else,
everything that worked before still works, and things that didn't work
before can now be made to work.

> Clearly, it would be nice to have something that expresses
> fractions exactly in the same way that we express integers exactly.
> IMO, in the long term, it's best to get rid of the distinction
> between integers, fractions, and doubles.  It would also be useful
> to have the concept of infinity, open ranges, etc.  At some point,
> somebody (probably me) will have to make sense of the current
> jumble, create a transition plan, write some code, and pull us
> out of the mess.  Personally, I would prefer that you use doubles,
> since it's fewer things to fix later.

I think that if you already feel we will need something in the future
that expresses fractions exactly, the code that already does it today
with GstFraction will be easier to migrate than having the current patch
adapted to use doubles then back again.


> I'm a little annoyed that you were so quick to belittle what I
> considered to be constructive criticism -- especially the use
> cases I proposed that don't fit your model well.  I throw out
> random usages ideas, because many times it reminds someone of
> an actual use case that doesn't work well.

I'm not belittling them at all.  I appreciate the fact that you try to
stimulate me into delving deeper in order to convince myself what the
correct solution should be.  I do expect you however to give me real
world use cases of theoretical problems you throw up, respecting the
fact that I'm better at solving real problems than theoretical ones.  I
don't mean to belittle the theoretical problems, I just can't get my
head round them without examples.  Clearly I respect people that do not
need these examples a lot.

So the real world use case about the video you thought couldn't be
displayed, I've said how to display it.  And the theoretical idea of
varying pixel aspect ratio smoothly makes no sense to me because I don't
have a practical example of where that would be useful, so please
provide me with one.


>   IMO, this is an
> important part of vetting code -- throw eggs at it and see what
> sticks.  Your attempts at trying to get around this kind of
> discussion (in my impression) makes me more likely to jump on
> major changes that you propose in the future.

And I'm sure you understand I have the same feeling from my experience. 
I'm asking for examples - the ones that have practical value I've
addressed, and the ones that are theoretical without a practical example
I can't answer.  I do not feel I have tried to get around them; all
practical issues have been dealt with IMO, and if not, bring them up
again.

Conversely, a few of the practical examples I've given on why floats are
not good enough you have not adressed, except with the blanket "but the
error is not relevant".  I'd be superhappy if you could convince me
floats are good enough, because then I don't have to bother.  Only you
give me the same feeling of getting around the discussion.

I know we both want the issue solved.  I appreciate the fact that you
don't want to add code that might not be needed.  But it sounds like you
already feel we eventually  need fractional code anyway.  And I'm not
sure you have given the actual practical issue of framerate conversion
and pixel aspect ratio a lot of thought or testing (maybe you have, but
I can't tell).  I'm having the feeling that if you were to do so you'd
reach the same conclusion as every one else.


I understand your resistance to change.  On the other hand, I'd like you
to step back and tell me what, ideally, you want me to do, in order to
get aspect ratio handling correct.  Because in the end I really do want
to get aspect ratio handled correctly, and we don't at the moment.  So
give me a good plan I can work on it.

I don't know what to do about this.  I'd like this patch in - so unless
you have a really good reason on why everyone else is wrong I'd like to
commit it today.

Thomas



Dave/Dina : future TV today ! - http://www.davedina.org/
<-*- thomas (dot) apestaart (dot) org -*->
And marry me baby
And sleep with me baby
We'll sleep with the lights on
And we'll sleep with our clothes on
<-*- thomas (at) apestaart (dot) org -*->
URGent, best radio on the net - 24/7 ! - http://urgent.fm/