[gst-devel] fractions and aspect ratios

[Thomas Vander Stichele]

On Wed, 2004-07-14 at 10:07, Benjamin Otte wrote:
> On Tue, 13 Jul 2004, Thomas Vander Stichele wrote:
> 
> > - David is convinced that GstFraction is overkill, and floats are
> > enough.  My argument is that going to a float loses the information of
> > which two integers you came from, making it impossible to figure out
> > what the intended target video size was supposed to be since you cannot
> > figure out anymore what possible WxH combinations are correct scaled
> > values of input WxH (ie, you cannot determine anymore which W-H values
> > are integer results of scaling)
> >
> Why is the information necessary from where you came?

Because you want to make sure you scale to an end width and height that
are integers.  Short of looping over all allowed integers and doing the
calculation on them, I don't see how you'd want easily figure out what
end WxH results are correct scales of the source material.

There probably are instances where a correct scale would give you too
big a window, and then you can still choose to crop a pixel in scaling
for the sake of reason, but in general a correct scale should be
prefered.

Same problem for framerate - libraries expect you to give them a
ratio/fraction/set of two integers.  This needs to be exact, not
approximative, and you need to figure out what the exact one is.  How do
you go from a float to the fraction it's supposed to represent ? Which
fraction did 14.985014 come from ?

> Also, what are other systems/libraries (theora, X, ...) using for pixel
> aspect ratios? 

mpeg2dec uses pixel_width and pixel_height.
theora uses aspect_numerator and aspect_denominator, which (I'm not
quite sure yet, but I assume it) expresses the physical display aspect
ratio you want the video to be displayed at.  (ie, w/h = DAR/PAR can
give you PAR).
avi doesn't express this, I think it leaves it up to the app to "guess"
what the eventual DAR is supposed to be - I'm sure Ronald can shed some
light on this beyond what he already posted.
matroska uses pixel_width and pixel_height.

Did I miss any important libraries ?

The fact that all of these chose two ints should tell you that using a
float is a bad idea.


X uses the stupid system of having the physical w and h in mm, and the w
and h in pixels, from which you can calculate the pixel aspect ratio,
but not the theoretically correct value because of the use of mm :)  I
asked vektor for some help and he had the exact same problems as I had,
resulting in similar code to figure out the correct pixel aspect ratio
based on these values.


Anyway, the requirements for us are slightly different since these
libraries only need to express the PAR of whatever they handle (the
decoded video for the decode libraries, the display's PAR for
displays).  We need to do calculations involving the PAR of the incoming
video data and the PAR of the display, as well as the ratio of the
source video and the ratio of the display window the video will use.

As an example of the problem for framerate, here's a quote from
mpeg2enc.cc:

/*
 * We can't use fractions in static pad templates, so
 * we do something manual...
 */
static void
add_fps (GstCaps * caps)
{
  GstStructure *structure = gst_caps_get_structure (caps, 0);
  GValue list = { 0 }, fps =
  {
  0};
  gdouble fpss[] = { 24.0 / 1.001, 24.0, 25.0,
    30.0 / 1.001, 30.0, 50.0,
    60.0 / 1.001, 60.0, 0
  };



> > - I also looked at creating a GST_TYPE_RANGE that would be implemented
> > with a GArray with three members (lower, upper, increment).  A lot of
> > code would be the same as for the LIST type, and we could convert over
> > the GST_TYPE_INT_RANGE and GST_TYPE_DOUBLE_RANGE to it.  What do you
> > think ?
> >
> I don't think you want an increment in it. Just for one rerason: It's not
> worth it. A well-behaving type for caps implements the operations
> intersect and subtract. Since you want a universal range type for every
> fixed GValue type (this includes strings), you will have to make
> subtractions and intersections work for stuff like this: (string) [
> "hello", "world, "!" ]

In my mind there is no point in having string ranges.  What would your
example represent ? What does the range from "hello" to "world"
represent even without the increment ?


> > - Having an increment value in RANGE types would also be nice for
> > framerate for webcams or width/height for encoders.  For webcams,
> > allowed framerates are a multiple of 15/16, and for encoders, typically
> > multiples of 8 or 16 only are allowed.  In the case of webcams, being
> > able to specify 75/16 instead of 4.6875 seems like a plus to me.
> >
> A huge list is probably not an option in that case? (I have no clue up to
> what framerate is supported, but webcams should have an upper limit of the
> framerate they support, and with even 100 values in that list, you'd get
> up to 1500/16 = 93.75 fps, which is quite a lot)

There are 63 possible values for the fps index.  What I'm saying is that
something like [ 15/16, 90/16, 15/16 ] is a hell of a lot more
understandable than having ( 0.9375, 1.875, ... (numbers snipped because
I can't be bothered to calculate them right now, which goes to prove my
point) )

Also, setting framerate=90/16 in filtercaps is a lot easier to remember
than 5.625.  Because if you set 5.65 instead, obviously linking will
fail.

> Also, does v4l(2) or whatever we use in these cases even allow such exact
> querying of capabilities? Because if they don't it's a bit of a moot point
> to add that now.

Yeah, they do.  Which is why I ran into it when making v4lsrc support
webcams in the first place.  I'm assuming the system of index * 15/16
was chosen so that both 7.5, 15.0 and 30.0 could be chosen as framerates
using less than 8 bits of index, while still allowing easy calculation
of it.

Thomas


Dave/Dina : future TV today ! - http://www.davedina.org/
<-*- thomas (dot) apestaart (dot) org -*->
Kiss me please kiss me
Kiss me out of desire baby not consolation
Oh you know it makes me so angry cause I know that in time
I'll only make you cry
<-*- thomas (at) apestaart (dot) org -*->
URGent, best radio on the net - 24/7 ! - http://urgent.fm/