[gst-devel] Aspect ratios, xvideosink and vdv
Cameron Hutchison
camh+gst at xdna.net
Thu Jan 9 06:59:04 CET 2003
I'm wondering how gstreamer handles differing pixel aspect ratios of the
images as it passes through a pipeline, or how a plugin writer should
handle it.
I've seen that mpeg2dec has the caps properties of pixel_width and
pixel_height, which specify the aspect ratio of the pixels in the stream
output by mpeg2dec.
Most examples that I've seen plug mpeg2dec into xvideosink, which uses
pixel_width and pixel_height to determine the the window size. Embedded
in this calculation is the assumption that the screen pixel aspect ratio
is 1:1 (ie. square pixels). I gather that xv internally scales the
picture to the window size, which corrects for the non-square pixels in
the source.
However, what if mpeg2dec was connected to an element that wrote the raw
data out to disk (assuming enough I/O bandwidth) in a format that
assumes square pixels (ie. no capability to store the pixel aspect ratio)?
Should that element know to check its sink's peer caps for "pixel_width"
and "pixel_height", and scale the image accordingly? I would guess not,
since that function should be performed by a videoscale element. But
how would an autoplugger know whether an element can self-scale an image
(such as xvideosink, or a hypothetical vidixsink)?
The reason I'm thinking about this is that I've been playing with taaz's
(David Lehn) vdv player (vdv.sf.net). This is a simple DVD player, very
much in development.
When the mouse moves over the displayed image, the pointer coordinates
are sent to the dvdnav library for processing. The problem with this is
that the mouse event coordinates are in screen coordinates, whereas
dvdnav wants the coordinates in pre-scaling image coordinates. As you
move the mouse, the dvdnav highlight events come out for the wrong
positions.
Even though nothing is highlighted yet, it would be nice for the
highlights to correspond with the mouse positions when highlighting gets
implemented.
I've got a couple of patches - one against xvideosink and the other
against vdv. I'm mainly posting them as a point of discussion rather
than asking that they be committed. It may also be of interest to Julien
Moutte, since it would probably affect the events he'll be handling.
The patch to xvideosink exposes four struct members as properties and
adds two more members which are also exposed as properties.
- width and height are available as the properties input_(width|height)
- pixel_width and pixel_height are available as properties of the same
name
- I've added output_width and output_height both as struct members and
as properties with the same name. This corresponds to "real_x" and
"real_y" as delivered in the have_size signal.
The names of the properties probably suck, and the input and pixel sizes
are available from the caps properties, except I dont know how to get
the caps properties from the have_size signal handler. I dont think
gst_pad_get_caps will return them until after the function that emits
have_size returns (gst_xvideosink_sinkconnect).
With these patches, vdv now gets dvdnav emitting the right highlight
events for the mouse position.
-------------- next part --------------
Index: sys/xvideo/xvideosink.c
===================================================================
RCS file: /cvsroot/gstreamer/gst-plugins/sys/xvideo/xvideosink.c,v
retrieving revision 1.28
diff -u -r1.28 xvideosink.c
--- sys/xvideo/xvideosink.c 27 Dec 2002 23:03:46 -0000 1.28
+++ sys/xvideo/xvideosink.c 9 Jan 2003 14:55:44 -0000
@@ -56,6 +56,12 @@
ARG_TOPLEVEL,
ARG_AUTOSIZE,
ARG_NEED_NEW_WINDOW,
+ ARG_INPUT_WIDTH,
+ ARG_INPUT_HEIGHT,
+ ARG_OUTPUT_WIDTH,
+ ARG_OUTPUT_HEIGHT,
+ ARG_PIXEL_WIDTH,
+ ARG_PIXEL_HEIGHT,
};
#define FORMAT_IS_RGB(format) ((format) == GST_MAKE_FOURCC ('R','G','B',' '))
@@ -137,6 +143,24 @@
"Needs a new X window",
"Request that a new X window be created for embedding",
TRUE, G_PARAM_WRITABLE));
+ g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_INPUT_WIDTH,
+ g_param_spec_int ("input_width", "Input Width", "Image width from source (pre-scaling)",
+ G_MININT, G_MAXINT, 0, G_PARAM_READABLE));
+ g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_INPUT_HEIGHT,
+ g_param_spec_int ("input_height", "Input Height", "Image height from source (pre-scaling)",
+ G_MININT, G_MAXINT, 0, G_PARAM_READABLE));
+ g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_OUTPUT_WIDTH,
+ g_param_spec_int ("output_width", "Output Width", "Image width on output (post-scaling)",
+ G_MININT, G_MAXINT, 0, G_PARAM_READABLE));
+ g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_OUTPUT_HEIGHT,
+ g_param_spec_int ("output_height", "Output Height", "Image height on output (post-scaling)",
+ G_MININT, G_MAXINT, 0, G_PARAM_READABLE));
+ g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_PIXEL_WIDTH,
+ g_param_spec_int ("pixel_width", "Pixel Width", "Pixel width of source (scale factor)",
+ G_MININT, G_MAXINT, 0, G_PARAM_READABLE));
+ g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_PIXEL_HEIGHT,
+ g_param_spec_int ("pixel_height", "Pixel Height", "Pixel height of source (scale factor)",
+ G_MININT, G_MAXINT, 0, G_PARAM_READABLE));
gobject_class->set_property = gst_xvideosink_set_property;
gobject_class->get_property = gst_xvideosink_get_property;
@@ -313,22 +337,22 @@
}
static void
-gst_xvideosink_get_real_size (GstXVideoSink *xvideosink,
- gint *real_x, gint *real_y)
+gst_xvideosink_get_output_size (GstXVideoSink *xvideosink,
+ gint *output_x, gint *output_y)
{
gint pwidth, pheight;
- *real_x = xvideosink->width;
- *real_y = xvideosink->height;
+ *output_x = xvideosink->width;
+ *output_y = xvideosink->height;
pwidth = xvideosink->pixel_width;
pheight = xvideosink->pixel_height;
if (pwidth && pheight) {
if (pwidth > pheight) {
- *real_x = (xvideosink->width * pwidth) / pheight;
+ *output_x = (xvideosink->width * pwidth) / pheight;
}
else if (pwidth < pheight) {
- *real_y = (xvideosink->height * pheight) / pwidth;
+ *output_y = (xvideosink->height * pheight) / pwidth;
}
}
}
@@ -361,11 +385,15 @@
else
xvideosink->pixel_height = 1;
+ gst_xvideosink_get_output_size (xvideosink, &xvideosink->output_width,
+ &xvideosink->output_height);
+
print_format = GULONG_FROM_LE (xvideosink->format);
- GST_DEBUG (0, "xvideosink: setting %08x (%4.4s) %dx%d\n",
+ GST_DEBUG (0, "xvideosink: setting %08x (%4.4s) %dx%d -> %dx%d\n",
xvideosink->format, (gchar*)&print_format,
- xvideosink->width, xvideosink->height);
+ xvideosink->width, xvideosink->height,
+ xvideosink->output_width, xvideosink->output_height);
g_mutex_lock (xvideosink->lock);
@@ -403,17 +431,12 @@
g_mutex_unlock (xvideosink->lock);
- {
- gint real_x, real_y;
-
- gst_xvideosink_get_real_size (xvideosink, &real_x, &real_y);
-
- if (xvideosink->auto_size) {
- _gst_xwindow_resize (xvideosink->window, real_x, real_y);
- }
- g_signal_emit (G_OBJECT (xvideosink), gst_xvideosink_signals[SIGNAL_HAVE_SIZE], 0,
- real_x, real_y);
+ if (xvideosink->auto_size) {
+ _gst_xwindow_resize (xvideosink->window, xvideosink->output_width,
+ xvideosink->output_height);
}
+ g_signal_emit (G_OBJECT (xvideosink), gst_xvideosink_signals[SIGNAL_HAVE_SIZE], 0,
+ xvideosink->output_width, xvideosink->output_height);
return GST_PAD_CONNECT_OK;
}
@@ -441,6 +464,10 @@
xvideosink->toplevel = TRUE;
xvideosink->width = 100;
xvideosink->height = 100;
+ xvideosink->output_width = 100;
+ xvideosink->output_height = 100;
+ xvideosink->pixel_width = 1;
+ xvideosink->pixel_height = 1;
xvideosink->formats = formats;
xvideosink->lock = g_mutex_new();
xvideosink->disable_xv = FALSE;
@@ -675,6 +702,24 @@
break;
case ARG_AUTOSIZE:
g_value_set_boolean (value, xvideosink->auto_size);
+ break;
+ case ARG_INPUT_WIDTH:
+ g_value_set_int (value, xvideosink->width);
+ break;
+ case ARG_INPUT_HEIGHT:
+ g_value_set_int (value, xvideosink->height);
+ break;
+ case ARG_OUTPUT_WIDTH:
+ g_value_set_int (value, xvideosink->output_width);
+ break;
+ case ARG_OUTPUT_HEIGHT:
+ g_value_set_int (value, xvideosink->output_height);
+ break;
+ case ARG_PIXEL_WIDTH:
+ g_value_set_int (value, xvideosink->pixel_width);
+ break;
+ case ARG_PIXEL_HEIGHT:
+ g_value_set_int (value, xvideosink->pixel_height);
break;
default: {
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
Index: sys/xvideo/xvideosink.h
===================================================================
RCS file: /cvsroot/gstreamer/gst-plugins/sys/xvideo/xvideosink.h,v
retrieving revision 1.11
diff -u -r1.11 xvideosink.h
--- sys/xvideo/xvideosink.h 27 Dec 2002 23:02:40 -0000 1.11
+++ sys/xvideo/xvideosink.h 9 Jan 2003 14:55:44 -0000
@@ -54,6 +54,7 @@
guint32 format;
gint width, height;
+ gint output_width, output_height;
gint pixel_width, pixel_height;
gint64 correction;
GstClockID id;
-------------- next part --------------
Index: vdv/gst.py
===================================================================
RCS file: /cvsroot/vdv/vdv/vdv/gst.py,v
retrieving revision 1.3
diff -u -r1.3 gst.py
--- vdv/gst.py 8 Jan 2003 07:11:21 -0000 1.3
+++ vdv/gst.py 9 Jan 2003 14:55:30 -0000
@@ -80,6 +80,9 @@
#return self.run()
self.setClosedCaptionsEnabled(0)
+ self.pixel_width = 1
+ self.pixel_height = 1
+
def addIdleFunctions(self, idle_add):
idle_add(self.idle)
@@ -252,10 +255,12 @@
print 'buttoninfo total:%d' % (total,)
def pointerSelect(self, x, y):
- self.src.emit('pointer_select', x, y)
+ (tx, ty) = translatePixelCoords(x, y)
+ self.src.emit('pointer_select', tx, ty)
def pointerActivate(self, x, y):
- self.src.emit('pointer_activate', x, y)
+ (tx, ty) = translatePixelCoords(x, y)
+ self.src.emit('pointer_activate', tx, ty)
def src_notified(self, sender, pspec):
if pspec.name == 'streaminfo':
@@ -263,8 +268,27 @@
elif pspec.name == 'buttons':
self.buttons(sender, pspec)
+ def translatePixelCoords(self, x, y):
+ """Translate a screen coordinate to an image coordinate
+
+ The coordinate system of the screen output differs from the
+ coordinate system of the image source due to the different
+ pixel aspect ratios of each. scalePixelAspect will translate
+ from a screen coordinate to an image coordinate so that the
+ correct cursor positions are passed to the DVD menu handling
+ code.
+ """
+ (tx, ty) = (x, y)
+ if self.pixel_width > self.pixel_height:
+ tx = (x * self.pixel_height) / self.pixel_width
+ elif self.pixel_height > self.pixel_width:
+ ty = (y * self.pixel_width) / self.pixel_height
+ return (tx, ty)
+
def have_size(self, videosink, width, height):
self.getUI().setVideoSize(width, height)
+ self.pixel_width = videosink.get_property("pixel_width")
+ self.pixel_height = videosink.get_property("pixel_height")
def have_xid(self, videosink, xid):
self.getUI().setXID(xid)
More information about the gstreamer-devel
mailing list