[gst-devel] gdk_pixbuf output sink?

[Jeremy Gwinnup]

Thanks for getting back to me.

The main concern I have is that I am developing a video annotation tool,
as opposed to a strictly playback-oriented video.  Being able to click
on points to create metadata is crucial to me.

I've developed a version of this tool that takes jpegs from each frame
of the video, but I want to be able to use the video data natively
(primarily to prevent an explosion of jpeg files)

I wanted to leverage as much of the display code from this static app as
possible. This was the motiviation for asking about a gdk-pixbuf sink. 
You mention that Gustavo has an overlay plugin - is there more
information on this (other than on the mailing list?)

Topically, to put data into a gdkpixbuf, you allocate a buffer's pixel
space, grab a handle to the data and put in the pixel info - 3 bytes for
each pixel (red, green, blue), since I'm using a simple 24-bit
colorspace.

I'm not concerned with playback speed, just the ability to display my
metadata on the video frame - I did not see an easy way for the
xvideosink to do this for me.

I've included a link to a screenshot of the static image application so
you can get an idea of what I'm trying to accomplish. (it's big - about
550k!)

http://www.gwinnup.org/ttv.png

As you can see from the image, for some brames, I have bounding boxes on
the vehicles. This data will be available every 3-15 frames, but the
interval is erratic. There is also vehicle state information (color,
weather conditions, etc) being set in a dialog, but that can be
displayed elsewhere.

Thanks for your help!
-Jeremy

On Wed, 2003-08-20 at 06:18, Thomas Vander Stichele wrote:
> Hi,
> 
> 
> > I'm new to the list, and I have an output sink question. I've
> > reconstructed the example from the gstreamer talk at GUADEC this summer,
> > and have modified it to read video data (mpeg1, using the mpeg2dec
> > decoder). I'm currently using the xvideosink output sink, but it's not
> > _quite_ meeting my needs.
> 
> What is it about xvideosink that doesn't meet your needs ?
> I believe gst-player contains some sort of gtk-ish abstraction for the
> videosink that allows it to embed it in other apps, maybe that is more
> to your taste.
> Julien, David, maybe you can comment here ?
> 
> > I'm developing a GTK2 application that plays back mpeg1 video, and I
> > want to composite the video data into a GdkPixbuf, display metadata on
> > top of the imagery, and then render that frame to a GtkDrawingArea.
> 
> That would probably work, but I have a feeling that this would be very
> slow and, if you want full FPS, will probably not succeed :)
> 
> > Does anybody have hints on how I can get towards this goal? Adding
> > another input source is not the answer here, since I do not have a 1:1
> > correspondence between the video data and my custom metadata.
> 
> I don't know how gdkpixbuf itself works.  But basically, you would write
> a plugin that accepts the colorspace/video format that gdkpixbuf handles
> (which I assume would be some raw format).  The plugin would encode to
> the pixbuf format (if this takes any encoding at all).  You could
> connect it to fakesink, and connect a signal to the handoff signal on
> fakesink, which would give you pointers to the buffer data each time it
> gets one.
> 
> As a source you would take whatever source you need, and inbetween
> source and your plugin you'd put a colorspace convertor, either
> colorspace or the new lcs one (depending on what gst version you use).
> 
> Be advised that we are generally against "plugins that feed data to apps
> through the fakesink hack".  We promote that everything be done through
> a complete gstreamer pipeline.  Even overlaying your metadata would be
> better done in a pipeline, and already works (though I need to sync back
> some hacking I did on Gustavo's overlaying plugin).  Doing your work
> that way benefits all of GStreamer too.
> 
> 
> So let us know what doesn't work for you with xvideosink, and we can
> take it from there :)
> 
> Thomas
> 
> Dave/Dina : future TV today ! - http://davedina.apestaart.org/
> <-*- thomas (dot) apestaart (dot) org -*->
> Please put me somewhere near the sea
> With one carrion angel waiting for me
> who'll be holding my heart in it's hand
> But most of all I'd like to go with a friend
> <-*- thomas (at) apestaart (dot) org -*->
> URGent, best radio on the net - 24/7 ! - http://urgent.rug.ac.be/
-- 
Jeremy Gwinnup
Software/Systems Engineer
AIS Engineering Services
phone: (937) 904-9023
fax:   (937) 656-7425
email: jgwinnup at mbvlab.wpafb.af.mil