[gst-devel] GStreamer and Closed-Captioning support

[Bryen Yunashko]

On Jan 8, 2008 10:01 AM, David I. Lehn <dlehn at vt.edu> wrote:

> * Stefan Kost <ensonic at hora-obscura.de> [2008-01-08T04:10:44-0500]:
> > Quoting Stefan Kost <ensonic at hora-obscura.de>:
> > > Quoting Bryen Yunashko <byunashko at gmail.com>:
> > >
> > >> Can anyone send me information on whether gstreamer supports
> > >> closed-captioning with gstreamer?  Particularly, I'm trying to find
> it in
> > >> Totem.
> > > Could you elaborate a bit what that is? Is it reading out subtitles?
> > > Any pointers? I want to prepare a list of taks for the next google
> > > summer-of-code application. This could be a potential subproject.
> >
> > Replying to myself:
> > http://en.wikipedia.org/wiki/Closed_captioning
> >
> > Bryen, do you have any info about how this differs from subtitles? Do
> > you have some sample content that has closed captioning tracks that
> > you could give to the gstreamer developers?
> >
>
> When I was active here back in the day (years ago) I was playing with CC
> code in GStreamer.  I think some of my code may have been lost to time
> though.
>
> At one point I had GStreamer decoding the CC data from DVDs.  It wasn't
> doing full positional on-screen rendering but just dumping the raw text
> to the console.  At the time I was quite thrilled to get that far but
> moved on to other projects.
>
> To get it working I made a specific Bin source on the mpeg demuxer (I
> think?) to output the "user_data" packets.  On a DVD that is how they
> store the VBI data.  I then stole the VBI code from the very excellent
> tvtime project to do the hard work.  I think it's still in the "bad"
> plugins repo in the vbidec plugin.  Then you just route the user_data to
> vbidec and the output of vbidec to a console.  (Or close to that...
> might have been stripping off control codes to just print the text.)  It
> worked just fine but was mostly just a testing hack for GStreamer, DVD
> code, Python bindings, and other stuff... all at once.  Would be easier
> today.
>
> To make this code useful there's lots of work to be done.
>
> - Update vbidec to latest plugin standards and APIs.  I'm sure it's
>  quite bitrotten by now.
> - Look at status of the "user_data" mpeg fields.  I have no idea what it
>  is these days.  I think it was ripped out at some point due to the
>  core not being able to handle multiple bin sources at once properly or
>  something.  Might be easier today or already done.  This is the
>  easiest way to test CC data.  It's a correct VBI bitstream on any DVD
>  with CC data.  Live video signals would work too but might have errors
>  in the VBI data.  Though it's supposed to handle some level of errors.
> - Try pushing V4L streams through vbidec.  I don't recall if I ever
>  tried this.  I also can't remember if you have to strip out the VBI
>  info or if the vbidec code can find that for you.
> - Take the decoded VBI info and do something more useful with it.  There
>  is data in there for display timing, colors, symbols, fonts,
>  positional info... etc etc.  TVTime also supported other type of info.
>  For instance the tv channel/program info (XDS) and the menu system
>  more popular in .eu than .us.
> - Create something to render the CC data.  (Re-use subtitle related
>  work)
> - Create something to composite that rendered data stream on top of a
>  video stream properly.  (Re-use subtitle related work)
> - Hook it all up, integrate into DVD and V4L apps and add some controls.
> - Check with the tvtime.sourceforge.net project to see if they have any
>  newer code.  Also you might need to rip more code to support more than
>  just CC data.  TVTime would be _excellent_ reference code for the
>  non-GStreamer-specific parts of this task.
>
> There may be an issue with the vbidec code being in GStreamer at all.
> It comes from a GPL project.  I'm not sure what the licensing issues are
> these days with that.  Might need to convert bits of tvtime into an
> external library and keep GStreamer specific parts in gst plugins.
>
> This would all make an interesting and useful project.  With example
> code like TVTime out there it's probably not extremely hard either.
>
> I had always tried to convince vektor (Billy Biggs) from tvtime fame to
> just use GStreamer as a base for tvtime but that never happened.  They
> were focused on making the most elite tv app ever and succeeded.  Might
> be possible at this point to port over pieces of it beyond just the CC
> related parts.  There are some issues with GStreamers frame based
> algorithms (I assume it's still like that?) and tvtimes scan-line
> algorithms.  Fun stuff.  Good luck.
>
> -dave
>
> -------------------------------------------------------------------------
>
Well, Dave certainly gave lots of additional information.  :-)  Do you need
anything additional from me?

I do want to make one thing clear.  subtitling and CC are not the same.  CC
uses a black block background, which myself and many in the US prefer
because subtitling uses white font colors which can blend into the media if
there's white or light colors behind it, making the subtitling completely
useless for the moment.

Also, in the US, all broadcast programming is required under the FCC to
broadcast in CC, and all televisions sold in the US are required to have the
built-in CC chip.  This has been so since the early 90's.

DVDs come distributed with either CC capabilities, subtitling capabilities,
or both.  Whenever we have a choice, we typically choose CC.

I did find a reference on this page that claims gstreamer has built in CC
support:
http://www.linuxtv.org/v4lwiki/index.php/Text_capture.

Bryen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20080109/25fb117c/attachment.htm>