[gst-devel] [Telepathy] OLPC Open Video Chat Announcement

[Sjoerd Simons]

On Thu, Jun 24, 2010 at 04:18:13PM -0400, Taylor Rose (RIT Student) wrote:
> The Open Video Chat project started in March 2010. The primary goal was to
> develop a wholly Free/Open Source video chat program using an unencumbered
> video codec, Theora. We also needed to make the program run at 15 FPS or
> better in order to provide smooth enough video to communicate with American
> Sign Language.
>
> Since reaching our 15+ FPS goal we have been working to improve the program
> and add features.

Congrats on reaching your goal there!

> Except for some minor tweaks our attempts have consistently been blocked by a
> bug with RTP.  Right now we do not have a reliable way to ensure that the
> header packets will reach their destination.  TCP would ensure packet
> delivery but would bog down the XO's too much.  Our current system with UDP
> is too unreliable.  If we were to stop the stream to change settings there is
> no guarantee that the stream will successfully connect again.  RTP would
> provide the best of both worlds.  It would ensure the header packets get
> through like TCP but then provide the streaming speed of UDP.  Unfortunately
> we can't get RTP working with open codecs.

Actually RTP doesn't solve your issue here. RTP is just a way of packetizing
media that (normally) gets send over UDP. What you actually want in the case of
codecs like Theora is to send your configuration data both over the signalling
channel and over RTP. Like is already done in XMPP and SIP for example...

Now configuration data isn't everything, if you make sure it is reliably
transmitted you can at least decode all frames, but if you miss key frames the
output will still look bad (and might not be clear enough for signing) untill
you get the next key frame. One thing you can do with RTCP is to request the
sender to re-sent a keyframe to minimize that issue.

Another thing you probably want to do is monitor network conditions (by using
RTCP) and scale you bandwidth usage according to that to minimize packet loss
and to give a general better experience on lower bandwidth links.

We're currently working on improving farsight to support the last two items
(requesting keyframes and what is called tcp friendly rate control)

> Currently there is a bug in Gstreamer that does not support passing
> configuration data inband with RTP and Theora
> (link<https://bugzilla.gnome.org/show_bug.cgi?id=574416>).

I'm slightly confused why you point to this bug. As you made clear above that
you don't want to pass configuration data purely inband for reliability
reasons, which is what this bug is actually abouti.. Also it has been fixed
almost two months ago?

> Our other option is to use Telepathy-Farsight but this has also given us
> problems.  We have been able to get Farsight to work inconsistently with
> closed codecs but we could not get it to work at all with Theora which is a
> requirement of our project.

Given that i do most if not all of my testing and work using telepathy,
farsight and theora, i'm pretty sure this works nicely.

Earlier this year we did some changes in both farsight and Gstreamer which
caused issues when combining certain versions of the two. Ofcourse if you had
raised the issue you were seeing with us at the time we could have helped you
out there...

> So at this point we need help from the upstream.  Without either the
> Gstreamer bug fixed or some help using Farsight we are at an impasse.  We
> would appreciate any help from upstream community members.  Feel free to
> pull our repo and check out our code.  If you're unfamiliar with the XO's
> you should know that the code requires the Sugar environment to run.

So at this point i need to go of on a bit of a rant. I did check out your code
and it looks like "My first video conferencing application", which is fair
enough really as for most if not all of you it will be your first video
conferencing application.

It is sad though that a project that afaik is partially meant to teach how to
do open source development seems to have completely failed at actually using
open source properly.

The power of FOSS is partially that you can stand on the shoulders of giants
and that you can work together to move the state of the art forward in
such a way that it benefits all parties. As an added benefit, in most active
projects there are always some people that are happy to help you with the
initial learning curve and/or issues you're having.

What i've seen from this project over the past few months is someone once or
twice appearing on our IRC channel, scattergunning a load of questions, getting
some help and then disappearing again. Which doesn't seem like a great way of
engaging an open source project... To make matters worse at the end of march i
sent an e-mail to some of your projects leads, asking how things were going and
if any mentoring or other help was needed, to which i've never got any reply..
Seems like they missed a much-needed oppurtunity there.

Judging from looking at your code and the experiences above, your project
worked in the same way a lot of companies that don't actually understand open
source work.  You grabbed various open source bits that were available and that
you could get to work without spending too much effort on them, wrote
your program in somewhat of a vacuum, slapped a nice open source license on it
and said: look, look, we did open source isn't it amazing...

What you seem to have ended up with is a stepping stone that allowed you to
look over the fence that was directly in front of you, only to see there is a
big and dense jungle on the other side blocking your path. While you could
have been part of an expedition with many years of combined experience on how
to get through such a jungle...

> We have begun planning an OVC/Telepathy hackfest at OLPC headquarters in
> Boston the week of July 12th.  We'll send out a mailing list blast once
> we've secured a time and place.  More information can be found at
> http://foss.rit.edu/events/ovchackfest

That sounds cool and hopefully you'll get some useful things out of it. I
assume you're going to make sure you've got at least one or more people there
who are quite knowledgable about Telepathy/Farsight/Gstreamer and video
conferencing in general as those seem to be the areas you do need help with.

  Sjoerd
-- 
And ever has it been known that love knows not its own depth until the
hour of separation.
		-- Kahlil Gibran