[Telepathy] Issues with the StreamedMedia channel type interface

[George Kiagiadakis]

Hello folks,

As you may already know, I am a gsoc student working on a voip application for 
KDE, kcall, which uses the telepathy framework. While working on this project, 
I am studying a lot the telepathy specification, various library docs and 
examples in order to understand how to build my application. My impression so 
far is that the whole multimedia spec is quite complicated and I am writing 
this email to discuss various issues with it.

Let's start from the StreamedMedia channel type spec. This interface provides 
several "streams". Each stream seems to be associated with a contact handle, 
since when you request a stream, you pass the contact handle as an argument 
and when a stream is added, the signal that notifies you takes a contact handle 
as an argument too. However, the spec says that contact handle may be 0 if the 
stream contains multiple participants. So, confusion starts right here: does 
the channel always only have one stream per type (i.e. one audio and one video 
stream) and each stream can be associated with multiple participants? or do 
multiple streams exist, one for each combination of participant and type (with 
maximum 2 streams per participant, one for audio and one for video)? This is 
kind of analogous to the difference between a farsight session and a farsight 
stream in the farsight2 API.

Let's see another problem now. Each stream has a state (Disconnected, 
Connecting, Connected). When I was starting writing kcall, I assumed that if 
the remote end has not answered the call, the stream would be in "Connecting" 
state. However, this is not the case. The stream changes directly from 
"Disconnected" to "Connected". While I was trying to figure out how to know 
that the remote end has not answered the call, I found that there are many 
different ways to do it. I first looked at the call example from the telepathy-
qt4 library. This example assumes that the remote end has answered when the 
"src-pad-added" signal is sent from the telepathy-farsight stream. Of course 
this example assumes that there is only one participant and one stream. I also 
looked into empathy, which seems to be doing the same thing!! Then I took a 
closer look at the spec. There are two ways to get this information from 
there. First, from the Group interface, if the other participant is in the 
remote pending group, then he hasn't answered yet. Second, from the CallState 
interface, there is a "Ringing" state available per participant (or so it 
seems). I ended up using the group interface, but in a way that I think it 
doesn't support multiple participants. The code looks like this:
------
- Initially set the internal state to "Connecting".
- When you receive the StreamStateChanged signal, if the internal state is 
"Connecting" and the stream has changed to "Connected" and there are 
participants in the remote pending members, set internal state to "Ringing".
- When you receive the GroupMembersChanged signal, if the internal state is 
"Ringing" and there are no more people in the remote pending members, set the 
internal state to "Connected" and now the call is established.
-------
I am not sure how to do this more properly. It seems correct, but complicated 
and I am not sure if that would work in a channel with multiple participants 
or multiple streams, mostly because I am not sure about the order in which 
signals from different interfaces will come.

So, I believe some changes should be made to the spec to address these issues. 
In my opinion the spec should specify something like this:
- Each stream should be associated with only one contact.
- Each stream should have only one direction (send or receive) and type (audio 
or video).
- Streams that have "send" direction should be associated with the selfcontact 
(i.e. myself), so that there can be only two "send" streams, one for audio and 
one for video.
- Each stream should specify more states, for example "Ringing" should be a 
possible stream state. If a "receive" stream is in "Ringing" state, that means 
that the remote contact has not accepted the stream. If a "send" stream is in 
"Ringing" state, that means that I have not accepted the stream, and this 
should happen in cases where for example I have an audio conversation and the 
remote contact requests video from me. Then the StreamAdded signal would be 
emitted with this stream, to allow me accept/decline the request. This would 
allow the msn-style "This contact has requested to view your webcam. 
Accept/decline". Doing this with streams that have multiple directions seems 
much more complicated.
- Eventually, the "StreamDirectionChanged" signal should be taken out, it's 
complicated and useless if each stream can only have one direction.
- Possibly, all the states in the CallState interface could be states of the 
stream itself. I don't see why they are in a separate interface. If a 
connection manager doesn't support some of them, it will simply never put the 
stream in the states that it doesn't support.

All this would make it much easier to work with. For example, I would be able 
to know if I am sending a specific type of stream without having to search 
through all the streams to find those that match this type and that have a 
"sending" direction. I would be able to stop sending a specific type of stream 
by just removing my "send" stream for this type instead of requesting 
direction change for all the streams that have a "sending" direction (have you 
ever imagined what could happen in case where one of the direction changes in 
this case fails with an error? I would be left sending only to one person, and 
it's not really easy to handle that in the ui so that the user can understand 
what is going on. Ok, you may say that this is not possible to happen, but the 
spec perfectly allows it from my pov).

I'll stop here because this email is becoming too long. What do you think with 
all this? Do you have any answers to my problems? Do you find my proposal good?

Best regards,
George