[SyncEvolution] SyncEvolution in Fremantle

[Ove Kaaven]

Patrick Ohly skrev:
> This is becoming more SyncEvolution specific. Should we continue
> cross-posting to maemo-developers?

Dunno, I wasn't subscribed to the syncevolution list yet. I suppose I
should subscribe to it. But I think some Maemo developers may have some
interest in the progress of this anyway?

> On Sat, 2009-12-26 at 13:53 +0000, Ove Kaaven wrote:
>> Patrick Ohly skrev:
>>> I was planning to do a 0.9.2 update release in a few weeks. Should we
>>> merge your code and include the Maemo support in the release
>>> announcement?
>> If you want.
> 
> It's your call. Given that you raised some questions below about
> improvements which cannot be done in backwards-compatible way between
> releases of the calendar backend (change tracking!), it might be better
> to give it some more time and release with 1.0 (tentative goal: end of
> March).

Well, I think many people could benefit from synchronization before
March, even if an incompatible change is introduced later. Is then the
best way to just keep my "unofficial" builds available, without upstream
integration before 1.0?

>> It's not clear from the website whether this "Linux Foundation license
>> agreement" is really to be sent to the moblin guys, or to some other
>> address.
> 
> My reading is that it needs to go to the Linux Foundation. I'll check.

OK. I'm not sure what's up with having to sign "Linux Foundation Moblin
Workgroup Individual or Corporate Contribute License Agreement v1.0"
(phew) if SyncEvolution isn't a Moblin project. Perhaps I'll just stick
with the other options for now.

> I was hoping that TrackingSyncSource.h had enough information to be
> useful, but you are right, it doesn't explain the big picture. A more
> general introduction of "source", "item", etc. would be needed. Time to
> resurrect Doxygen and write some additional pages, I suppose. Would that
> have helped?

Yes, probably.

> IMHO *any* framework is difficult to start with, regardless of the
> language :-/

Well, I've usually found it fairly easy to get into frameworks written
in C or Python, but not C++. Not sure if that's just coincidence...

>> - I'm not sure how to properly write those integration tests in the
>> *Register.cpp
> 
> I can add those when merging the code.

OK.

>> - Do I need to worry about getSupportedTypes() or anything
> 
> You only need to implement the pure virtual methods, everything else has
> reasonable defaults.

Well, I just imagine "reasonable default" is not always "perfect" or
"efficient" or anything...

> getSupportedTypes() is legacy code which was inherited from the Funambol
> library. It became obsolete when moving to libsynthesis and is already
> removed on "master" (well, almost - just found a copy of it in a derived
> class). Funambol required that sources deal with data conversion
> themselves, whereas with Synthesis this is done by the engine.

Allright.

>> - The Maemo's calendar-backend can return entries that have changed
>> after a particular date (typically you'd use the time of the previous
>> sync). Is there a way to use this to improve on the TrackingSyncSource
>> method, so it won't be necessary to load and generate revision strings
>> for the whole database every time?
> 
> As Congwu said, this can be done by inheriting from SyncSource directly
> and adding the SyncSource* building blocks that you want to reuse. You
> can use the TrackingSyncSource as an example how that is done.

Well, as an example, it's really not very enlightening. Its
implementation has no explanation. What is all this multiple-inheritance
mess really doing? How many (and which) virtual base classes would I
have to derive from to implement a more efficient change tracker, with
the least amount of code? Would I have to spend a month digging into the
internals of every single class of this huge forest of
multiple-inheriting classes and take notes about how they work and
interact, before I'd be able to write code that does something as simple
as considering a known subset of the database as potentially modified
since last sync, instead of checking the whole database? (The reasons I
stopped using C++ are all coming back to me now...)

> The "time of last sync" is something that could be stored either in an
> internal source property or (better) in the SyncML anchor string,
> handled by the Synthesis engine. However, this will require changes to
> the Synthesis/SyncEvolution and SyncSource interfaces. We have to work
> on this anyway for resume support in the SyncML server, so my suggestion
> is to address this in 1.0 like this:
>       * make those API changes
>       * create a new DateSyncSource which requires the following
>         information from derived classes:
>               * complete list of all local item IDs
>               * list of changed item IDs since a certain time
>       * change FileSyncSource so that it is derived from DateSyncSource

Works for me.

> Do you have easy access to all IDs of existing items? This is necessary
> to detect deleted items.

Sure, a full list of IDs can be queried easily, with minimal overhead
(unlike their associated last-modified-times). I'm not totally sure
whether deleted IDs are included or not, though, might have to test.
Deleted IDs are kept in a Trash table along with their deletion times,
so that it's possible to query IDs of all deleted times after a certain
time (last sync) with a simple API call. I'm also not sure when this
Trash table is (or, depending on whose responsibility it is, should be)
purged.

>> - The Maemo addressbook (which is still ebook-based), as well as the
>> calendar (which has APIs to convert to vcal 1.0 and ical 2.0),
> 
> Do you have a pointer to these APIs and perhaps the underlying source
> code? I'm curious how the vCalendar 1.0 support is done.

It's Nokia closed source, and not very well documented, but you can see
its documentation here:
http://maemo.org/api_refs/5.0/5.0-final/calendar-backend/annotated.html

(Most of the actual sync logic would have to interact with CCalendar,
while opening and creating calendars go through CMulticalendar.)

It basically does it much like ebook: it represents events and stuff
natively in C++ classes. Then you've got some API (ICalConverter) that
can serialize it to/from vcal/ical strings, and you tell it what format
you want.

You can also look at the backend I already wrote, I suppose. It's in the
tarball in the same place I put the Maemo debs:
http://people.debian.org/~ovek/maemo/

>>  stores
>> some non-standard fields. I noticed something on the SyncEvolution
>> webpage about mimeprofiles to specify what fields are stored locally. Is
>> there a way to specify that, so that these fields are not destroyed on
>> the Maemo device when syncing with a server that doesn't support them,
>> even though the backends do convert from/to the standard vcard and ical
>> formats?
> 
> This is already possible, but we aren't sure yet how to maintain the
> different profiles. Right now, src/syncclient_sample_config.xml contains
> a "contacts" field list (internal representation) and "vCard" profile
> that is used both for the SyncML peer and the local backends, with some
> tweaks to let some properties be handled differently on both sides
> ("EVOLUTION" rule).

Hmm, okay... I know that for me, the addressbook field X-SKYPE tend to
get lost when syncing (and I think I've also had PHOTO and maybe BDAY
get lost, for some reason, but that may have other reasons). Anything
else I can think of seem to already be listed in there. Perhaps I'll
just need to add fields for Skype, SIP, Ovi, and such stuff then. I
guess I just add Skype like the IM fields already listed in there?

> If you are interested, then I suggest that you modify the .xml config
> directly without caring about the effect on other backends. This is
> something that we'll sort out when merging your changes.

OK, can try...