From ovek at arcticnet.no Sun Dec 20 20:05:12 2009 From: ovek at arcticnet.no (Ove Kaaven) Date: Sun, 20 Dec 2009 21:05:12 +0100 Subject: SyncEvolution in Fremantle In-Reply-To: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> Message-ID: <4B2E8378.7010909@arcticnet.no> Hi, Patrick. I'm new to Maemo, but I've been a Debian Developer for a long time. I recently got a N900, and decided I really want to sync my stuff. I've managed to build the current syncevolution git on Maemo5 (in Scratchbox), however it did not build cleanly without a few changes (and I disabled shared libraries since cppunit was not available). Also when built with optimization it crashes immediately, but appears to at least start OK when built without optimization. I used the included debian/rules with a couple of extra environment variables set. (I'll allow anyone desperate enough to get the .deb I ended up with last night at http://www.ping.uio.no/~ovehk/maemo/ if they really, really, want, but it's obviously not anywhere near end-user-ready, there's no GUI, it's built without optimization, it seems to include flashmemory-space-wasting .h and .a files, and I disabled regular expressions.) Since N900's calendar application no longer uses the evolution backend (but something Nokia-specific, I guess - some C++ API on top of a sqlite database), I guess I may have to write a brand new backend in order to sync the calendar. N900's addressbook still uses the evolution backend, though. I managed to sync the addressbook in scratchbox, haven't tried the actual device yet. Patrick Ohly skrev: > I'd love to see the latest SyncEvolution releases packaged properly for > Maemo, and so do users [6]. 0.8.1 still works fine on the older Maemo > releases it is available for, but 0.9 has several relevant improvements, > for example synchronization with Google Contacts and a GTK GUI. Hmm. If someone also made direct sync with Google Calendar, then it would be really useful... > I'm posting here because I hope that an interested developer or > maintainer will step up and take over packaging for Maemo. You can be > sure that this will have full support when it comes to merging patches > and including the Maemo port as first-class citizen in releases. I don't suppose anyone else started working on this yet? If not, do you have any recommendations on where to start? What version of the software would it be best/easiest to try packaging, for instance, or is git head "stable" enough? Ove From david at dgreaves.com Sun Dec 20 23:58:04 2009 From: david at dgreaves.com (David Greaves) Date: Sun, 20 Dec 2009 23:58:04 +0000 Subject: SyncEvolution in Fremantle In-Reply-To: <4B2E8378.7010909@arcticnet.no> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> Message-ID: <4B2EBA0C.8030807@dgreaves.com> Ove Kaaven wrote: > Hi, Patrick. > > I'm new to Maemo, but I've been a Debian Developer for a long time. I > recently got a N900, and decided I really want to sync my stuff. Hi Ove, welcome to maemo-dev :) Just breaking lurker status on this thread and saying that I'm really pleased that someone's making an effort - if you could push your git tree to gitorious.org (maemo's de-facto git-sharing service) that would be really nice even though I realise it is likely to be ugly atm. Also maybe start scribbling on the maemo.org wiki too? Cheers David/lbt -- "Don't worry, you'll be fine; I saw it work in a cartoon once..." From congwu.chen at intel.com Mon Dec 21 01:50:15 2009 From: congwu.chen at intel.com (Chen, Congwu) Date: Mon, 21 Dec 2009 09:50:15 +0800 Subject: [SyncEvolution] SyncEvolution in Fremantle In-Reply-To: <4B2E8378.7010909@arcticnet.no> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> Message-ID: Hello Ove! Patrick is on vocation these days so I will try to answer your question first. Ove Kaaven wrote: > >Hi, Patrick. > >I'm new to Maemo, but I've been a Debian Developer for a long time. I >recently got a N900, and decided I really want to sync my stuff. > >I've managed to build the current syncevolution git on Maemo5 (in >Scratchbox), however it did not build cleanly without a few changes (and >I disabled shared libraries since cppunit was not available). Also when >built with optimization it crashes immediately, but appears to at least >start OK when built without optimization. I used the included >debian/rules with a couple of extra environment variables set. > >(I'll allow anyone desperate enough to get the .deb I ended up with last >night at http://www.ping.uio.no/~ovehk/maemo/ if they really, really, >want, but it's obviously not anywhere near end-user-ready, there's no >GUI, it's built without optimization, it seems to include >flashmemory-space-wasting .h and .a files, and I disabled regular >expressions.) > >Since N900's calendar application no longer uses the evolution backend >(but something Nokia-specific, I guess - some C++ API on top of a sqlite >database), I guess I may have to write a brand new backend in order to >sync the calendar. N900's addressbook still uses the evolution backend, >though. I managed to sync the addressbook in scratchbox, haven't tried >the actual device yet. Great progress! Regarding the calendar backend, you are right. We need to write a new backend to support it. We have a bug entry tracking this issue [1], however we have no resource to work on it at this time. If you can come up and take it, that will be great. >Patrick Ohly skrev: >> I'd love to see the latest SyncEvolution releases packaged properly for >> Maemo, and so do users [6]. 0.8.1 still works fine on the older Maemo >> releases it is available for, but 0.9 has several relevant improvements, >> for example synchronization with Google Contacts and a GTK GUI. > >Hmm. If someone also made direct sync with Google Calendar, then it >would be really useful... That's not possible with SyncML protocol, webDav(which is not really sync) and ActiveSync may works, but SyncEvolution currently does not support either. >> I'm posting here because I hope that an interested developer or >> maintainer will step up and take over packaging for Maemo. You can be >> sure that this will have full support when it comes to merging patches >> and including the Maemo port as first-class citizen in releases. > >I don't suppose anyone else started working on this yet? If not, do you >have any recommendations on where to start? What version of the software >would it be best/easiest to try packaging, for instance, or is git head >"stable" enough? SyncEvolution 0.9.1 is the latest stable release, that will be a good version to start. There are heavy developments towards 1.0 release so I will not recommend git head at the moment. [1] http://bugzilla.moblin.org/show_bug.cgi?id=8511 [2] http://git.moblin.org/cgit.cgi/syncevolution/tag/?id=syncevolution-0-9-1 Best Regards, Congwu From ovek at arcticnet.no Wed Dec 23 05:24:35 2009 From: ovek at arcticnet.no (Ove Kaaven) Date: Wed, 23 Dec 2009 06:24:35 +0100 Subject: [SyncEvolution] SyncEvolution in Fremantle In-Reply-To: References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> Message-ID: <4B31A993.6080102@arcticnet.no> Chen, Congwu skrev: > Great progress! Regarding the calendar backend, you are right. We need > to write a new backend to support it. We have a bug entry tracking this issue > [1], however we have no resource to work on it at this time. If you can come > up and take it, that will be great. Yeah, I'll have a go at it soon. (By the way, I did successfully sync the addressbook on the actual device using that .deb I posted a link to, so now it's time to debug the crash with optimization enabled, and start coding what's left to be done, I guess...) > That's not possible with SyncML protocol, webDav(which is not really sync) and > ActiveSync may works, but SyncEvolution currently does not support either. Oh well. Best to stick with ScheduleWorld and/or GooSync, then... > SyncEvolution 0.9.1 is the latest stable release, that will be a good version to start. > There are heavy developments towards 1.0 release so I will not recommend git head > at the moment. What about the libsynthesis? Is there any recommended tag for that? David Greaves skrev: > Hi Ove, welcome to maemo-dev :) Thanks. > Just breaking lurker status on this thread and saying that I'm really > pleased that someone's making an effort - if you could push your git > tree to gitorious.org (maemo's de-facto git-sharing service) that > would be really nice even though I realise it is likely to be ugly > atm. Hmm. It was my impression that Patrick wanted Maemo patches to go into syncevolution's own git; if so, it would probably be fairly meaningless to also push it to another git repository. > Also maybe start scribbling on the maemo.org wiki too? What would I write? From congwu.chen at intel.com Wed Dec 23 05:45:55 2009 From: congwu.chen at intel.com (Chen, Congwu) Date: Wed, 23 Dec 2009 13:45:55 +0800 Subject: [SyncEvolution] SyncEvolution in Fremantle In-Reply-To: <4B31A993.6080102@arcticnet.no> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> <4B31A993.6080102@arcticnet.no> Message-ID: Ove Kaaven wrote: >Chen, Congwu skrev: >> SyncEvolution 0.9.1 is the latest stable release, that will be a good version to >start. > >What about the libsynthesis? Is there any recommended tag for that? > There is also a 0.9.1 tag in libsynthesis repo hosted on moblin.org, see http://git.moblin.org/cgit.cgi/libsynthesis/tag/?id=libsynthesis_3.2.0.35%2bsyncevolution-0-9-1 -- Best Regards, Congwu From ovek at arcticnet.no Wed Dec 30 06:42:33 2009 From: ovek at arcticnet.no (Ove Kaaven) Date: Wed, 30 Dec 2009 07:42:33 +0100 Subject: [SyncEvolution] SyncEvolution in Fremantle In-Reply-To: <1262091676.3705.158.camel@pohly-mobl1.ikn.intel.com> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> <4B332042.3080509@arcticnet.no> <1261822284.3705.62.camel@pohly-mobl1.ikn.intel.com> <4B361557.6040106@arcticnet.no> <1262091676.3705.158.camel@pohly-mobl1.ikn.intel.com> Message-ID: <4B3AF659.1090603@arcticnet.no> Patrick Ohly skrev: > This is becoming more SyncEvolution specific. Should we continue > cross-posting to maemo-developers? Dunno, I wasn't subscribed to the syncevolution list yet. I suppose I should subscribe to it. But I think some Maemo developers may have some interest in the progress of this anyway? > On Sat, 2009-12-26 at 13:53 +0000, Ove Kaaven wrote: >> Patrick Ohly skrev: >>> I was planning to do a 0.9.2 update release in a few weeks. Should we >>> merge your code and include the Maemo support in the release >>> announcement? >> If you want. > > It's your call. Given that you raised some questions below about > improvements which cannot be done in backwards-compatible way between > releases of the calendar backend (change tracking!), it might be better > to give it some more time and release with 1.0 (tentative goal: end of > March). Well, I think many people could benefit from synchronization before March, even if an incompatible change is introduced later. Is then the best way to just keep my "unofficial" builds available, without upstream integration before 1.0? >> It's not clear from the website whether this "Linux Foundation license >> agreement" is really to be sent to the moblin guys, or to some other >> address. > > My reading is that it needs to go to the Linux Foundation. I'll check. OK. I'm not sure what's up with having to sign "Linux Foundation Moblin Workgroup Individual or Corporate Contribute License Agreement v1.0" (phew) if SyncEvolution isn't a Moblin project. Perhaps I'll just stick with the other options for now. > I was hoping that TrackingSyncSource.h had enough information to be > useful, but you are right, it doesn't explain the big picture. A more > general introduction of "source", "item", etc. would be needed. Time to > resurrect Doxygen and write some additional pages, I suppose. Would that > have helped? Yes, probably. > IMHO *any* framework is difficult to start with, regardless of the > language :-/ Well, I've usually found it fairly easy to get into frameworks written in C or Python, but not C++. Not sure if that's just coincidence... >> - I'm not sure how to properly write those integration tests in the >> *Register.cpp > > I can add those when merging the code. OK. >> - Do I need to worry about getSupportedTypes() or anything > > You only need to implement the pure virtual methods, everything else has > reasonable defaults. Well, I just imagine "reasonable default" is not always "perfect" or "efficient" or anything... > getSupportedTypes() is legacy code which was inherited from the Funambol > library. It became obsolete when moving to libsynthesis and is already > removed on "master" (well, almost - just found a copy of it in a derived > class). Funambol required that sources deal with data conversion > themselves, whereas with Synthesis this is done by the engine. Allright. >> - The Maemo's calendar-backend can return entries that have changed >> after a particular date (typically you'd use the time of the previous >> sync). Is there a way to use this to improve on the TrackingSyncSource >> method, so it won't be necessary to load and generate revision strings >> for the whole database every time? > > As Congwu said, this can be done by inheriting from SyncSource directly > and adding the SyncSource* building blocks that you want to reuse. You > can use the TrackingSyncSource as an example how that is done. Well, as an example, it's really not very enlightening. Its implementation has no explanation. What is all this multiple-inheritance mess really doing? How many (and which) virtual base classes would I have to derive from to implement a more efficient change tracker, with the least amount of code? Would I have to spend a month digging into the internals of every single class of this huge forest of multiple-inheriting classes and take notes about how they work and interact, before I'd be able to write code that does something as simple as considering a known subset of the database as potentially modified since last sync, instead of checking the whole database? (The reasons I stopped using C++ are all coming back to me now...) > The "time of last sync" is something that could be stored either in an > internal source property or (better) in the SyncML anchor string, > handled by the Synthesis engine. However, this will require changes to > the Synthesis/SyncEvolution and SyncSource interfaces. We have to work > on this anyway for resume support in the SyncML server, so my suggestion > is to address this in 1.0 like this: > * make those API changes > * create a new DateSyncSource which requires the following > information from derived classes: > * complete list of all local item IDs > * list of changed item IDs since a certain time > * change FileSyncSource so that it is derived from DateSyncSource Works for me. > Do you have easy access to all IDs of existing items? This is necessary > to detect deleted items. Sure, a full list of IDs can be queried easily, with minimal overhead (unlike their associated last-modified-times). I'm not totally sure whether deleted IDs are included or not, though, might have to test. Deleted IDs are kept in a Trash table along with their deletion times, so that it's possible to query IDs of all deleted times after a certain time (last sync) with a simple API call. I'm also not sure when this Trash table is (or, depending on whose responsibility it is, should be) purged. >> - The Maemo addressbook (which is still ebook-based), as well as the >> calendar (which has APIs to convert to vcal 1.0 and ical 2.0), > > Do you have a pointer to these APIs and perhaps the underlying source > code? I'm curious how the vCalendar 1.0 support is done. It's Nokia closed source, and not very well documented, but you can see its documentation here: http://maemo.org/api_refs/5.0/5.0-final/calendar-backend/annotated.html (Most of the actual sync logic would have to interact with CCalendar, while opening and creating calendars go through CMulticalendar.) It basically does it much like ebook: it represents events and stuff natively in C++ classes. Then you've got some API (ICalConverter) that can serialize it to/from vcal/ical strings, and you tell it what format you want. You can also look at the backend I already wrote, I suppose. It's in the tarball in the same place I put the Maemo debs: http://people.debian.org/~ovek/maemo/ >> stores >> some non-standard fields. I noticed something on the SyncEvolution >> webpage about mimeprofiles to specify what fields are stored locally. Is >> there a way to specify that, so that these fields are not destroyed on >> the Maemo device when syncing with a server that doesn't support them, >> even though the backends do convert from/to the standard vcard and ical >> formats? > > This is already possible, but we aren't sure yet how to maintain the > different profiles. Right now, src/syncclient_sample_config.xml contains > a "contacts" field list (internal representation) and "vCard" profile > that is used both for the SyncML peer and the local backends, with some > tweaks to let some properties be handled differently on both sides > ("EVOLUTION" rule). Hmm, okay... I know that for me, the addressbook field X-SKYPE tend to get lost when syncing (and I think I've also had PHOTO and maybe BDAY get lost, for some reason, but that may have other reasons). Anything else I can think of seem to already be listed in there. Perhaps I'll just need to add fields for Skype, SIP, Ovi, and such stuff then. I guess I just add Skype like the IM fields already listed in there? > If you are interested, then I suggest that you modify the .xml config > directly without caring about the effect on other backends. This is > something that we'll sort out when merging your changes. OK, can try... From ovek at arcticnet.no Wed Dec 30 07:14:19 2009 From: ovek at arcticnet.no (Ove Kaaven) Date: Wed, 30 Dec 2009 08:14:19 +0100 Subject: [SyncEvolution] SyncEvolution in Fremantle In-Reply-To: <4B3AF659.1090603@arcticnet.no> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> <4B332042.3080509@arcticnet.no> <1261822284.3705.62.camel@pohly-mobl1.ikn.intel.com> <4B361557.6040106@arcticnet.no> <1262091676.3705.158.camel@pohly-mobl1.ikn.intel.com> <4B3AF659.1090603@arcticnet.no> Message-ID: <4B3AFDCB.4040709@arcticnet.no> Ove Kaaven skrev: >> This is already possible, but we aren't sure yet how to maintain the >> different profiles. Right now, src/syncclient_sample_config.xml contains >> a "contacts" field list (internal representation) and "vCard" profile >> that is used both for the SyncML peer and the local backends, with some >> tweaks to let some properties be handled differently on both sides >> ("EVOLUTION" rule). I found a case I'm not sure about. If you add a Jabber address to a contact, the vcard looks like: X-JABBER;TYPE=jabber:address but if you add a "Ovi by Nokia" address, the vcard looks like: X-JABBER;TYPE=nokiachat:address (If you add both, they not unexpectedly become separate entries, on separate lines.) Is there a good way to model that? From patrick.ohly at gmx.de Wed Dec 30 10:33:27 2009 From: patrick.ohly at gmx.de (Patrick Ohly) Date: Wed, 30 Dec 2009 11:33:27 +0100 Subject: [SyncEvolution] SyncEvolution in Fremantle In-Reply-To: <4B3AF659.1090603@arcticnet.no> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> <4B332042.3080509@arcticnet.no> <1261822284.3705.62.camel@pohly-mobl1.ikn.intel.com> <4B361557.6040106@arcticnet.no> <1262091676.3705.158.camel@pohly-mobl1.ikn.intel.com> <4B3AF659.1090603@arcticnet.no> Message-ID: <1262169205.3705.198.camel@pohly-mobl1.ikn.intel.com> On Wed, 2009-12-30 at 06:42 +0000, Ove Kaaven wrote: > Patrick Ohly skrev: > > This is becoming more SyncEvolution specific. Should we continue > > cross-posting to maemo-developers? > > Dunno, I wasn't subscribed to the syncevolution list yet. I suppose I > should subscribe to it. But I think some Maemo developers may have some > interest in the progress of this anyway? Might be; so let's continue cross-posting until someone objects. > > On Sat, 2009-12-26 at 13:53 +0000, Ove Kaaven wrote: > >> Patrick Ohly skrev: > >>> I was planning to do a 0.9.2 update release in a few weeks. Should we > >>> merge your code and include the Maemo support in the release > >>> announcement? > >> If you want. > > > > It's your call. Given that you raised some questions below about > > improvements which cannot be done in backwards-compatible way between > > releases of the calendar backend (change tracking!), it might be better > > to give it some more time and release with 1.0 (tentative goal: end of > > March). > > Well, I think many people could benefit from synchronization before > March, even if an incompatible change is introduced later. Is then the > best way to just keep my "unofficial" builds available, without upstream > integration before 1.0? I'm fine either way. I'm still on vacation until January 11th, then I'll have a closer look at the current state. In the meantime, let's continue the discussion and perhaps see what users think about it. > >> It's not clear from the website whether this "Linux Foundation license > >> agreement" is really to be sent to the moblin guys, or to some other > >> address. > > > > My reading is that it needs to go to the Linux Foundation. I'll check. > > OK. I'm not sure what's up with having to sign "Linux Foundation Moblin > Workgroup Individual or Corporate Contribute License Agreement v1.0" > (phew) if SyncEvolution isn't a Moblin project. Perhaps I'll just stick > with the other options for now. There's no "SyncEvolution Foundation". Instead of creating one it seemed easier to just use what was already established, even if it is not a perfect fit. > >> - Do I need to worry about getSupportedTypes() or anything > > > > You only need to implement the pure virtual methods, everything else has > > reasonable defaults. > > Well, I just imagine "reasonable default" is not always "perfect" or > "efficient" or anything... That's something that should be pointed out during a code review. I appreciate your diligence, though ;-} > >> - The Maemo's calendar-backend can return entries that have changed > >> after a particular date (typically you'd use the time of the previous > >> sync). Is there a way to use this to improve on the TrackingSyncSource > >> method, so it won't be necessary to load and generate revision strings > >> for the whole database every time? > > > > As Congwu said, this can be done by inheriting from SyncSource directly > > and adding the SyncSource* building blocks that you want to reuse. You > > can use the TrackingSyncSource as an example how that is done. > > Well, as an example, it's really not very enlightening. Its > implementation has no explanation. What is all this multiple-inheritance > mess really doing? First, avoiding code duplication. A sync source must implement multiple different, orthogonal aspects (change tracking, item access, backup/restore, ...). There are multiple different possible implementations for each, which must be combined somehow without copying the source code. Second, multiple-inheritance is used to introduce pure virtual methods. Derive from TrackingSyncSource and once your code compiles, you know that you have provided all the necessary custom code. I agree that the way how this was done in C++ is not particularly nice. I couldn't come up with a more elegant approach despite thinking about this quite a bit, so you may have a point about C++ making it worse than in other languages. > How many (and which) virtual base classes would I > have to derive from to implement a more efficient change tracker, with > the least amount of code? You need to understand the SyncSource base class and what functionality it can and should provide. Then you need to know which existing implementations of that functionality exist. An overview page in Doxygen with links to more detailed API descriptions would be useful here, but doesn't exist yet. > Would I have to spend a month digging into the > internals of every single class of this huge forest of > multiple-inheriting classes and take notes about how they work and > interact, before I'd be able to write code that does something as simple > as considering a known subset of the database as potentially modified > since last sync, instead of checking the whole database? No. You would need to understand only one part, the one currently implemented by SyncSourceRevisions. > > The "time of last sync" is something that could be stored either in an > > internal source property or (better) in the SyncML anchor string, > > handled by the Synthesis engine. However, this will require changes to > > the Synthesis/SyncEvolution and SyncSource interfaces. We have to work > > on this anyway for resume support in the SyncML server, so my suggestion > > is to address this in 1.0 like this: > > * make those API changes > > * create a new DateSyncSource which requires the following > > information from derived classes: > > * complete list of all local item IDs > > * list of changed item IDs since a certain time > > * change FileSyncSource so that it is derived from DateSyncSource > > Works for me. Looking at the code again and in particular SyncSourceRevisions::updateRevision(), I remembered one problem with the "changes since last sync" approach of change tracking. Suppose the following chain of events: 1. sync starts at time T1, with no local changes 2. a changed item A is sent by the SyncML server and updated locally 3. the user edits a different item B while the sync runs 4. sync ends at time T2 Now in the next sync, which time stamp should be used to find changed items that need to be sent to the server? When using T1, the changed item A would be sent to the server, which is wrong (additional traffic, unnecessary risk of conflicts on the server). When using T2, the locally modified item B would not be sent. Updates of the database would have to be disabled while a sync runs to avoid this problem. Is that possible? The SyncSourceRevisions class allows concurrent editing by distinguishing changes made by SyncEvolution and others. It's not perfect at the moment, because an API change in EDS would be needed to avoid race conditions - we have patches pending for that. > > Do you have easy access to all IDs of existing items? This is necessary > > to detect deleted items. > > Sure, a full list of IDs can be queried easily, with minimal overhead > (unlike their associated last-modified-times). I'm not totally sure > whether deleted IDs are included or not, though, might have to test. They don't have to when we keep a list of known IDs in the last sync. Any ID no longer listed then is known to be from a deleted item. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. From ovek at arcticnet.no Wed Dec 30 12:07:32 2009 From: ovek at arcticnet.no (Ove Kaaven) Date: Wed, 30 Dec 2009 13:07:32 +0100 Subject: [SyncEvolution] SyncEvolution in Fremantle In-Reply-To: <1262169205.3705.198.camel@pohly-mobl1.ikn.intel.com> References: <1256197555.4205.67.camel@pohly-mobl1.ikn.intel.com> <4B2E8378.7010909@arcticnet.no> <4B332042.3080509@arcticnet.no> <1261822284.3705.62.camel@pohly-mobl1.ikn.intel.com> <4B361557.6040106@arcticnet.no> <1262091676.3705.158.camel@pohly-mobl1.ikn.intel.com> <4B3AF659.1090603@arcticnet.no> <1262169205.3705.198.camel@pohly-mobl1.ikn.intel.com> Message-ID: <4B3B4284.1030608@arcticnet.no> Patrick Ohly skrev: > Looking at the code again and in particular > SyncSourceRevisions::updateRevision(), I remembered one problem with the > "changes since last sync" approach of change tracking. Suppose the > following chain of events: > 1. sync starts at time T1, with no local changes > 2. a changed item A is sent by the SyncML server and updated > locally > 3. the user edits a different item B while the sync runs > 4. sync ends at time T2 > > Now in the next sync, which time stamp should be used to find changed > items that need to be sent to the server? > > When using T1, the changed item A would be sent to the server, which is > wrong (additional traffic, unnecessary risk of conflicts on the server). > When using T2, the locally modified item B would not be sent. > > Updates of the database would have to be disabled while a sync runs to > avoid this problem. Is that possible? I don't think so. I think this is why I originally was talking about improving on TrackingSyncSource, not about replacing it; then I could make the backend retrieve all IDs modified after time T1, and then TrackingSyncSource would check which of these IDs has already been synced the same way as it currently does. I'd expect that to still be much faster than checking every ID in the whole database. >>> Do you have easy access to all IDs of existing items? This is necessary >>> to detect deleted items. >> Sure, a full list of IDs can be queried easily, with minimal overhead >> (unlike their associated last-modified-times). I'm not totally sure >> whether deleted IDs are included or not, though, might have to test. > > They don't have to when we keep a list of known IDs in the last sync. The point was that it could be a problem if it did include deleted IDs, since if it did, there'd be no way of distinguishing it from a non-deleted ID without doing additional database queries (like querying the list of deleted IDs with getAllDeletedItems(), and cross-referencing, or something). Hopefully getIdList() doesn't include the deleted IDs then; it probably doesn't, but I can't be sure until I test.