<font color="#000000"><font><font face="trebuchet ms,sans-serif">Hi,</font></font></font><div><font color="#000000"><font><font face="trebuchet ms,sans-serif"><br></font></font></font></div><div><font color="#000000"><font><font face="trebuchet ms,sans-serif">Sorry for delay - I've been busy a bit...</font></font></font></div> <div><font color="#000000"><font><font face="trebuchet ms,sans-serif"><br></font></font></font></div><div>> This part is quite confusing:<br>> <br>> "What are the Global AVL trees mentioned above? [...]<br> > <br>> @note Actually, this problem is solved a bit different way. There is no Global<br>> AVL trees [...]"<br>> <br>> Please describe the actual state of things, not what could have been done and<br> > was not done ;)<br></div><div><font color="#000000"><font><font face="trebuchet ms,sans-serif"><br></font></font></font></div><div><font face="trebuchet ms, sans-serif"><div>When I've read this docs after a while I was amazed - it's just a stream of mind :)</div> <div>That was definitely not my day :)</div><div>I've updated it to be more or less understandable (<a href="http://vilkov.github.com/libxdg/">http://vilkov.github.com/libxdg</a>).</div></font></div><div><font color="#000000"><font><font face="trebuchet ms,sans-serif"><br> </font></font></font></div> <div>> Because of this confusing documentation, I'm still not sure how one is<br>> supposed to see a "global tree" when using the caches directly (rather than<br>> when using the library API). What does "fixing the references" mean?<br> > <br>> This is about the case where a local mimeapps.list refers to a globally-<br>> installed .desktop file, right ?</div><div><font color="#000000"><font><br></font></font></div><div><div>Right. The thing is when we have several directories, ".list" files from one directory could have</div> <div>references to ".desktop" files from another directory (usually this is the case). It means that</div><div>after we loaded a cache file or after we parsed some ".desktop" file, we must check that ".list"</div> <div>files from another directories had the correct references (pointers) to just loaded data.</div></div><div><br></div><div>> There will never be so many mimetypes. On a modern linux system there are<br>> about 400 mimetypes in application, and 660 overall.<br> > In any case, cutting the mimetypes in two, and performing two AVL-tree<br>> lookups, doesn't sound faster than performing one AVL-tree lookup with the<br>> full mimetype name... But anyway. I don't have time to experiment with that,<br> > and neither do you apparently, so let's leave it as such.<br>> But let's not use "thousands or tens of thousands" as an argument, it's not a<br>> valid argument :)</div><div><font color="#000000"><font><br> </font></font></div><div><font color="#000000"><font>Agree :)</font></font></div><div><font color="#000000"><font><br></font></font></div><div><div>> > > > Each value of this tree is an AVL tree of sub types (e.g. html), which<br> > > > > contains a list of pointers to XdgApp structures.<br>> > ><br>> > > What does this structure contain? The full contents of the desktop file,<br>> > > again?<br>> > > Or do I misunderstand that?<br> > > > Surely there's no need to duplicate the contents again, for each mimetype<br>> > > associated with the application. Wouldn't it be enough to write in this<br>> > > tree,<br>> > > the name of the desktop file, in order to then look it up in the other<br> > > > tree if one wants to get more details about it?<br>> > > Or is this space-optimized anyway, by pointing to the same data in the<br>> > > on-disk cache?<br>> </div>> These questions remain.</div> <div><br></div><div>It's just association of a mime type with ".desktop" files. In turn, ".desktop" file</div><div>can be loaded/parsed dynamically or can be a part of a binary cache file.</div><div> <br><div><h3 style="line-height:12px;font-family:inherit;font-style:inherit;margin:0px;padding:0px;border-width:0px;outline:0px;font-weight:inherit;vertical-align:baseline;color:rgb(51,51,51)"> <font><div style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> > > With a tree for the associations coming from desktop files, and another<br>> > > tree<br>> > > for the contents of defaults.list/mimeapps.list, one has to look up into<br> > > > multiple trees to be able to answer that question. Wouldn't it be faster,<br>> > > and<br>> > > simpler (higher level) to have a single tree, combining these two?<br>> > > I.e. a simple tree with<br> > > ><br>> > > key = mimetype -> value = list of desktop files<br>> > ><br>> > > ("merging" .list files into the desktop files) would give an immediate<br>> > > result,<br> > > > compared to three lookups (initial list, then added associations, then<br>> > > removed<br>> > > associations), all this multiplied by the number of paths in XDG_DATA_DIRS<br>> > > plus one local dir, of course.<br> > ><br>> > Everything just the way you describe it - the library returns merged<br>> > results from<br>> > all XDG_DATA_DIRS directories and local directory too.<br>> > When one asks the library<br> > > about, e.g. default associations the library returns a list of apps from<br>> > all ".list" files<br>> > (system-wide) there is no initial list and there is no need (and actually<br>> > possibility too)<br> > > to scan XDG_DATA_DIRS. All this implementation details (like XDG_DATA_DIRS<br>> > and<br>> > local dir) encapsulated in the library.<br>> </div><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">Yes I know that the library does all this. My question is: could we make this</span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"> <span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">more optimized and reuseable, by having this logic at the time of building up</span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"> <span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">the binary cache, rather than at the time of reading into the cache?</span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"> <span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">Otherwise we're really not gaining much by having this cache, if it only has</span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"> <span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">raw unprocessed copy of the file contents.</span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"> <span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">The binary cache only makes sense if it contains data in 'ready to use' form</span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"> <span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">> </span><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small">(as much as possible).</span><br style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"> </font></h3><div style="font-family:Arial,Helvetica,'Nimbus Sans L',sans-serif;line-height:12px"><font><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"><br></span></font></div><div> There is no extra operations on cache - not during loading, nor using.</div><div>Cache is a number of AVL trees mmap'ed into memory. So, there is only two necessary</div><div>operations after cache is mmap'ed:</div> <div> - updating of pointers in mmap'ed memory (to allow AVL tree to travers through its nodes);</div><div> - updating of pointers in other [cache files/parsed ".list" files] (to allow ".list" files from other</div> <div>directories to refer to the actual data).</div><div><br></div><div>The only way it could be more optimaized is to have only one cache file for whole</div><div>system.</div><div><br></div><div style="font-family:Arial,Helvetica,'Nimbus Sans L',sans-serif;line-height:12px"> <font><div style="font-family:arial;line-height:normal">> > > Do you think this could be done? Or am I overlooking some difficulty here?<br>> ><br>> > It could easily be done by adding one more function which will group the<br> > > results from<br>> > different groups ("Default Associations", "Added associations", etc).<br>> > By the way, this functionality is implemented in example in online<br>> > documentation.<br> > </div><span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">I'm not asking for more functions, but for a different layout in the on-disk</span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">cache :-)</span><br style="font-family:arial;line-height:normal"><span style="font-family:arial;line-height:normal">> </span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">But yes, this goes together. The most common use case for this library should</span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">require a single function (plus iteration loop) rather than three (plus</span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">iteration loop), and the on-disk cache should be optimized for this use case.</span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><br style="font-family:arial;line-height:normal"><span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">If the user-preferences-handling code needs to explicitely access "added" and</span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">"removed" lists, it can just use the mimeapps.list files, as it does already.</span></font></div> <div style="font-family:Arial,Helvetica,'Nimbus Sans L',sans-serif;line-height:12px"><span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">So if you want to keep this API in the library, it could just read these files</span></div> <div><font><span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">directly. Or don't provide it.</span><br style="font-family:arial;line-height:normal"><span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">But the goal of the on-disk cache is really to pre-process the data in order</span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">to make the "which app(s) handle this mimetype" lookup as fast as possible,</span><br style="font-family:arial;line-height:normal"> <span style="font-family:arial;line-height:normal">> </span><span style="font-family:arial;line-height:normal">and as simple as possible for developers.</span><br style="font-family:arial;line-height:normal"><div style="font-family:arial;line-height:normal"> </div></font></div><div style="font-family:Arial,Helvetica,'Nimbus Sans L',sans-serif;line-height:12px"><font><span style="color:rgb(0,0,0);font-family:arial;line-height:normal;font-size:small"><br></span></font></div> <div><div>Yes, it definitely is. I will merge trees representing associations of mime types with</div><div>"added", "default", "all other" and "removed" lists in the next release (within couple</div> <div>weeks).</div></div><div><br></div><div><span style="color:rgb(51,51,51);font-family:inherit;font-style:inherit;line-height:12px">------------------------------</span><span style="color:rgb(51,51,51);font-family:inherit;font-style:inherit;line-height:12px">--</span></div> </div><div style="line-height:12px;font-family:Arial,Helvetica,'Nimbus Sans L',sans-serif"><h3 style="font-family:inherit;font-style:inherit;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;border-top-width:0px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;border-style:initial;border-color:initial;outline-width:0px;outline-style:initial;outline-color:initial;font-weight:inherit;vertical-align:baseline;color:rgb(51,51,51)"> <font>Best regards, Dmitriy.</font></h3></div><br> <br><br><div class="gmail_quote">2012/6/14 David Faure <span dir="ltr"><<a href="mailto:faure@kde.org" target="_blank">faure@kde.org</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div>On Monday 28 May 2012 02:23:49 DAV wrote:<br> > Hi, all!<br> ><br> > I finally did it :) I have update libxdg documentation:<br> > - here is the small Wiki with basic info about the library:<br> > <a href="https://github.com/vilkov/libxdg/wiki" target="_blank">https://github.com/vilkov/libxdg/wiki</a><br> > - here is updated online Doxygen documentation:<br> > <a href="http://vilkov.github.com/libxdg" target="_blank">http://vilkov.github.com/libxdg</a><br> > - and code itself: <a href="https://github.com/vilkov/libxdg" target="_blank">https://github.com/vilkov/libxdg</a><br> ><br> > Could you possibly take a look at the documentation please, especially if<br> > you are Stake Holder, like e.g. David :)<br> <br> </div>This part is quite confusing:<br> <br> "What are the Global AVL trees mentioned above? [...]<br> <br> @note Actually, this problem is solved a bit different way. There is no Global<br> AVL trees [...]"<br> <br> Please describe the actual state of things, not what could have been done and<br> was not done ;)<br> <br> Because of this confusing documentation, I'm still not sure how one is<br> supposed to see a "global tree" when using the caches directly (rather than<br> when using the library API). What does "fixing the references" mean?<br> <br> This is about the case where a local mimeapps.list refers to a globally-<br> installed .desktop file, right ?<br> <div><br> > > > * List of XdgFileWatcher structures<br> </div><div>> It's just like you said - there is no any file watching facility. This<br> > structures is used just<br> > for checking validity of cache files.<br> <br> </div>OK, I see. I got confused by the naming because of the resemblance with<br> QFileSystemWatcher (FAM/inotify stuff).<br> <div><br> > > "text" is not the name of a mime type. "text/html" is. Is there any<br> > > reason for<br> > > splitting text/html into two leaves in the tree? Is this for supporting<br> > > mimetypes like text/* or image/*? Hmm, why not. But otherwise it's a bit<br> > > pointless and confusing.<br> ><br> > Hmm... It could be so, but the reason of this is performance. When we have<br> > only tens of mime types it won't be a problem, but when we are speaking<br> > about<br> > thousands or even tens of thousands for each mime group/sub type<br> > (i.e. application/ text/ etc) it will be a serious problem.<br> <br> </div>There will never be so many mimetypes. On a modern linux system there are<br> about 400 mimetypes in application, and 660 overall.<br> In any case, cutting the mimetypes in two, and performing two AVL-tree<br> lookups, doesn't sound faster than performing one AVL-tree lookup with the<br> full mimetype name... But anyway. I don't have time to experiment with that,<br> and neither do you apparently, so let's leave it as such.<br> But let's not use "thousands or tens of thousands" as an argument, it's not a<br> valid argument :)<br> <div><br> > > > Each value of this tree is an AVL tree of sub types (e.g. html), which<br> > > > contains a list of pointers to XdgApp structures.<br> > ><br> > > What does this structure contain? The full contents of the desktop file,<br> > > again?<br> > > Or do I misunderstand that?<br> > > Surely there's no need to duplicate the contents again, for each mimetype<br> > > associated with the application. Wouldn't it be enough to write in this<br> > > tree,<br> > > the name of the desktop file, in order to then look it up in the other<br> > > tree if one wants to get more details about it?<br> > > Or is this space-optimized anyway, by pointing to the same data in the<br> > > on-disk cache?<br> <br> </div>These questions remain.<br> <div><br> > Well as I mentioned before old documentation was a bit outdated and, lets<br> > say, didn't describe whole things clearly enough... I hope new one will.<br> <br> </div>Well you changed "structures" into "items", but that doesn't really answer my<br> questions :-)<br> <br> Does the second tree point into the first one, or does it have its own items?<br> <div><div><br> > > With a tree for the associations coming from desktop files, and another<br> > > tree<br> > > for the contents of defaults.list/mimeapps.list, one has to look up into<br> > > multiple trees to be able to answer that question. Wouldn't it be faster,<br> > > and<br> > > simpler (higher level) to have a single tree, combining these two?<br> > > I.e. a simple tree with<br> > ><br> > > key = mimetype -> value = list of desktop files<br> > ><br> > > ("merging" .list files into the desktop files) would give an immediate<br> > > result,<br> > > compared to three lookups (initial list, then added associations, then<br> > > removed<br> > > associations), all this multiplied by the number of paths in XDG_DATA_DIRS<br> > > plus one local dir, of course.<br> ><br> > Everything just the way you describe it - the library returns merged<br> > results from<br> > all XDG_DATA_DIRS directories and local directory too.<br> > When one asks the library<br> > about, e.g. default associations the library returns a list of apps from<br> > all ".list" files<br> > (system-wide) there is no initial list and there is no need (and actually<br> > possibility too)<br> > to scan XDG_DATA_DIRS. All this implementation details (like XDG_DATA_DIRS<br> > and<br> > local dir) encapsulated in the library.<br> <br> </div></div>Yes I know that the library does all this. My question is: could we make this<br> more optimized and reuseable, by having this logic at the time of building up<br> the binary cache, rather than at the time of reading into the cache?<br> Otherwise we're really not gaining much by having this cache, if it only has<br> raw unprocessed copy of the file contents.<br> <br> The binary cache only makes sense if it contains data in 'ready to use' form<br> (as much as possible).<br> <div><br> > > Do you think this could be done? Or am I overlooking some difficulty here?<br> ><br> > It could easily be done by adding one more function which will group the<br> > results from<br> > different groups ("Default Associations", "Added associations", etc).<br> > By the way, this functionality is implemented in example in online<br> > documentation.<br> <br> </div>I'm not asking for more functions, but for a different layout in the on-disk<br> cache :-)<br> <br> But yes, this goes together. The most common use case for this library should<br> require a single function (plus iteration loop) rather than three (plus<br> iteration loop), and the on-disk cache should be optimized for this use case.<br> <br> If the user-preferences-handling code needs to explicitely access "added" and<br> "removed" lists, it can just use the mimeapps.list files, as it does already.<br> So if you want to keep this API in the library, it could just read these files<br> directly. Or don't provide it.<br> But the goal of the on-disk cache is really to pre-process the data in order<br> to make the "which app(s) handle this mimetype" lookup as fast as possible,<br> and as simple as possible for developers.<br> <div><div><br> --<br> David Faure, <a href="mailto:faure@kde.org" target="_blank">faure@kde.org</a>, <a href="http://www.davidfaure.fr" target="_blank">http://www.davidfaure.fr</a><br> Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5<br> <br> </div></div></blockquote></div><br></div>