xdgmime

[DAV]

Wow! First of all, I would like to say thank you for so much attention to
my small contribution!
Really I didn't expect that, thanks! I'm glad that my work to someone in
good stead.

> Sorry for not replying to your earlier private email about this.
It's ok :)

> I have one major issue with the implementation you're suggesting though.
> Unlike the mimetype stuff which is based on a mmap'ed cached
(mime.cache), it
> seems to parse all the .desktop files in-process. This means, if you
start 10
> apps and they all need at some point to use xdg_mime_*_apps_lookup, they
will
> each suffer the runtime penalty (slowness and memory usage) of parsing
all the
> .desktop files on the system.
You are absolutely right! I'm thinking about some mechanism to cache all
that staff
like other parts of xdgmime does. But specification does not describe where it
could
be stored, so I decided to not to do it right now. But, maybe this is the
time :)

> So for application .desktop files, we could have another helper binary,
say
> update-app-database (part of shared-mime-info maybe, so that we don't
need to
> depend on xdgmime, which we don't use in KDE and I think gnome might not
> either, or in a new module), which updates a new cache, say
> "application.cache" in a given directory. RPMs and other packages would
run
> that script when installing .desktop files, and then xdgmime could just
mmap
> that and look things up directly, without the need to parse any .desktop
files
> during the application runtime. Now if you implemented that, it would
> definitely be the best Christmas ever, in my eyes :-)
I think this is a great idea! I will begin the implementation of mmap'ed
cache
on the weekend.

> So overall, I'm not sure what this would be fixing, apart from the
> lack of an icon spec implementation at that level of the stack. There's
> already one in KDE, one in Qt, and surely a few in the gnome/glib/gtk
world,
> but I suppose none of this fits your bill otherwise you wouldn't have
written
> it :-)
This implementations do not suits me because, well they are in a huge
libraries :)
I'm adherent of small modules each of which is implementing strictly
defined task
with strictly defined interface.

> I agree the contents of the theme file itself is probably not as urgent to
> cache.
I agree with you, but maybe the thing is that I'm an perfectionist :) and I
think
if I can make an improvement I should do it.


Let me briefly explain how current implementation works, I hope it will
clarify
some things and help to understand what to do next.

There are two entry points, first one in
"xdgmime.c:143: xdg_mime_init_from_directory(...)". There is a call to
private
method "_xdg_mime_applications_read_from_directory(...)". This entry point
is
for "Desktop Entry Specification" (xdgmimeapp{.h, .c, _p.h}).

"_xdg_mime_applications_read_from_directory(...)" does parsing of all
"*.desktop"
and "*.list" files found according to "The Desktop Base Directory
Specification"
then stores all "*.desktop" files into AVL tree using the file name as
key (I've taken
into consideration that there could be subdirs which means prefix for
the file name
in global scope, like "kde4-"). As a value of key-value pair for AVL tree
another
AVL tree is used with contents of "*.desktop" file. This makes access to
any field
of any "*.desktop" file in the system very fast, I would say
as fast as possible (this
was the major task for me). Contents of all "*.list" files is stored in one
AVL tree.
Also, all associations of mime type with "*.desktop" files (from
"*.desktop" files)
are stored in another AVL tree.

So, it was about internals. Let me describe public interface of the
"Desktop Entry Specification" (file: "xdgmimeapp.h"). As already mentioned
by David, the are three interesting methods:

 - "xdg_mime_default_apps_lookup(...)" - this method searches for all
applications
associated with the given mime type in section "Default Applications" of
all found
"*.list" files (do you remember, contents of all "*.list" files is stored
in one
AVL tree - it's like we merged all "*.list" files into one);

 - "xdg_mime_user_apps_lookup(...)" - this method searches for all
applications
associated with the given mime type in section "Added Associations" of all
found
"*.list" files. As mentioned by David, it is not quite correct
because there is
"Removed Associations" section about which I didn't know (or missed it);

 - "xdg_mime_known_apps_lookup(...)" - this method searches for all
applications
associated with the given mime type using third AVL tree I have described at
very beggining (where all associations of mime type with "*.desktop" files);

 - "xdg_mime_app_icon_lookup(..)" - well, it depends on "Icon theme
specification"
and will be described later;

 - "xdg_mime_app_group_lookup(...)" - this method can return a group
from "*.desktop" file (like "Desktop Entry");

 - "xdg_mime_app_entry_lookup(...)" - this method can return array of entry
values
from group of "*.desktop" file (like "Name");

 - "xdg_mime_array_app_item_at(...)" - this method is needed for traversing
through
array of XdgApp ("*.desktop" files).

Note: I'm thinking about localized fields... I think it should be merged
with parent field
(for example Name[de] with Name). Provide convinient interface for reading
of
localized values of the same field.

We got to the second entry point which is in
"xdgmime.c:449: xdg_mime_init(...)".
There is a call to private
method "_xdg_mime_themes_read_from_directory(...)".
This entry point is for "Icon Theme Specification" (xdgmimetheme{.h, .c,
_p.h}).
This implementation has a "special" entry point (not in "convinient" place)
because it
relies on diffrent layout of directories than "The Desktop Base Directory
Specification"
describes (includes "~/.icons/" and "/usr/share/pixmaps/").

Initialization of this mechanism works similar to previous one with one
difference - it
reads "index.theme" files. "_xdg_mime_themes_read_from_directory(...)" does
parsing of
all "index.theme" files found according to directory layout from "Icon
Theme Specification"
then stores all "index.theme" files into AVL tree using theme directory
name as key.
As a value of key-value pair for AVL tree another AVL tree is used with
contents of
"index.theme" file. This approach eliminates need of scanning whole theme
directory and
directory of parent theme (and parent of parent, etc) to find icon.

In general, for algorithm of filesystem scanning we have absolute path to
directories
where icon could be. The only thing we have to do is to check existence of
the icon file
in this directories. If its not there take the parent theme and so on.

Here is the public interface of the "Icon Theme Specification" (file:
"xdgmimetheme.h"):

 - "xdg_mime_type_icon_lookup(...)" - this method can return absolute path
to icon file
for given mime type, icon size and theme name. It converts mime type to
icon name
by replacing "/" to "-" then searches for theme by given theme name (for
example "hicolor")
and looks for icon in directories listed in the theme file ("index.theme");

 - "xdg_mime_icon_lookup(...)" - this method can return absolute path to
icon file for
given icon name, icon size, context and theme name. It searches for theme
by given
theme name (for example "oxygen") and then looks for icon in directories
listed in
the theme file ("index.theme");

 - "xdg_mime_app_icon_lookup(..)" - It was mentioned in "Desktop Entry
Specification"
interface. This method takes icon name from "*.desktop" file and searches
for it in a given
theme.

Note: I think this implementation could be improved by adding an AVL tree
for indexing
(by size and/or context) directories inside of theme file. But I'm not
assured would it
be really improvement or not.

P.S.: sorry for by bad english.
--------------------------------
Best regards, Dmitriy.



24 января 2012 г. 2:20 пользователь David Faure <faure at kde.org> написал:

> On Sunday 15 January 2012 17:40:27 DAV wrote:
> > I added missing features:
> >  - C-implementation of AVL trees;
> >  - indexed (by using of AVL trees) access to all ".desktop" files and its
> > contents (implementation of "Desktop Entry Specification");
>
> Hello Dmitriy,
>
> Sorry for not replying to your earlier private email about this. But let's
> discuss this here, it's a better idea indeed.
>
> I'm very excited by the idea of having a cross-desktop solution to the very
> common issue of getting the list of apps associated with a given mimetype,
> I've been thinking that we need that, for quite some time (when I saw your
> mail, I thought "wow, it's Christmas!").
>
> (We use ksycoca in kde4 for this purpose, but I'd like to get rid of it for
> kde5, so the timing is perfect.)
>
> I have one major issue with the implementation you're suggesting though.
> Unlike the mimetype stuff which is based on a mmap'ed cached (mime.cache),
> it
> seems to parse all the .desktop files in-process. This means, if you start
> 10
> apps and they all need at some point to use xdg_mime_*_apps_lookup, they
> will
> each suffer the runtime penalty (slowness and memory usage) of parsing all
> the
> .desktop files on the system.
>
> We don't do that with mimetypes: shared-mime-info has a binary called
> "update-
> mime-database" which does the collecting and parsing of the (mime) files,
> and
> generates a binary file called mime.cache, whose on-disk layout is such
> that it
> can be used via mmap. This way, the applications are fast.
>
> So for application .desktop files, we could have another helper binary, say
> update-app-database (part of shared-mime-info maybe, so that we don't need
> to
> depend on xdgmime, which we don't use in KDE and I think gnome might not
> either, or in a new module), which updates a new cache, say
> "application.cache" in a given directory. RPMs and other packages would run
> that script when installing .desktop files, and then xdgmime could just
> mmap
> that and look things up directly, without the need to parse any .desktop
> files
> during the application runtime. Now if you implemented that, it would
> definitely be the best Christmas ever, in my eyes :-)
>
> Alex, Bastien: do you agree about the approach? Would gnome/gtk/glib use
> it,
> if it was done that way? (Otherwise half the RPMs/DEBs in the world will
> forget to run the update-app-database... so to me this solution will only
> work
> out in practice if it's used by everyone.)
>
> > const XdgArray *xdg_mime_default_apps_lookup(const char *mimeType);
> > const XdgArray *xdg_mime_user_apps_lookup(const char *mimeType);
> > const XdgArray *xdg_mime_known_apps_lookup(const char *mimeType);
>
> A bit of documentation wouldn't hurt ;-) What's each of those doing
> exactly?
> It seems user_apps looks at the contents of mimeapps.list (but allows any
> *.list file? Is that in the spec?), but it only looks at [Added
> Associations],
> so this is missing support for [Removed Associations]. I think "user apps"
> should return the final set: default apps plus those added by the user,
> minus
> those removed by user. That's what client code cares mostly about.
>
> There's also the issue that the defaults should be desktop-dependent, but
> that's an entire new topic. The current situation is that KDE uses
> InitialPreference in the desktop files while Gnome uses a defaults.list
> file
> (iirc). One could say it sucks that this is non-standard, but on the other
> hand it gives "defaults are desktop-dependent" de facto.
> If we were to use a common way (which would definitely please distros and
> ISVs), then we would still need a way to make the defaults
> desktop-dependent,
> I would think, somehow (maybe a *.list file per desktop). (Although I
> definitely
> prefer the more modular KDE solution where an app can be installed later on
> and not have to hack into a global file to take precedence).
> I'm sure we've had this discussion already on this list, but I forgot the
> outcome of it :-}
>
> >   - indexed (by using of AVL trees) access to icon themes and its
> contents
> > (implemetation of "
> >
> http://standards.freedesktop.org/icon-theme-spec/icon-theme-spec-latest.html
> > " and
> > "
> >
> http://standards.freedesktop.org/icon-naming-spec/icon-naming-spec-latest.ht
> > ml ").
>
> My first reaction was: this, too, only seems interesting to me if it's
> based on
> a mmap'ed binary cache. Otherwise apps can just look things up directly on
> disk, as they already do currently. But either it only caches the contents
> of
> the theme files, and then it seems not so useful (there are only very few
> of
> them, typically, right?), or it also caches the names of all available
> icons,
> but that might not be a big saving compared to looking up in the file
> system
> directly. So overall, I'm not sure what this would be fixing, apart from
> the
> lack of an icon spec implementation at that level of the stack. There's
> already one in KDE, one in Qt, and surely a few in the gnome/glib/gtk
> world,
> but I suppose none of this fits your bill otherwise you wouldn't have
> written
> it :-)
> Anyhow, no real objection from me, but the target audience probably needs
> to
> be defined, and like xdgmime itself (AFAIK), I think it wouldn't be much
> used.
>
> --
> David Faure, faure at kde.org, http://www.davidfaure.fr
> Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/xdg/attachments/20120125/66cef620/attachment-0001.htm>