file/libmagic

[Thomas Leonard]

On Fri, May 16, 2003 at 12:22:22AM +1000, Jeff Waugh wrote:
> <quote who="Thomas Leonard">
> 
> > The people who currently maintain the various existing MIME databases
> > should ask for CVS access on freedesktop.org so that they can update the
> > central database (and, perhaps, make new releases).
> > 
> > There should be a list of such people on the site so people know where to
> > send corrections and additions.
> 
> (Are we / Why aren't we) cooperating with the maintainers of file, and using
> the existing system that is deployed on countless systems already? Why are
> we using an entirely new format?

The maintainer of file suggested it would be better to use a binary
format, based on the existing format. This is what we are using. It is
very easy to convert. Updating Geoff Youngs' freemime library from a
text based format to the new binary format required a patch which added
473 lines, and removed 2016. The total size of the library after this was
1026 lines. Thus, the binary format makes it much easier for people to
implement the specification, and removes a likely source of bugs.

GNOME, KDE, ROX and others each deployed their own MIME databases for a
reason: we can't wait 2 years for Debian to bring out a new stable release
before people can use any new software. This means you need merging rules,
priorities and extensibility.

The new system provides a much cleaner way to specify the rules (the
escaping rules in the original format were quite nasty), validates the
rules carefully, and provides helpful error messages. It also provides an
easy way for distributions to generate the old format files from the new.

Actually, I thought that was Debian's plan for the new system already.
Debian already merges MIME information from multiple packages with it's
update-mime command; our system can replace this with one that also
handles magic, namespaces, translated descriptions and other extensions.

> I'm perplexed and concerned at the lack of discussion of existing systems on
> this list, the eager uptake of NIH standards that don't integrate with
> what's already out there in the real world, and the massive chasm between
> "desktop standards" and what everyone else is / has been doing.
> 
> Having a desktop standards group doesn't mean we *have* to create new
> standards, and it certainly doesn't mean we should limit our integration
> work to desktop oriented projects only! :-) Surely we want to interoperate
> as widely as possible?

As far as I am aware, file(1) is the only program that parses the existing
database (probably because it is rather complicated). So I'm not sure what
these other programs we're supposed to be interoperating with are? If you
change file (or libfile) to use the new database, any existing programs
should use that without problems.

You can even generate the old-style format too, if you want (it's similar
to the new format, except harder to parse and has a few extra operators
which hardly anyone was actually using).

I'd have thought that if there were any compatibility issues, that
changing the old format would cause far more problems than creating a new
one in parallel.


-- 
Thomas Leonard			http://rox.sourceforge.net
tal00r at ecs.soton.ac.uk		tal197 at users.sourceforge.net
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1