RFC: Free Software community-wide Hardware Database project

Sun Jun 20 15:23:09 PDT 2004

Hi,

Sorry for not responding earlier,

On Mon, 2004-06-14 at 22:00, Zenaan Harkness wrote:
> I am launching a kernel-, os-, distro- and vendor- neutral, free
> software, community-wide hardware database project.
> 
> This is a reqeust for comment, feedback, constructive criticism, etc.
> 
> Key points as I see them:
> 
> * Provide a single point of contact for hardware manufacturers,
> resellers, end users, distribution and driver developers to
> submit hardware information regarding how hardware devices
> relate to various free software projects, be they kernels,
> operating systems/ distributions, drivers, support libraries,
> etc.
> 

As a end user, I for one would like to see a single site with what
hardware works and what doesn't - too many times I've just taking the
chance and bought some hardware that turned out not to work. Oh well.

However, it sounds like an ambitious project though; please indulge some
of my ramblings :-).

> * The key focii of the project are:
>  - hardware manufacturers
>  - single point of contact
>  - free software (the superset of DFSG, FSF/GNU and OSI)
>  - neutrality
> 
> * Key classes of users:
> 1) end users
> 2) manufacturers
> 3) driver developers (at least those independant from class 2)
> 4) distro- and os- developers and vendors
> 
> * Neutrality:
> Operating System Neutral
> Kernel Neutral
> Distribution Neutral
> Vendor Neutral
> 
> * Manual Submission:
> ** Manual submission will produce better submissions (the hwdb team
> can engage the submitter where required (perhaps nearly always) to
> try to get more and better information.
> 
> ** As we build the DB, manual submissions will give us the scope to
> easily and readily modify/ fine-tune the schema as needed.
> 
> ** The submission interface will never be broken, incomplete or
> "down for maintenance" since it's just "send us an email".
> 
> ** We can launch immediately, and with a robust interface.
> Ie. we won't waste time having to get the software right
> before we are able to launch.
> 
> ** A robust submission interface is particularly important for
> manufacturers - being the primary target here (in my mind) we want
> to _never_ waste their time and _always_ have submissions working.
> 
> ** No elitism over whether we use PHP or Python, GTK or QT
> (OK, that's probably stretching it a bit :)
> 
> * Centralization:
> We assume that it is in the best interests of each Free Software
> Unix-like operating system distribution, each kernel (eg. Linux, *BSD,
> HURD) and in the best interests of the end users, to have a centralised/
> unified location for hardware information.
> 

Risking a limb here :-), but I can see a few advantages of also
including listings for closed source operating systems; from the top of
my head here are some advantages:

 1. It gives you more users, thus the hardware database might be
    populated faster; it's also an incentive for manufactures to 
    deliver information even though they may not support free software
    OS'es.

 2. Developers of Free operating systems can possibly use this
    information when writing a free software driver

 3. A good opportunity for saying this device works better on e.g.
    Linux or one of the BSD's than on e.g. Windows (you could have
    quality/performance indicators)

 4. The neutrality thing, realizing and accepting that there are
    still closed OS'es around  :-)

> This centralization is so that manufacturers have a single point of
> contact to submit their own hardware information to, however much or
> little (eg. perhaps even just contact information) that might be.
> 
> ---
> And now for those still hanging in and want a little more verbosity/
> rant:
> 
> First and foremost is to simplify the job of the manufacturers:
> 
> For Microsoft, they have a single point of contact.
> 
> Contrast this with the numerous HCLs (hw compat lists), hardware sites
> (such as www.linux1394.org and linuxprinting.org), kernels and
> distributions, such as Debian, Red Hat, *BSD (with apparently >60
> officially supported sub architectures) and a myriad of others.
> 
> As a manufacturer, it is simply impossible to (generally) go anywhere
> near supporting all these free software projects.
> 
> And so it is in the best interests of each of us individually, and
> collectively, if we can simplify the job of the manufacturer.
> 
> As a manufacturer of a widget, if I have a single, commonly known place
> to go to provide technical and contact information, as much or as little
> as I desire (even perhaps just bus IDs and product names), then I might
> actually do so.
> 
> We, as a community together, might just have a hope of keeping up to
> date as compared with the proprietary os's out there, namely MSW*.
> 

Ah, yeah the community thing. Right. I think it would be good to have
some kind of "user privilege" rating on the hardware database, karma or
something, much the same way that Advogato (when it's up, that is :-)
has a model of trust. Helps distribute work among several individuals.

> ---------
> Thus, The Project (TM).
> 
> Being os-, vendor-, distribution- and kernel- neutral will attract many
> otherwise disparate parties within our community, such as the BSDs and
> the GNU/Linux distros, driver developers, etc. And of course,
> manufacturers.
> 
> ---------
> The plan is to integrate to the greatest and most seamless extent
> possible with existing Distribution-specific HCLs and thereby due to
> this centralization provide a richer facility than is otherwise possible
> today.
> 

Btw, someone talked last year about doing this for Fedora, see

 http://people.ucsc.edu/~maxka/fhp/

but nothing really happened AFAIK.

Now, since this is on the HAL mailing list :-), let's discuss how HAL
can fit in - I can see HAL both as a consumer and provider of
information from/to your repository.

For consuming information, HAL uses what I call "Free Device Information
Files" or .fdi-files. Basically right now this is used to provide
additional information that cannot be probed from the device, such as
whether the device is a camera (that is, a real physical camera), a
music player (and what formats it is able to play, e.g. mp3, wma etc.
Think of an iPod or a cheapo USB-storage mp3 player) or some card reader
uses Compact Flash or Smart Media cards. 

Nothing special, but it enables desktops like KDE or GNOME to present a
better user interface than without it; e.g. suitable icons or ways to
interact with the device.

It's really simple as well, the .fdi for my camera looks like this

  <?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->

  <deviceinfo version="0.2">
    <device>
      <match key="info.bus" string="usb">
        <match key="usb.vendor_id" int="0x04a9">
          <match key="usb.product_id" int="0x3052">
            <merge key="info.category" type="string">camera</merge>
            <merge key="info.capabilities" type="string">camera</merge>
            <merge key="info.persistent" type="bool">true</merge>
          </match>
        </match>
      </match>
    </device>
  </deviceinfo>

So, the database might export a large set of .fdi files and/or provide a
web service with this information. At some point it would probably be
good to have cryptographic signature on the the .fdi files, your most
trusted users on the hardware database can then sign and verify that the
information is correct using some PKI stuff (Indeed, the 'users' may
also be corporate entities like hw vendors, OS vendors and of course
individual contributors).

Now, HAL proper has nothing to do with system policy such as which
driver to load, however through the hooks offered by HAL it may makes
sense to use it for this if e.g. the OS supports driver binding (e.g.
userspace gets to decide what kernel driver to use for a specific
device; AFAIK Linux doesn't support this yet?). So, indeed the .fdi file
might export the properties 

 info.kernel_module.name = bttv
 info.kernel_module.opts = card=5 pll=2

and when the device is discovered, a HAL callout simply does the
modprobe thingie that e.g. linux-hotplug and modprobe.conf does on
Linux.

The interesting thing is that these properties might stem from a .fdi
file where we have more sophisticaed conditionals than just <match> on
USB or PCI vendorId/productId, e.g. we merge bttv if we're on Linux 2.6,
bttv_some_future_module if on Linux 2.8 and bktv if on FreeBSD (I'm
making this up, it's only an example). 

The hardware database should be able to export such .fdi files, that
would make hardware detection "just work" without having the user to go
and edit /etc/modprobe.conf or what have you. In some distant future,
that is :-)

Some options on how to actually configure the hardware might be missing
or impossible to figure out, e.g. you'd have to ask the user, or at
least provide a user interface where he can change it. For example,
continuing the tv tuner card example, that would be whether the RF
signal being fed to the card is PAL or NTSC and what MHz range to scan 

(it wouldn't be enough just checking the timezone and guesstimating,
since the system used might be in a lab in Europe where a closed NTSC
broadcast network is being used - I tried that :-).

I've rambled a bit about that here 

 http://mail.gnome.org/archives/utopia-list/2004-April/msg00025.html

where the essence is that the .fdi file can specify what options the
user interface in e.g. GNOME or KDE need to export. Automated GUI
dialogs, yeah, uhmm :-)

Regarding HAL providing .fdi files my thinking right now is some star
trek version of hal-device-manager, e.g. 

 http://freedesktop.org/~david/hal-v4l.png

will be able to generate a .fdi file given the current configuration of
the device and how well it works. Then you would simply have a "Submit
to HW database" button :-)

Oh well, hope I didn't ramble too much - the thing is I think this is
very interesting; it's much in the line of "Making hardware just work"
which HAL is trying to solve a tiny bit of (e.g. there's so much work
left :-). It would be good to see the project you're describing
happening! It's ambitious, but it can probably be done incrementally, I
think providing automated tools for submission of e.g. .fdi files and
other vital product data may be one of the stepping stones to a
successful project.

Thanks,
David

_______________________________________________
hal mailing list
hal at freedesktop.org
http://freedesktop.org/mailman/listinfo/hal