[Openicc] apps dropping color metadata

[Hal V. Engel]

On Friday, January 28, 2011 01:11:28 pm Chris Murphy wrote:
> On Jan 28, 2011, at 2:04 PM, Chris Murphy wrote:
> > Ideally we want all applications to not opt-out, right? So it needs to be
> > made easy to do that. If I drag and drop an eciRGB JPEG from the desktop
> > to my email application, of course the JPEG should look correct. If I
> > drop it into OpenOffice for a presentation, of course it should look
> > correct. If I put it into the simplest application, it should look
> > correct.
> 
> A question for someone who knows how applications open files, read their
> contents, interpret them, then pass them onto system services for display.
> Like a JPEG or TIFF. How does this actually happen?
> 
> For example, a JPEG file contains a bunch of information in the file that
> is not image-data. One piece of data might be EXIF metadata from a camera,
> including the color space (sRGB or Adobe RGB or other, depending on a
> setting in the camera), or it might be an ICC profile. It's all in the
> image file itself. But somehow when applications that support JPEG open
> the JPEG file, they ignore or strip away this data and only send the image
> data to the system for displaying the image. And that's why the ICC/EXIF
> data is dropped.
> 
> Is the way to prevent this to encourage applications to effectively send
> the entire JPEG file to some intermediate service that consistently
> ensures ICC and/or EXIF are honored on Linux systems? Seems like a huge
> need. This used to be a big problem on Mac OS until basically Apple
> created a system wide set of services for managing all of this to ensure
> metadata wasn't being dropped by applications.
> 
> 
> Chris Murphy

Actually it is more complicated than that.  Apps typically use some type of 
"widget" library such as Qt (Krita for example) or GTK (GIMP) and these 
libraries supply widgets that are used to hold, manipulate and display 
graphics.  In general the graphics objects supplied by these widget sets are 
usually not well suited to high quality graphics since these are almost always 
8bpc only. In addition it is common to use libraries external to the widget 
library for loading and saving images and to overcome the limitations of the 
built in graphics data types of the widget set for those who are trying to do 
high quality work.  For example Vigra is one that I have used but there are 
others such as GraphicsMagick and the GIMP folks are working on GEGL and 
friends.

Using these external libraries allows for programmers to work with the images 
in high bit depth buffers for any manipulations that need to be done to the 
actual image and then the image is converted to display format based on the 
content of the high bit depth buffers as needed and passed to the widget 
library for display.

I will use Qt as my example since of how this works since I have worked with 
it.  In Qt the QImage class is one class used for storing image data in memory 
and it is limited to 8bpc images.  As far as I know none of these widget 
libraries directly support handling of any of the meta data that is part of an 
image file and it is up to the app developers to extract this data and decide 
what to do with it.   There are other libraries for extracting EXIF from image 
files for example.

For displaying an image in a Qt app programmers would use a widget like a 
QGraphicsView.  The QGraphicsView widget is used to define an area of the app 
window that is used to display graphics data.  This allows for things like 
resizing and scrolling of the graphics display area.   QGraphicView is only 
one example widget that can be used to display graphics in an app since many 
Qt widgets can do this even things like buttons and text boxes.  But 
QGraphicsView is specifically designed to provide full support for this type of 
thing.   

What is displayed in the QGraphicsView can be almost anything from graphics 
that are drawn at runtime using program logic (using the QPainter class for 
example) or a QImage that was loaded from an external graphics file or some 
combination of these.  All of the logic for how the graphics are moved to the 
display are hidden from the programmer although this is open source code and 
it is possible to look at the code to figure out what it does.  Also keep in 
mind that these libraries are generally cross platform and one of the goals is 
to get these to function in exactly the same way on all platforms.   So 
dealing with platform differences is a major issue for those who are 
maintaining these libraries.  It also means things tend to be lowest common 
denominator which is likely the reason that we only have 8bpc support.

I have done some work with Qt using CM so I will comment on how it works so 
that non-programmers have an idea what is going on under the covers.   I will 
try to avoid too much technical detail.

First a little side note - Qt now has a QColor class that supports 16 bit float 
RGBA, CMYK and HSV formats but I have not been able to find a way to use 16 bit 
float QColors with any of the other graphics stuff in Qt (QImage and QPixMap 
only support 8bpc for example).  But I have not spent much time looking into 
this.  The float and CMYK and HSV support is new to Qt 4.x and didn't exist in 
Qt 3.x where QColors are RGBA 8bpc variants only.   I think the 16 bit float 
and CMYK and HSV work is to provide a foundation for better color processing 
in the future.

The CM image display flow in my Qt apps looks like this.

1. Open the file with Vigra and convert it into 32 bit float format and store it 
in a Vigra image object.  I do this because I need to handle what ever image 
format I am given without loss and the image file loading routines in Qt always 
reduce the data to 8bpc and only handle a limited number of image formats (for 
example you need a plug-in to handle tiff files).  Using Vigra I can avoid these 
issues.   Also the high bit depth Vigra buffer is where the real work is done 
to/with the image.

2. Check the image file for an ICC profile and if it has one create an LCMS 
profile object from it.

3. If it exists get the Monitor ICC_PROFILE atom create an LCMS profile object 
from it.

4. If both source and dest profiles where found create an LCMS transform with 
8bpc RGB output.

5. Create a QImage object of the correct size to hold the image being 
displayed.

6. Run the vigra float image data through the transform push that into the 
QImage object.  In this process I also have to default the alpha channel of 
the QImage to 255 since the QImage is expects this.

7. Assign the QImage object to the QGraphicView so that it is displayed.   
Note that this could be done at any point in this process since the QImage can 
be manipulated while it is being displayed by the QGraphicaView.

A non-color managed flow is much simpler since there may not be a need for an 
external library to handle the image data files and the image data can be 
directly loaded into a QImage or QPixMap. 

Note that this does not take into account things like windows that span more 
than one monitor and this would make things significantly more complex.  I 
don't know a any open source apps that handle this case currently other than 
through CompICC.

Hal

> _______________________________________________
> openicc mailing list
> openicc at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/openicc