ITK/Release 5/DICOM/Color

Multiple Components in DICOM

ITK 5.0

DICOM standard allow storing of image with more than one components. As of today, only 1 or 3 components are considered valid DICOM SOP Class instances (for all IODs).

By design the itk::GDCMImageIO was designed and implemented so that an input YBR_FULL image would be loaded as such (no implicit conversion to RGB was done). The main reason for that is that ITK is a processing toolkit, so quantitative analysis is supposed to be done on the best possible pixel representation. Since conversion from integer YBR_FULL to integer RGB colorspace is a lossy operation (truncation in floating point representation), the conversion has never been implemented.

As side note MONOCHROME1 color space is not converted to MONOCHROME2 (itk::GDCMImageIO / ITK 5.0)

ITK currently does not have any standard way of noting the color space of an Image. Originally, there was a direct mapping between the `ImageIOBase::PixelType` and the intended pixel type used for the template parameter in the `Image` class. Modern usage of ITK frequently uses the generalized `VectorImage` to hold multi-component images so there may even be less knowledge of the content in the Image buffer. Additionally, we need to consider the default behavior of `ConverPixeltBuffer` (https://itk.org/Doxygen/html/classitk_1_1ConvertPixelBuffer.html), which may be applied when an image is loaded by the `ImageFileReader`. For example if the ImageIO loads a 3 component pixel buffer and the ImageFileReader is templated over a scalar image default a RGB to Gray scale conversion occurs based on the CIE luminance for the 3-components.

Issue

This behavior was considered ok, as long as the image was not directly loaded in viz application such as Slicer, where unexpectedly the image would appears with a weird color scheme. (original bug report from Mihail Isakov)

BCL: After hearing the descriptions of the issue, I considered loading YBR directly as a bug because GDCMImageIO was reporting the pixel type as `IOPixelType::RGB` and the buffer was in YBR color space. If the image is loaded in YBR color space is should have a different `IOPixelType` may be `VECTOR` or `FIXEDARRAY` for lack of a more descriptive `IOPixelType`.

Notation

1. JPEG refers to the subset of ITU-T T.81, ISO/IEC IS 10918-1 (lossy 8bits)
2. JPEG 2000 refers to ITU-T T.800, ISO/IEC IS 15444-1

ITK 5.x

The behavior for YBR_FULL vs RGB color model has been discussed and it seems consensus would be to convert to RGB always. Since DICOM, JPEG and TIFF can be somewhat relates for this matter, we should strive to keep the behavior consistent.

Current behavior of ITK 5.0:

Photometric Interpretation	JPEG	TIFF	GDCM	DCMTK
MONOCHROME1	X	MONOCHROME2 (aka min-is-black)	MONOCHROME1	?MONOCHROME1?
MONOCHROME2	MONOCHROME2	MONOCHROME2 (aka min-is-black)	MONOCHROME2	MONOCHROME2
PALETTE COLOR	X	RGB (??)	Converted to RGB	?Converted to RGB? (need to check compressed LUT)
RGB	RGB	RGB	RGB	?RGB?
HSV
ARGB
CMYK
YBR_FULL	ITU 81 enforce sub-sampling	??	YBR_FULL	??
YBR_FULL_422	RGB	RGB	YBR_FULL	??Maybe RGB?
YBR_PARTIAL_422
YBR_PARTIAL_420
YBR_ICT	ITU 81 does not support this colorspace	??	RGB	no J2K plugin
YBR_RCT	ITU 81 does not support this colorspace	??	RGB	no J2K plugin

Desired behavior of ITK 5.0:

Photometric Interpretation	JPEG	TIFF	GDCM	DCMTK
MONOCHROME1	X	MONOCHROME2 (aka min-is-black)	MONOCHROME2	MONOCHROME2
MONOCHROME2	MONOCHROME2	MONOCHROME2 (aka min-is-black)	MONOCHROME2	MONOCHROME2
PALETTE COLOR	X	RGB	Converted to RGB	RGB
RGB	RGB	RGB	RGB	?RGB?
HSV
ARGB
CMYK
YBR_FULL	ITU 81 enforce sub-sampling	??	RGB	RGB
YBR_FULL_422	RGB	RGB	RGB	RGB
YBR_PARTIAL_422
YBR_PARTIAL_420
YBR_ICT			RGB	no J2K plugin
YBR_RCT			RGB	no J2K plugin

When a Photometric Interpretation is specified in red, it means this is not supposed to happen (does not make any sense). When a Photometric Interpretation is specified in silver background, it means the Photometric Interpretation is retired.

Discussion

Participants:

1. Mathieu Malaterre
2. Mihail Isakov
3. Bradley Lowekamp

For visualization people it is important that:

1. MONOCHROME1 (min-is-white) is converted to MONOCHROME2 (min-is-black)
2. YBR_FULL / YBR_FULL_422 are converted to RGB

The requirements for quantitative analysis may be different, please list reqs here:

1. ? Extract Luminance out of YBR ?
2. ? Need MONOCHROME1 native support, because <fill reason ??>
3. ? Is there a need to manipulated native palette data ? I doubt so
   1. BL: The `ExpandRGBPalette` attribute was just added to `ImageIOBase`. TIFFImageIO support this option. For feature consistency it would be nice for GDCMImageIO to support this feature. https://itk.org/Doxygen/html/classitk_1_1ImageIOBase.html#a04804c60d755ae102e104261536a0f46 https://github.com/InsightSoftwareConsortium/ITK/pull/1219/files

For ITK consistency it is important that the `ImageIOBase::PixelType` is set correctly so that users of the `ITK::ImageIOBase` API know generally what the image buffer is and how to load and use the image. We likely should improve the documentation in the ImageIOBase class about the expectations for how these are used. Here is a list of type pertinent to this discussion:

1. RGB
   1. BL: should only be used for general RGB buffers. This is not very descriptive of the color space sRGB, AdobeRGB, Linear RGB etc., but using this as a type for general "color" buffers should be avoided IMHO.
2. SCALAR
   1. BL: Images of unapplied palettes, when ExpandRGBPalette is false, are currently labeled as scalars
3. VECTOR
   1. BL: Currently this is frequently used for displacement fields.
4. FIXEDARRAY
   1. BL: Not regularly used to my knowledge, and may be the most generic multi-component `PixelType`, so this may be the base label to used for YBR, CMYK etc.

Data

1. JPEG 8 bits mono2
2. JPEG 8 bits YBR_422
3. JPEG 8 bits RGB
4. TIFF mono1 min-is-white
5. TIFF mono2 min-is-black
6. TIFF YBR_422
7. TIFF RGB
8. DICOM mono1 min-is-white
9. DICOM mono2 min-is-black
10. DICOM YBR
11. DICOM YBR_422
12. DICOM JPEG/YBR_422
13. DICOM RGB
14. DICOM YBR_RCT
15. DICOM YBR_ICT

References

• http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.3.html#sect_C.7.6.3.1.2
• http://gdcm.sourceforge.net/wiki/index.php/Color_Space_Transformations