ITK Release 4/Discussion Points

Discussion Points

Filenames and std::strings

• Strings are std::strings: Filenames should be std::strings, not const char* _arg
  • Set character string. Creates member Set"name"() (e.g., SetFilename(char *)). The macro assumes that the class member (name) is declared a type std::string. */

Threading model modifications

1. Thread pools?
2. Hierarchal thread handlers?
3. Can threads deal with pushing data across hardware units with inconsistent memory models (i.e. 4 core SMP + GPU#1 + GPU#2)

Need Explicit Transition Documentation

1. To facilitate transition from ITK3 to ITK4, a set of migration documents and tools will be beneficial to researchers who will be migrating to their tools to ITK4. As we adapt the reference applications to the changes occurring in ITK4, we will document the necessary source code changes to use the new version of ITK4. When possible, we will write a set of scripting tools that transform ITK3 to ITK4, or identify old code that has become invalidated, and suggest alternative options. We will deliver this documentation both as a standalone document and as an online WIKI-based web resource.

Transforms

1. Need ability to deal with concatenated physical space mappings (Rigid-»Affine-»BSpline-»Displacement-»Rigid-» etc)
2. Better protocols for reading/writing/ transforms including the ambiguous way that compound (i.e. not concatenated) transforms are represented.

Image Physical Space/Domain

1. There is a need an image layer that does NOT have a physical space representation
   1. ImageBase-»DigitalImage-»ContinuousImage-»[Image(the current interpretation)|GeoSpatialImage|Other physical space]
   2. Most PixelWise filters will be defined over the DigitalImage layer to explicitly identify them as not dealing with physical space
   3. Digital to Continuous conversion API is consistent (TransformIndexToPhysical & PhysicalToIndexTransform), but may be adaptable to address the needs of our expanding user base
   4. The current "Image" designator is explicitly defined as an (LPS anatomical, East|North|Altitude) with Direction cosigns, spacing, origin and appropriate
   5. Units: A designation that can hint to the application level how to interpret physical space. Perhaps the filters would just verify that units are the same. (default being non-existing and therefore the same).

Platform support

• a platform is a [Hardware, OS, Compiler, Compiler options]

1. Reduce build burden -- Explicit Templates & Pre-compiled headers:
   1. Under the assumption that the scope of the simple & wrapping frameworks will define a large swath of instantiations that almost all applications also need. By defining and pre-compling those common (perhaps driven by the wrapping choices) tools, compiling the application layers will be significantly faster. This should reduce the burden on application developers by shifting build time costs away form the time of building the applications.
   2. We do have to be careful to limit this (or provide options to limit this) so that the default build does not take 10 hours.
   3. This does imply that the compiler MUST support explicit instantiation of templates
   4. ---- As a side note, perhaps we also need to investigate pre-compiled header support for compilers to lower the development burden
2. An explicit deprecation cycle for ALL compilers
   1. It is CRUCIAL that we define upfront a MINIMUM time frame for how long each version of ITK4 we commit to supporting all platforms. It is expected that these dates will shift into the future, but not earlier.
      1. Windows 7, Visual Studio 2010 will be supported at least until Jan 1, 2017
      2. Windows XP, Visual Studio Express 8 will be supported at least until Jan 1, 2015
      3. Linux gcc version 4 (greater than 4.0.1) will be supported at least until Jan 1, 2012
3. Compilers need to be compliant with the C++03 standards. http://en.wikipedia.org/wiki/C%2B%2B

IO interface changes

1. Transforms components "Can be written", but there is insufficient information to un-ambiguously derive the intent and appropriate use of those transform parameters. In particular, the parameters for BSpline are written to disk in two components, and the BSpline parameters are linearly written, but the implied grid for those parameters is not specified in the output file. As a use case, the BSpline transform type SHOULD be able to be instantiated from just the information in the file.
2. Are there IO factory simplifications that could be done? i.e. remove separatedFactories for each type

Meta Data Dictionary

1. MetaData Dictionary items need "Traits" to help identify the "social status" of the element (such as ITK_SUPPORTED_PUBLIC, ITK_SUPPORTED_PRIVATE, UNSUPPORTED_PRIVATE, UNSUPPORTED_PUBLIC).
2. The first contribution of the Iowa ITK development team was to introduce the “MetaDataDictionary” storage mechanism to all ITK Objects that could contain arbitrary key-value pairs of data. The MetaDataDictionary was not part of the original ITK framework definition, and has therefore not been consistently implemented throughout the ITK toolkit. We propose to revisit the implementation of the MetaDataDictionary to improve and simplify the ABI so that it is easier to use. In particular, the Encapsulate() and Expose() functions can now be simplified when using standards-compliant C++ compilers, and new features for describing a dynamically created dictionary at run time will be introduced. A common use of the MetaDataDictionary has been to temporally append features that were missing in the initial ITK framework. In many cases, the use of the MetaDataDictionary was subsequently followed by addition of formal support of the feature to ITK, and this lead to redundancy of information that was not always kept synchronized. Additionally, as new developers adopted the MetaDataDictionary for different uses, they each created inconsistent definitions for similar uses of the dictionary. We will review the internal ITK uses of the MetaDataDictionary, remove uses that are now redundant with formal accepted functionality, and consolidate similar uses of the Dictionary. The MetaDataDictionary is most commonly used to pass additional information to or from the Input/Output mechanisms. We will define a protocol so that data object histories can be tracked through consistent instrumentation of ITK process objects to tag output objects. We will review the Input/Output and develop a standard practices guideline for how to map the extra data to and from ITK in a consistent way. Wherever possible, we will use tags and definitions consistent with the dictionaries specified in the DICOM standard.

ITK Orientation

1. Instrument the itk::ImageIO methods to record spatial image orientation in the MetaDataDictionary prior to it’s inclusion as a formal feature. This solution allowed representation of which of 48 possible 3D orientations the image was in. This was represented symbolically by 3 letter combinations, based on permuting (A)nterior/(Posterier), (I)nferior/(S)uperior, (L)eft/(R)ight. For example RAS meant that the voxels are oriented Right-to-Left, Anterior-to-Posterior, Superior to Inferior, corresponding to Row/Column/Slice. This method of specifying orientation is useful but very limited; many anatomical scanners allow scanning at oblique angles with respect to anatomy. So after long discussion in the ITK developer community, a more general method was developed, which represented the orientation as a direction cosine matrix (DCM). The direction cosines specify a general 3D rotation of the image voxels with respect to the anatomy being scanned. The long experience with brain imaging research at Iowa will inform our implementation of these changes. However, we also recognize and highly value the expertise of the other members of the ITK developer community, and the implementation will be guided by the consensus judgment of the community as a whole. We will enhance the code infrastructure of ITK with respect to issues of position and orientation of images. Deliverables will comprise both changes to the core ITK library code, and extensive regression tests to verify correct operation with the rest of the ITK Library. There are three ways in which the handling of orientation in ITK needs to be improved:
2. Currently the size of the direction cosines is linked to the number of dimensions in the image – for a 2D Image, they comprise a 2x2 DCM, for a 3D Image 3x3, etc. Conceptually, the use of the current orientation description is most suited for describing images with exactly 3 spatial dimensions. We believe this needs to be changed and generalized to accommodate describing data sets within a higher dimensional space, or with respect measurement direction that is different from scanning direction. First, any 2D scan of anatomy will have an implied third axes, parallel to the direction of image acquisition. In particular, 2D DICOM slice images always report the three-dimensional image position of the scan with respect to the patient. At present, when ITK reads a single 2D DICOM image, there is no way to preserve it’s relationship to 3D-space. Finally, the orientation of a measurement frame with respect to sub-volumes is necessary to properly represent complex acquisition types (such as diffusion weighted imaging). Similarly, 2D Images can have a three-dimensional image origin with a well-defined anatomical meaning. Discarding that when reading the image is a loss of information. In both cases (orientation and origin) it will be necessary to enforce backwards compatibility. This will be achieved in two ways: by adding new class methods that are explicitly three-dimensional, and by careful testing of all ITK classes to ensure their continued correct operation.

FFT

• Need more FFT adaptors, and a slight refactoring of FFT adaptor base to meet several platform

1. It would be nice to have a compliant FFT built in with reasonable performance (vnl does not work)
2. Mac Accelerate framework, Intel MLK,

Numerics

• Need adaptors for some subsets of (SVD/LAPACK/BLAS) numerics:

1. Should be able to take advantage of the Mac Acclerate, Intel MLK, Atlas or others with minimal end user configuration

Neural Networks

• Remove NeuralNetwork

1. The current implementation in ITK requires that the network architecture and Input/Output routines be completely defined at compile-time. This severely limits the utility of the current implementation. The Input/Output mechanism of the current Neural Network currently only supports a proprietary network file format. This limits the ability to analyze the networks with complex external tools such as Matlab Neural Network toolkit.

• Consider linking with FANN

Spatial Objects

• Use cases
  1. Load, save, and manipulate segmentation results (multiple objects) produced by different methods
     • multiple objects = labelmaps, parametric object representations, etc in a common coordinate/reference system
     • different methods = segmentation, registration, simulation
     • manipulate = register, send to visualization package, use as a mask for subsequent processing
  2. Load, save, and manipulate multiple instances of the same object from multiple people
     • e.g., use a collection of objects to conduct a population-based study
  3. Load, save, and manipulate multiple objects over time and space
  4. Represent objects in a hierarchy
     • tree of tubes, liver contains vessels, ...
     • perhaps different views of the same collection of objects should be possible (e.g., all vessels -vs- liver's vessels)
• Changes needed
  • Support ND Objects
  • Clean API to remove secondary/obscure effects (e.g., tree depth of -1 means the entire tree)
  • Support time
• Wish-list
  • Have parametric objects that can be fit using itk optimizers
  • Consider 3rd-party compatibility (wrapping or data exchange)
• Notes
  • Consider lessons learned from 3rd-party solutions (strengths and weaknesses)
  • Related 3rd-party solutions
  • MRML
  • ghost (haptics scene graph from Phantom)
  • OpenInventor
  • OpenCV (?)
  • SOFA
  • VTK (?)
  • SOViewers
• Spatial Object Refactoring project page

Coding style, and enforcement

1. There should be some system somewhere that is compliant with the ITK coding style. It would be preferable if the primary development editors (VS, vim, emacs) could all be configured to support the style consistently.
2. The uncrustify program MAY allow a conduit for helping new developers format to be KWSTYLE compliant.

Pipelines and Objects

• All process object outputs should be available as data objects (including native types)

1. There are times when a filter computes a value that is used as input into later stages of the pipeline, there needs to be a way to easily link filters and have the pipeline work properly even when the linkage is a simple integer.

Memory

• Should the default behavior of all filters be to release their input data

1. This may address the complaint that ITK has unacceptable memory usage.

Code Clean-up

• All class objects that end in "2" as a mechanism to fix bugs, and not break backwards compatibility should be removed.
• itk_hashtable.h should be investigated to determine if a more standard implementation could replace the existing version. The current version has many different implementations for dealing with different compiler bases.

CMake Flag Clean-up

1. Need to review the ITK_LEAN_AND_MEAN flags
   • Should this be controled by a CMakeFlag? It used to be bound to the BORLAND compiler defintions, but since that is now gone, it's use is less clear.
   • What is the intended purpose?
   • Can this be formalized better?