Selection and Data Analysis

There has been much work done already on selection in ParaViewIII. Previous discussion includes Selection_Use_Cases, Selection_In_ParaQ, Selection, Selection_Implementation and Integrating_Interactive_Selection. Several improvements need to be made before the upcoming full release in order to improve useability and fully support quantitative analysis. To put it simply, we want ParaView to have intuitive selection capabilities (note the plural) and we want it to be easy to filter and/or perform quantitative analysis on the results of a selection. Previous discussion of quantitative analysis in paraview includes Plotting Parallel Data (PGraph), PGraph_Implementation, Plot_Data_Pipeline, Statistics_View, Plot_View_Management, Data_Analysis_Design and Client_Side_Delivery.

The feature wish list

• Pick point in space (on surface) or type coords in.

• Picking by global ids, pedigree id, proc+cell/pt id.
  • choose a list of IDs manually. We'll start by letting user pick a single id.
  • manually refine the list of selected ids
  • extend the element inspector to do this (or possibly the Statistics_View). We need to use ranges so that we don't explode millions of entries on a big dataset. This is especially true with frustum selection.

• Frustum (volume/deep) selection - this was in long long ago. It used vtkFrustumExtractor to extract all pts/cells in a frustum. GUI interface to choose what type of selection through a tiny model drop down next to selection button. {surface, frustum, or single point}

• Proposed updates to element inspector:
  • ids shown
  • enter in an id (or ids)
  • tab formating
  • grouped ranges instead of one line per element.

• Tie histogram to selection - choose bars in histogram and use the output as input to a threshold filter.

• Saving and reusing selection:
  • after making a selection we should be able to reuse it
    • ex 1) applying a filter to just the output of the selection
    • ex 2) changing the resolution of a source that was selected and getting more cells out
  • Should we save output of selection or input of selection? If we save the input to selection for surface selection, the results will be meaningless if the camera has been changed.We will save the output of the selection operator and reextract as necessary see [[Selection Types].

• Show selections in the pipeline view so we can apply multiple filters to the output of a selection. State saving and restoring may become an issue. We need to save all inputs to the selection to recreate.

• Need to be able to make props unpickable in the gui somehow.
  • low level picking classes can do this, need to expose the pickable property on the GUI
  • pipeline browser - seems natural to relate this to the EYE
  • for masking the existing pickable property will not be enough (if unpickable it is ignored entirely). Need a new actor flag that says render me as background/invisible or with ids.

• Data analysis - output of a selection should be immedately available for quantitative discovery.
  • many filters will produce 1D rectgrid, all of which can be plotted
  • polydata with one vertex - example output of vtkMinMax
  • prefered/compatible viewtypes in XML say what output of each filter/source can and prefers to be. Have icons in pipeline browser that lets you hide/show each one.

• Anchoring selection in space or to ids (plot vals at location over time or plot vals of an id over time).

• Choice of interpolation or using nearest neighbor (probe vs pick).

• On screen id and value labelling - draw ids in the render window.
  • Have to be careful to use global ids in parallel for piece invariance.
  • Have to be careful not to clutter view with too much text - n% of pixels get a label.
  • Great to be able to display any particular attribute value (not just ids).

• Have to be able to select blocks within a multiblock dataset.

• Multiple selections sub selections, refining selections, adding to selections. These shouldn't be hard to implement because the vtkSelection structure is a tree structure.

How it Works Now

VTK level (see also Selection_Implementation)

• vtkSelection - A (node in a) selection tree. Stores the parameters of a selection operation.

• vtkExtractCells Extracts cells with matching Ids from any type of dataset.

• vtkFrustumExtractor Extract cells within a frustum from any type of dataset.

• vtkVisibleCellExtractor Use color buffer selection to extract visible polygonal suface cells.

ParaView ServerManager level (see also Selection_Implementation)

• vtkMultiBlockExtractSelection Run vtkExtractCells on multiple data sets.

• vtkPVSelectionInformation Gathers selection information from the server to the client.

• vtkSelectionSerializer Serialize/deserialize vtkSelection to/from xml.

• vtkSelectionConverter Converts from polygonal surface cell selection to one level deep 3D cell selection.

• vtkPVVisibleCellSelector ParaView version of vtkVisibleCellExtractor.

• vtkVolumeSelector ParaView vertsion of vtkFrustumExtractor.

• vtkSMSelectionProxy A proxy for a vtkSelection and logic to run an algorithm to create it.

ParaView3 Qt Client level (see also Selection_In_ParaQ)

• MainWindow and pqMainWindowCore Client main. These share a pqSelectionManager and call it in response to UI events.

Qt level selections. Selection portions of "Models" that are synchronized and shared between "Views" of the Models.

• QItemSelectionModel
• QItemSelection
• QModelIndex

These correspond to the ParaView3 Qt/Client level selections:

• pqServerManagerModel Represents the state of vtkSMProxyManager to the GUI.
• pqServerManagerSelectionModel A list of selected or deselected pqServerManagerModelItems.
• pqServerManagerModelItem A selected proxy. Child classes are: pqProxy (pq wrapped SMProxy) and pqServer (a server connection)

• pqSelectionAdaptor Abstract base class for adaptors that link a QItemSelectionModel to a pqServerManagerSelectionModel so that paraview selections are automatically synchonized by Qt. Child classes are:
  • pqDataInformationModelSelectionAdaptor The selection of cells and points shown in the statistics view.
  • pqPipelineModelSelectionAdaptor The selected filters highlighted in the pipeline view.

• pqElementInspectorWidget The view that shows high level information about data arrays. This is somewhat similar to the statistics view but it does not have a pq*ModelSelectionAdaptor. Instead it just watches the pqSelectionManager's onSelectionChanged() slot.

• pqSelectionManager - The guts of selection in ParaView3. The main entry points into this class are the updateSelection() and updateSelections() methods which together are a two pass algorithm for making a selection and for converting it into a Qt level structure that can be shared amongst views. It has a pqSelectionManagerImplementation inside which stores the:
  • mouse button press observer,
  • selection parameters,
  • saved interactor styles,
  • ServerSelections - for each server a SelectionDataSource (a MBESel and the most recent vtkSMSelection)
  • Displays - for each view window a data object display for the selected cells
  • ClientSideDisplays - includes saved source proxys, the MBESel extractor and the GenericViewDisplay for the selected cells.

Data flow:

User makes a selection in the UI and the MainWindow passes GUI and mouse button events to pqSelectionManager.

• pqSelectionManager::updateSelection() - prepares to make a vtk level selection happen. This sets the servermanager state needed to make a selection.
  • creates a vtkMultiBlockExtractSelection MBESel (if none created yet)
  • creates a new vtkSMSelectionProxy
  • tells the vtkSMSelectionProxy what screen rectangle to select in
  • pqSelectionManager::setActiveSelection() - makes the the vtkSMSelectionProxy the Active Selection
    • connects the vtkSMSelectionProxy to the input to the MBESel
    • swaps out (unregisters) old selection proxy and swaps in (registers) the new
    • pqSelectionManager::createDisplayProxies() - creates displays for the results, but these get deleted later.
  • deletes the new vtkSMSelectionProxy
  • emits the selectionMarked signal which eventually switches the selection mode back to INTERACTION.

Now we are ready to make a selection. We can also get to this point when the program directly sets the servermanager state during redo or state playback. In any case when a selection proxy is registered in the servermanager the following happens.

• pqSelectionManager::updateSelections() - checks if anybody changed a selection proxy, and if so executes it, and then converts the vtk level selection results to a paraview and Qt level selection
  • vtkSMSelectionProxy()::UpdateSelection()
    • tells the renderer to do a color buffer selection (vtkSMRenderManager::SelectVisibleCells())
    • uses a vtkSelectionConverter to convert vtk level results (actor pointers) to paraview level results (client server ids of filter proxies), and to find the 3D "face" cells that produced each selected polygon in the shell (reverse vtkDataSetSurfaceFilter).
  • the MBESel executes. It takes the vtkSelection it is given containing a list of cell ids in various datasets and runs each dataset through vtkExtractSelection to extract those cells
  • pqSelectionManager::selectionChanged() - converts the paraview level results into a pqServerManagerSelectionModel now Qt can update the views in several windows.
  • pqSelectionManager::clearClientDisplays() - I told you so.
  • pqSelectionManager::createNewClientDisplays() - shows the selection
    • runs multiblock results from MBESel through vtkExtractBlockFromSelection to produce an ugrid for each selected dataset.
    • makes a vtkGenericViewDisplayProxy for the ugrid that will render in pink wireframe
    • tells the view modules to update so that they show the newly created actors for the wireframes

Selection Types

A vtkSelection is an abstract container for selection information. It is structured as a tree where each node has an vtkDataArray containing data, a vtkInformation object containing keys which explain the meaning of the data, and pointers to child nodes. The output of the color buffer selection algorithm (vtkVisibleCellSelector) contains nodes that have processor and actor indices and data arrays that contain cell and point indices. Paraview transforms this into nodes that contain client server ids and cell and point ids. To support the new features we are going to add to selection we need to flesh out the selection data structure to contain additional types of selection information and add additional selection extraction and conversion filters.

The proposed expansion to the content of the selection types are:

• Ids - These are mostly supported at present.
  • surface ids (aka output of visiblecellselector) proc+cell/pt ids
  • client server ids
  • globalids or pedigreeids similar to ids
• Frustum - bounds of a region of 3D space.
• Point in space - a single surface or interior point.
• Thresholds - data attribute values which can be extracted with a vtkThreshold filter.

Should we also except multiple ranges of data values? Such is possible with the histogram/barchart view, but not with the vtkThreshold filter.

• --Ken 17:14, 7 Feb 2007 (EST)

Yes we should.

• --DaveDemarle 14:43, 9 Feb 2007 (EST)

Furthermore we will move vtkSelection from being a subclass of vtkObject to vtkDataObject. This will allow use to easily pass selections through the pipeline and make it easier to create filters that manipulate selections. This should help to streamline our existing selection and extraction infrastructure and make it more easy to add features in the future. In the future it is likely that vtkPicker and its subclasses will become filters that produce vtkSelections.

Extraction Filters

The user will create selections in several ways.

• selecting a point on the screen (similar to hitting 'p' in ParaView2's picker)
• drawing a rubber band on the screen, which will create a frustum or id type selection.
• entering ids in a form
• clicking on bars in a histogram view

These selections will then be saved and the data they identify will be extracted by running one of several new filters. An extraction filter will take two inputs, a data set and an vtkSelection, and produce some output. A selection conversion filter will take in vtkSelections and produce a modified selection. Two examples are selection intersection and selection union filters. Each selection filter can inspect the selection meaning keys and decide if it can handle to given selection. A frustum extraction filter will not produce any output when given a vtkSelection node with threshold information. In ParaView3, each selectable dataset will be attached to one or more selection algorithms and on each output these algorithms will produce new datasets.

Note that each extraction filter takes 1 dataset and 1 selection as input. In the cases where multiple datasets are selected, ParaView will create and maintain multiple extraction filters. The following diagram shows the planned set of extraction filters and their major features.

One goal of this redesign is to clarify how an application decides what objects are connected to what selection ids. In general the interpretation of the selection contents within various applications is the responsibility of the application developer. For example, at the VTK level there is presently a bare minimum level of support for finding the particular data set corresponding to an actor Id by calling vtkVisibleCellSelector's GetActorFromId(). This relies on a lookup table that the vtkRenderer builds only during the color buffer selection render pass. It must be used very carefully to ensure correctness. ParaViewIII does this correctly and its selection code maps that pointer to a client server id that corresponds to a particular algorithm's output. With selections and extraction filters linked directly to vtkDataSets, it is hoped that application developers will have an easier job of reliably interpretating a selection result.

There seems to be an awful lot of different filters to do pretty much the same thing. At worst, it looks like there can be a separate filter for each possible selection-type data-type pair. There appears to be separate filters to use if the selection happens to be ids, frustums, or thresholds.

I recommend building into the vtkSelection class a query mechanism that filters and other components can use to get the selected cells. For example, there could be a method that takes a vtkDataSet and a vtkTypeIdArray and fills the array with all the (local) ids of cells in vtkDataSet that are marked by the vtkSelection. That way, the extract filter can be agnostic to the type of selection being employed. At worst, there would need to be one type of extract filter for every type of input, and these filters could actually be combined into a single filter that changed its behavior based on the input type.

--Ken 17:10, 7 Feb 2007 (EST)

I don't want to put the actual extraction into the vtkSelection. We want to more clearly separate the process of taking out the data from the information about what to take out. So the vtkSelection will become a vtkDataObject (the information) and the process of taking the data out will be these new algorithms. About the number of filters, the vtkExtractSelection will have a switch inside to call one of the other core filters. And I agree that we should keep down the number of filters, so we will have 5, one for each extraction type. The implementation of the different extraction operations is sufficiently different to justify one filter each.

--DaveDemarle 14:43, 9 Feb 2007 (EST)

Integration with the GUI

The section will eventually describe how selection results are given to the client and used by the GUI.

Full case list

We would like to be able to complete the following sentence taking any path (except where it doesn't make any sense). Please add to / modify this list with additional cases.

Given any vtkDataObject subclass, such as ...

• vtkPolyData
• vtkImageData
• vtkUnstructuredGrid
• vtkGraph
• vtkTree
• vtkTable

... select the ...

• cells with associated points
• points only
• points with associated cells
• rows (for a vtkTable)

... which are ...

• inside a frustum
• among a list of ids
• within thresholds in a data array
• at or close to a set of world points
• inside a bounding box

... and give the result as ...

• a boolean array attached to the data object.
• a vtkSelection of type "ids" (the most general selection type).
• an extracted dataset (possibly converted to a more generic data type, e.g. vtkUnstructuredGrid).

The following are some conversions from original type to extracted type:

• vtkPolyData -> vtkPolyData
• vtkImageData -> vtkUnstructuredGrid
• vtkUnstructuredGrid -> vtkUnstructuredGrid
• vtkTree -> vtkGraph
• vtkGraph -> vtkGraph
• vtkTable -> vtkTable

Ideas

Polydata painter backdoor to draw selected cells quickly. For very fast highlighting switch in a selection painter which draws the Id selected polygons with just the pink outline.

Boolean operations on selections: implement as tree traversals or on the vtkInsidedness array?

Selection tree: Why did we make it a tree? Probably for multiblock. We should make the convention that the trees will be flat unless multiblock, in which case the multiblock data set gets a sub tree.

Group structured data selections into mulitblock of largest possible regular regions within the selection (bottom up build of neighbor cells).

It may make sense to also have a "box" selection type which has x,y,z min/max bounds. Then the filter vtkKdTreeSelector would take a box selection as input and convert it to a "ids" selection on output.

Current development plan

At Kitware we were discussing what we need to do to get selection and data analysis working before the 3.0 release. Because there is much to do we are focussing on adding as little as possible to get the 2.6 features in. We are also going to spend time making sure that it is reliable. In the process we are going to lay the groundwork for planned near term improvements.

Our plans are:

• disable selection of multiple datasets at the same time - this will simplify the code and the user interface greatly
• disable frustum selection - because this has the potential to bring everything to the client
• disable threshold selection - for the same reason as frustum selection
• make selection results reusable:

    create a SelectionSource class
    add vtkPVExtractSelection to the filters menu
    when pvExtractSelection is applied the currently selected data will be put into a SelectionSource class and the source and the filter will be put into the pipeline browser

• make the property that contains the list of ids in the selectionsource editable on the property page
• make Selection by ID use the vtkGlobalIds array when it is available
• enable Selection of Points by id
• create a filter that uses the new time support in the vtk pipeline to accumulate over time

   this will work by giving each point and cell its own set of attribute arrays, this will enable plotting multiple values over time but this feature will be disabled and only one point/cell will be plottable over time

• make the extract filters request the whole extent from their inputs (because pieces are not guaranteed to coincide in space)
• address bugs

  selection with no geometry view crashes
  selection of custom filter
  selection of superquadric
  select ids out of range crash
  select location in 2D cells
  selection in parallel client server bug - turn on compositing during selection unless user has turned compositing off in which case we will disable it