Client Side Delivery

The goal of client side delivery is to provide functions that can be called from a python script to perform a parallel aggregation of data with the result obtained on the client. This goal is accomplished with a combination of new, modified, and existing filters. The new function paraview.fetch() in the paraview server python module automates the creation and execution of the results gathering pipeline.

Design

vtkSMGenericViewDisplayProxy

This unmodified filter orchestrates gathering results in paraview for the GUI. The filter internally uses a vtkReductionFilter to direct communications between the server nodes and a vtkClientServerModeData filter that transmits data from the server to the client. For our purposes we use vtkSMGenericViewDisplayProxy's FIRST_NODE_ONLY reduction type setting, which simply transmits data from the root node of the server to the client.

vtkReductionFilter

This modified filter does the communication needed to perform the aggregation. This filter defers all data manipulation to a user provided algorithm. The filter has been modified to run an additional user provided algorithm, which each node executes in parallel _before_ sending their data to the root node. This pregather algorithm is essential for scalability. After the root node obtains the intermediate results from all nodes it runs a user provided algorithm as before. Both pre and post gather algorithms can refer to the same object. The vtkReductionFilter was also modified to allow simple passthrough of data and to be able to pass though any one particular cluster node's data.

vtkMinMax

A new filter that performs an operation on the attribute data of its input. As of this writing the operation is one of MIN, MAX or SUM. The filter iterates through whatever datasets it is given and calls a templated operate function on each value. The templated function is necessary to avoid double conversion and loss of precision on non double valued arrays. This filter can take one or more input datasets on its first input port and always produces a single unstructured grid output that contains exactly one point and one cell.

paraview.fetch

A new function added to the paraview.py module that uses the above components to obtain data.

Example use

import paraview
#connect to a running pvserver with one or more nodes
paraview.ActiveConnection = paraview.Connect("localhost", 11111)

#create a sample data set
sphere = paraview.CreateProxy("sources", "SphereSource")
elev = paraview.CreateProxy("filters", "ElevationFilter")
elev.SetInput(sphere)

#choose the operation to perform in parallel
op = paraview.CreateProxy("filters", "MinMax")
op.SetOperation("MAX")
op.UpdateVTKObjects()

#do the fetch
out = paraview.fetch(elev, op)

#show the results
arr = out.GetPointData().GetArray("Elevation")
arr.GetValue(0)

Status

To progress further we need to better define the set of aggregation operations that we need. The min/max/sum operation is admittedly very simple and what makes it so is that it is geometry independent and simply throws away all of the input geometry it is given. Specific use cases that involve geometry aggregation are needed.