Client Side Delivery

The goal of client side delivery is to provide functions that can be called from a python script to perform a parallel aggregation of data with the result obtained on the client. This goal is accomplished with a combination of new, modified, and existing filters. A python module automates the creation and execution of the results gathering pipeline.

Design

vtkSMGenericViewDisplayProxy

This unmodified filter orchestrates gathering results in paraview for the GUI. The filter internally uses a vtkReductionFilter to direct communications between the server nodes and a vtkClientServerModeData filter that transmits data from the server to the client. For our purposes we use vtkSMGenericViewDisplayProxy's FIRST_NODE_ONLY reduction type setting, which simply transmits data from the root node of the server to the client.

vtkReductionFilter

This modified filter does the communication needed to perform the aggregation. This filter defers all data manipulation to a user provided algorithm. The filter has been modified to run an additional user provided algorithm, which each node executes in parallel _before_ sending their data to the root node. This pregather algorithm is essential for scalability. After the root node obtains the intermediate results from all nodes it runs a user provided algorithm as before. Both pre and post gather algorithms can refer to the same object. The vtkReductionFilter was also modified to allow simple passthrough of data and to be able to pass though any one particular cluster node's data.

vtkMinMax

A new filter that performs an operation on the attribute data of its input. This filter iterates through whatever datasets it is given and calls a templated operate function on each value. The templated function is necessary to avoid double conversion and loss of precision on non double valued arrays. This filter can take one or more input datasets on its first input port and always produces a single unstructured grid output that contains exactly one point and one cell.

fetcher.py

A python module that automates the collection of data to the client in a pvpython session. The code for the module is included here:

# fetcher.py

"""
This module contains a utility to automate results gathering from the paraview
python client.
"""

import paraview

def fetch(myinput, operator, node=-1):
   """ 
   A convenience method that instantiates a pipeline that applies the given
   operator to myinput and returns the result to the client.
   The optional node argument when set to -1 applies to operation to all server
   nodes, or when set to a number returns the results from that node.
   """
   
   #create the reduction pipeline
   gvd = paraview.CreateProxy("displays", "GenericViewDisplay")
   #connect gvd to the desired input
   gvd.SetInput(myinput)
   
   #tell gvd that its vtkReductionFilter should apply the given operator   
   gvd.SetPreReductionHelper(operator)
   gvd.SetReductionHelper(operator)
   
   #tell gvd to select either one, or all nodes to gather
   gvd.SetPassThrough(node)
   #results are always gathered to the root before sending to client
   gvd.SetReductionType(3)   
   
   #go!
   gvd.UpdateVTKObjects()
   gvd.Update()
   
   return gvd.GetOutput()

Example use

import sys
sys.path.append('locationof/fetcher.py')
import paraview, fetcher
#connect to a running pvserver with one or more nodes
paraview.ActiveConnection = paraview.Connect("localhost", 11111)

#create a sample data set
sphere = paraview.CreateProxy("sources", "SphereSource")
elev = paraview.CreateProxy("filters", "ElevationFilter")
elev.SetInput(sphere)

#choose the operation to perform in parallel
op = paraview.CreateProxy("filters", "MinMax")
op.SetOperation(1) #MAX
op.UpdateVTKObjects()

#do the fetch
out = fetcher.fetch(elev, op)

#show the results
out = fetch.GetOutput()
arr = out.GetPointData().GetArray("Elevation")
arr.GetValue(0)