<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div>
<div>
<div>Mark,</div>
</div>
</div>
<div><br>
</div>
<div>That is more clear. I suspect the crash you are seeing is actually the opposite problem. Although you are correct in that there is not a magic scatter/gather in between every filter, the integrate attributes filter is special in that it is parallel aware.
At the end of its operation it does a reduce internal to the operation of the filter. It returns valid data in node 0 and an empty data set in all other nodes. I bet your crash occurs when you try to use one of these empty data sets.</div>
<div><br>
</div>
<div>-Ken</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span><Vanmoer>, Mark W <<a href="mailto:mvanmoer@illinois.edu">mvanmoer@illinois.edu</a>><br>
<span style="font-weight:bold">Date: </span>Friday, March 28, 2014 1:03 PM<br>
<span style="font-weight:bold">To: </span>Kenneth Moreland <<a href="mailto:kmorel@sandia.gov">kmorel@sandia.gov</a>>, David E DeMarle <<a href="mailto:dave.demarle@kitware.com">dave.demarle@kitware.com</a>><br>
<span style="font-weight:bold">Cc: </span>paraview <<a href="mailto:paraview@paraview.org">paraview@paraview.org</a>><br>
<span style="font-weight:bold">Subject: </span>[EXTERNAL] RE: [Paraview] parallel filter question<br>
</div>
<div><br>
</div>
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Ken and David,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">The end goal is to be able to chart the ratio of different volumes over a time series.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">I started with a hand built pipeline in the client: a wavelet source and a threshold. Attached to each is an Integrate Variables filter which provides
the volumes and the ratio is calculated with a Python calculator filter. Then I created a custom filter out of this pipeline. However I discovered that the custom filter is particular about how it is setup. E.g., since it was setup with a wavelet source, it
expects a wavelet source and if I attach it to a vtk image data set the result is garbage.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">That was when I figured I could write a VTK filter and load that as a plugin. I wrote a multiple input connection filter that creates two vtkIntegrateAttribute
ivars and calculates the ratio. Following vtkIntegrateAttributes, it outputs the result as a scalar attached to a vertex cell. This works great in serial. Crashes as soon as I attach it to a pipeline when I’m running in parallel.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">For some reason I had it in my head that each filter stage of the pipeline did a scatter and gather and that my ivars would magically be gathering
for me. But when I was reviewing the ParaView guide it finally sunk in that I was getting 2 vtkIntegrateAttribute ivars for each process and that the only gathering was the final image compositing.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Mark<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size: 11pt; font-family: Calibri, sans-serif; ">From:</span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif; "> Moreland, Kenneth [<a href="mailto:kmorel@sandia.gov">mailto:kmorel@sandia.gov</a>]
<br>
<b>Sent:</b> Friday, March 28, 2014 10:22 AM<br>
<b>To:</b> David E DeMarle; Vanmoer, Mark W<br>
<b>Cc:</b> paraview<br>
<b>Subject:</b> Re: [Paraview] parallel filter question<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">Mark,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">By the way, we might be able to answer your question better if you gave extra information about what you are trying to do. Short of writing a book on the
ins and outs of parallel processing in ParaView, we can only guess what you might need.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">Most filters, even though they are run in parallel only need to consider the local partition of data they are given. There are, of course, exceptions, but
if you give us some hints at what you are trying to do we might be able to give you better advice.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">-Ken<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: black; ">From:
</span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: black; ">David E DeMarle <</span><a href="mailto:dave.demarle@kitware.com"><span style="font-size: 11pt; font-family: Calibri, sans-serif; ">dave.demarle@kitware.com</span></a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: black; ">><br>
<b>Date: </b>Thursday, March 27, 2014 9:21 AM<br>
<b>To: </b>"Vanmoer, Mark W" <</span><a href="mailto:mvanmoer@illinois.edu"><span style="font-size: 11pt; font-family: Calibri, sans-serif; ">mvanmoer@illinois.edu</span></a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: black; ">><br>
<b>Cc: </b>paraview <</span><a href="mailto:paraview@paraview.org"><span style="font-size: 11pt; font-family: Calibri, sans-serif; ">paraview@paraview.org</span></a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: black; ">><br>
<b>Subject: </b>[EXTERNAL] Re: [Paraview] parallel filter question<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">Pretty much yeah. ParaView does interprocess communication as little as it can.
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">Grep for vtkMPIController in ParaViewCore/VTKExtensions and VTK/Filters/Parallel for examples of the mechanics of having parallel filters communicate amongst
themselves.<o:p></o:p></span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><br clear="all">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">David E DeMarle<br>
Kitware, Inc.<br>
R&D Engineer<br>
21 Corporate Drive<br>
Clifton Park, NY 12065-8662<br>
Phone: 518-881-4909<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">On Thu, Mar 27, 2014 at 11:00 AM, Vanmoer, Mark W <</span><a href="mailto:mvanmoer@illinois.edu" target="_blank"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">mvanmoer@illinois.edu</span></a><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; ">>
wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black">Hi all, I’m actually new to nuts and bolts parallel programming, so this question might be a bit basic. I was re-reading section 13.4 of the ParaView Guide
and wanted to clarify, is it correct to say that ParaView only does gathering for the image output that IceT is going to composite? That for filters that need to use any other data from the pipeline for non-image output, that gathering has to be done by hand?<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#888888"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:#888888">Mark<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><br>
_______________________________________________<br>
Powered by </span><a href="http://www.kitware.com" target="_blank"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">www.kitware.com</span></a><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><br>
<br>
Visit other Kitware open-source projects at </span><a href="http://www.kitware.com/opensource/opensource.html" target="_blank"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">http://www.kitware.com/opensource/opensource.html</span></a><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: </span><a href="http://paraview.org/Wiki/ParaView" target="_blank"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">http://paraview.org/Wiki/ParaView</span></a><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
</span><a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; ">http://www.paraview.org/mailman/listinfo/paraview</span></a><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p></o:p></span></p>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; color: black; "><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>