Yeah, for structured grids you'll need to set the extent on each process as well as use <span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">vtkCPInputDataDescription::SetWholeExtent() to specify the global grid extent. The partitioned arrays in addfields() for single component tuples needs to be the same length as the number of points/cells on the local, partitioned grid. <br>
<br>The extent support for coprocessing is a git dodgy as we're trying to do things in a way that the pipeline doesn't like. One solution to that is to use a vtkMultiPieceDataSet but I haven't tried that enough yet to see how well it works.<br>
<br>Andy<br></span><br><div class="gmail_quote">On Tue, Sep 25, 2012 at 6:12 PM, Vanmoer, Mark W <span dir="ltr"><<a href="mailto:mvanmoer@illinois.edu" target="_blank">mvanmoer@illinois.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Ah, apparently I’m not setting the extents with SetExtent(), vtkCPInputDataDescription::SetWholeExtent() as mentioned on slide 79 of the SC12 slides.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I’ve been working off of the Phasta example. So if I understand the general idea right, I need to be sending the extent information from the application to
the coprocessor call, then<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I need to make sure that those also get passed to the image data set creator, and I need to also send the partitioned arrays to the addfields_() call?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Mark<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Vanmoer, Mark W
<br>
<b>Sent:</b> Tuesday, September 25, 2012 3:28 PM</span></p><div class="im"><br>
<b>To:</b> 'Andy Bauer'<br>
<b>Cc:</b> <a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a><br>
<b>Subject:</b> RE: [Paraview] Python interpreter not being created for coprocessing?<u></u><u></u></div><p></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Right, single process. I’m working with one code that’s all CPU and another that’s mixed CPU/GPU. Coprocessing works well for both. There’s also MPI versions
of the codes that I’m trying to instrument. <u></u><u></u></span></p><div class="im">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The computational mesh is completely regular so I’ve been using vtkImageData. The MPI versions split the mesh over the ranks. When I run an MPI version with
–np > 1, then I get the following for each process:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">ERROR: In /home/mvanmoer/builds/ParaView/ParaView-3.14.1-Source/ParaViewCore/VTKExtensions/vtkKdTreeGenerator.cxx, line 168<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">vtkKdTreeGenerator (0x5ac6430): RegionIDs cannot be 0.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">It doesn’t crash, but it gives an incorrect image. I’ll look through your updated tutorials, I’d been looking through the SC10 slides.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
</div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Mark<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Andy Bauer
<a href="mailto:[mailto:andy.bauer@kitware.com]" target="_blank">[mailto:andy.bauer@kitware.com]</a>
<br>
<b>Sent:</b> Tuesday, September 25, 2012 2:15 PM</span></p><div><div class="h5"><br>
<b>To:</b> Vanmoer, Mark W<br>
<b>Cc:</b> <a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a><br>
<b>Subject:</b> Re: [Paraview] Python interpreter not being created for coprocessing?<u></u><u></u></div></div><p></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Tue, Sep 25, 2012 at 12:21 PM, Vanmoer, Mark W <<a href="mailto:mvanmoer@illinois.edu" target="_blank">mvanmoer@illinois.edu</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I pulled it directly from stage/13460_CP_FortranAdaptor. </span><u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><br>
Great, then it was me after all that fixed it!<br>
<u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I’m now at the point where it runs great on a single node with GPUs, I’m trying to extend it to multiple
nodes, which I think involves setting up vtkMultiBlockDataSets? I was going to look through the NPICAdaptor for hints on that.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><br>
By single node I'm assuming that you mean a single process, correct? What's the type of grid that you're planning on using? The PhastaAdaptor is an example of dealing with an unstructured grid. I also just updated the coprocessing tutorials page (now at
<a href="http://paraview.org/Wiki/Coprocessing_Tutorials" target="_blank">http://paraview.org/Wiki/Coprocessing_Tutorials</a>) to include the tutorial I'll be doing at SC12. It should have some useful information for doing this. You could just do a polydata input, just set
the center of the sphere source in your example based on the process rank (e.g. Sphere->SetCenter(10*rank, 0, 0); ) so that it's easy to tell the difference between each processes input part/domain partition.<br>
<br>
Andy<br>
<u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Mark</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Andy Bauer [mailto:<a href="mailto:andy.bauer@kitware.com" target="_blank">andy.bauer@kitware.com</a>]
<br>
<b>Sent:</b> Tuesday, September 25, 2012 10:13 AM</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><br>
<b>To:</b> Vanmoer, Mark W<br>
<b>Cc:</b> <a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a><br>
<b>Subject:</b> Re: [Paraview] Python interpreter not being created for coprocessing?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Did you get my changes from ParaView's stage branch? My changes didn't make it into ParaView master yet so I'm not sure what changed that caused things to work now. Well, if it's working
I'm glad -- I'm not sure I can personally take credit for it though :)<br>
<br>
Let me know how it goes with the GPU stuff.<br>
<br>
Andy<u></u><u></u></p>
<div>
<p class="MsoNormal">On Mon, Sep 24, 2012 at 5:48 PM, Vanmoer, Mark W <<a href="mailto:mvanmoer@illinois.edu" target="_blank">mvanmoer@illinois.edu</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Andy,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I’ve tested this and it works with both gcc/mpich2 and pgi/openmpi.
</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks again!</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Mark</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Vanmoer, Mark W
<br>
<b>Sent:</b> Wednesday, September 19, 2012 1:30 PM<br>
<b>To:</b> Andy Bauer<br>
<b>Cc:</b> <a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a><br>
<b>Subject:</b> RE: [Paraview] Python interpreter not being created for coprocessing?</span><u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hi Andy, that sounds like some deep debugging, thanks for your effort. I'll test this and let you know. It's on
my to do list to learn CMake, I've had the book on my desk for a month now.<br>
<br>
Also, I made another Fortran example that's closer in spirit to the wiki example, which uses the DataGenerator class as the C++ does, if you're interested.
<br>
<br>
Thanks, <br>
Mark</span><u></u><u></u></p>
</div>
</div>
<div class="MsoNormal" style="text-align:center" align="center">
<hr align="center" size="2" width="100%">
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:
</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Andy Bauer</span><br>
<b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Sent: </span>
</b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">9/19/2012 1:12 PM</span><br>
<b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">To: </span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Vanmoer, Mark W</span><br>
<b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Cc: </span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a></span><br>
<b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Subject: </span>
</b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Re: [Paraview] Python interpreter not being created for coprocessing?</span><u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Mark,<br>
<br>
I finally got around to fixing this. It's in the stage branch (remotes/stage/13460_CP_FortranAdaptor) and hopefully will go into the master branch on the next gatekeeper review. There were 2 issues fixed in this:<br>
1) for using newer GCC compilers and other compilers that hide symbols in the library (<a href="http://gcc.gnu.org/wiki/Visibility" target="_blank">http://gcc.gnu.org/wiki/Visibility</a>) the FortranAdaptor API methods weren't available to other libraries and
executables trying to link to them.<br>
2) Since 3.14.1 I changed around the way ParaView gets initialized through the coprocessing library (now through vtkCPProcessor::Initialize() as it should be instead of vtkCPPythonScriptPipeline::Initialize()). I forgot to make the change in the FortranAdaptorAPI.cxx
which caused the python interpretor to not be initialized properly, as you noticed.<br>
<br>
Thanks for the help in finding this problem. Sorry for the slow response on it. I created a CMakeLists.txt file to help build as I've been using CMake too long and can't remember how to work with Makefiles anymore! I'm attaching it in case you'd like to use
it. I didn't test it with an install of ParaView as I really only work directly with ParaView builds.<br>
<br>
Let me know if this solves your problems. I seem to remember there being more as far as using CUDA and PGI but hopefully this gets you close.<br>
<br>
By the way, I'll probably use what you sent me to create a Fortran test or example to help with making sure it doesn't get broken in the future.<br>
<br>
Andy<u></u><u></u></p>
<div>
<p class="MsoNormal">On Wed, Sep 12, 2012 at 4:42 PM, Vanmoer, Mark W <<a href="mailto:mvanmoer@illinois.edu" target="_blank">mvanmoer@illinois.edu</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Tarball attached, it’s not exactly the wiki example because I didn’t want to bother with passing
a DataGenerator object back and forth from C++ to Fortran, so I just make a sphere. The pipeline script is a shrink filter generated by pvclient.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I checked CMake and it looks like it’s finding the same version of python.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Mark</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Andy Bauer [mailto:<a href="mailto:andy.bauer@kitware.com" target="_blank">andy.bauer@kitware.com</a>]
<br>
<b>Sent:</b> Wednesday, September 12, 2012 1:02 PM</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><br>
<b>To:</b> Vanmoer, Mark W<br>
<b>Cc:</b> <a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a><br>
<b>Subject:</b> Re: [Paraview] Python interpreter not being created for coprocessing?<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Can you send me your Fortran version of the wiki example?<br>
<br>