<html><body><div style="color:#000; background-color:#fff; font-family:arial, helvetica, sans-serif;font-size:12pt"><div><span>Hi,</span></div><div><br><span></span></div><div><span>I guess I forgot about the data duplication issue. For most statistics that I calculate in parallel, the basic results are not affected that much (i.e. mean, standard dev). But for the Pearson correlation coefficient, which is a bit more complicated, there is a noticeable difference (0.833 for parallel and 0.99 for serial).</span></div><div><br><span></span></div><div><span>Will this point sharing information become available in later versions of ParaView? i.e. would it ever be easy to identify and count duplicates?<br></span></div><div><br></div><div>Thanks,<br><span></span></div><div><span>Sohail</span></div><div><br></div> <div style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif;
font-size: 12pt;"> <div dir="ltr"> <font face="Arial" size="2"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> David Thompson <david.thompson@kitware.com><br> <b><span style="font-weight: bold;">To:</span></b> Sohail Shafii <sohailshafii@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> "paraview@paraview.org" <paraview@paraview.org> <br> <b><span style="font-weight: bold;">Sent:</span></b> Friday, August 17, 2012 3:40 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Paraview] Numpy masking (via programm filter) not quite working in parallel<br> </font> </div> <br>
Hi Sohail,<br><br>This is likely caused by points shared on several processes. While ParaView splits the cells of a mesh across processes, cells on the boundary between processes share vertices. Thus if a vertex bounds cells split across 3 processes, that vertex will appear in 3 different lists of local vertices and thus be counted 3 times instead of once. There is nothing in ParaView for determining which vertices are shared (and by how many processes).<br><br>This also affects the statistics filters when run in parallel on point-centered data. For most large data, the number of vertices on inter-process boundaries is small compared to the total so we documented the behavior for the statistics filters but did not implement a solution (because the expected skew is small).<br><br> David<br><br>On Aug 17, 2012, at 5:05 PM, Sohail Shafii <<a ymailto="mailto:sohailshafii@yahoo.com"
href="mailto:sohailshafii@yahoo.com">sohailshafii@yahoo.com</a>> wrote:<br><br>> Hi,<br>> <br>> I need to use Numpy in a lot of the programmable filters that I write, and I've run into differences in how its masking feature works in serial and parallel. Masking allows one to filter out portions of an array that do not pass some condition.<br>> <br>> As an example, I've created a stock paraview wavelet, and saved it as a pvd file. I then load it in, and run this inside of a programmable filter:<br>> ---<br>> import numpy<br>> <br>> data = inputs[0].PointData['RTData']<br>> # create a mask that tells us which points are equal to one<br>> mask = <a target="_blank" href="http://numpy.ma.ma/">numpy.ma</a>.masked_equal(data, 1)<br>> # filter data array by the mask conditions (so that other points are excluded)<br>> maskedPnts = numpy.extract(mask, data)<br>> <br>> print len(maskedPnts)<br>> <br>>
---<br>> In serial mode, I get 9261 points. With two processes, I get 2 x 4851 or 9702. So masking always produces more points.<br>> <br>> Any ideas to why that is? Is there anything I can do/print out to see why masking doesn't quite work in parallel?<br>> <br>> Thanks, Sohail<br>> _______________________________________________<br>> Powered by <a target="_blank" href="http://www.kitware.com/">www.kitware.com</a><br>> <br>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html<br>> <br>> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView<br>> <br>> Follow this link to subscribe/unsubscribe:<br>> http://www.paraview.org/mailman/listinfo/paraview<br><br><br><br> </div> </div> </div></body></html>