<HTML>
<HEAD>
<TITLE>Re: [Paraview] Parallel Data Redistribution</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>In either case, I’m thinking this is not a great idea if the filter John is writing is intended to be used with anything other than his special reader. Otherwise, the reader is going to get the request and do something unpredictable.<BR>
<BR>
Maybe a safer approach is to make a new key like UPDATE_NO_DATA. The filter could then set request some arbitrary piece and number of pieces and also add UPDATE_NO_DATA to its input. If the reader understand UPDATE_NO_DATA, it will not actually read the data requested. If it does not understand, it will simply read a piece that is ignored: no big deal. That said, is this adding unnecessary complications to the pipeline mechanisms (which are already difficult for reader/filter designers)?<BR>
<BR>
-Ken<BR>
<BR>
<BR>
On 12/15/09 12:48 PM, "Berk Geveci" <<a href="berk.geveci@kitware.com">berk.geveci@kitware.com</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>In the past we had the pipeline execution stop if update number of<BR>
pieces was 0. We can easily add that back in. It would be a one line<BR>
change in vtkStreamingDemandDrivePipeline::NeedToExecuteData().<BR>
<BR>
Alternatively, the reader could check for UPDATE_NUMBER_OF_PIECES and<BR>
do nothing it is is 0. I think that's better that requested piece<BR>
being -1 which is ambiguous.<BR>
<BR>
-berk<BR>
<BR>
On Tue, Dec 15, 2009 at 2:41 PM, Moreland, Kenneth <<a href="kmorel@sandia.gov">kmorel@sandia.gov</a>> wrote:<BR>
> I think the part that John wants that is missing is the ability for a filter<BR>
> to say I want no data (or simply, I don’t care). The idea being that during<BR>
> the RequestUpdateExtent phase you could send a flag to tell the pipeline to<BR>
> not propagate the request upstream. Instead, simply call RequestData and<BR>
> propagate back down. However, like Berk said, this is a bad idea because<BR>
> any upstream objects that do parallel communication will lock up. I would<BR>
> hope that your reader itself does parallel communication. You’re going to<BR>
> get contention on your disks otherwise as different processes request the<BR>
> same pieces.<BR>
><BR>
> I suppose the filter could also simply request piece –1 and the reader could<BR>
> take that as a flag to read nothing. Assuming neither the executive or<BR>
> another filter tries to do anything with the piece number you should be OK,<BR>
> but I don’t know if that will just work.<BR>
><BR>
> -Ken<BR>
><BR>
><BR>
> On 12/15/09 9:21 AM, "Berk Geveci" <<a href="berk.geveci@kitware.com">berk.geveci@kitware.com</a>> wrote:<BR>
><BR>
> Hmmm. I don't exactly see the difference between the two (I mean<BR>
> streaming and distributing). In your case, I would think that a filter<BR>
> can arbitrarily change the pipeline request to either request nothing<BR>
> or ask the same piece on two processes. It would like this:<BR>
><BR>
> reader0 - request piece 0 of 1 - filter - request piece 0 of 1 -<BR>
> update suppressor<BR>
> reader1 - request piece 0 of 1 - filter - request piece 1 of 1 -<BR>
> update suppressor<BR>
><BR>
> This should work fine as long as the reader does not have another<BR>
> consumer (for example a representation if the visibility is on) that<BR>
> is asking for a different piece. The following should also work<BR>
><BR>
> reader0 - request piece 0 of 1 - filter - request piece 0 of 1 -<BR>
> update suppressor<BR>
> reader1 - request piece 0 of 0 - filter - request piece 1 of 1 -<BR>
> update suppressor<BR>
><BR>
> Of course, if you have any filters that do parallel communication in<BR>
> between, you are SOL.<BR>
><BR>
> Does that make sense?<BR>
><BR>
> Now if you have structured data and the frickin' extent translator<BR>
> gets in between, things are not as straightforward - which why I hate<BR>
> the extent translator. If I had some time/funding, I would write a new<BR>
> executive that does not use structured extents.<BR>
><BR>
> -berk<BR>
><BR>
> On Tue, Dec 15, 2009 at 11:07 AM, Biddiscombe, John A. <<a href="biddisco@cscs.ch">biddisco@cscs.ch</a>><BR>
> wrote:<BR>
>> Berk,<BR>
>><BR>
>> We had a discussion back in 2008, which resides here<BR>
>> <a href="http://www.cmake.org/pipermail/paraview/2008-May/008170.html">http://www.cmake.org/pipermail/paraview/2008-May/008170.html</a><BR>
>><BR>
>> Continuing from this, my question of the other day, touches on the same<BR>
>> problem.<BR>
>><BR>
>> I'd like to manipulate the piece number read by each reader. As mentioned<BR>
>> before, UPDATE_PIECE is not passed into RequestInformation at first (since<BR>
>> nobody knows how many pieces there are yet!), so I can't (directly) generate<BR>
>> information in the reader which is 'piece dependent'. And I can't be sure<BR>
>> that someone doing streaming won't interfere with piece numbers when using<BR>
>> the code differently.<BR>
>><BR>
>> For the particle tracer (for example), I'd like to tell the upstream<BR>
>> pipeline to read no pieces when certain processes are empty of particles<BR>
>> (currently they update and generate{=read} data when they don't need to). I<BR>
>> may be able to suppress the forward upstream somehow, but I don't know of an<BR>
>> easy way for the algorithm to say "Stop" to the executive to prevent it<BR>
>> updating if the timestep changes, but the algorithm has determined that no<BR>
>> processing is required (ForwardUpstream of Requests continues unabated). I'd<BR>
>> like to set the UPdatePiece to -1 to tell the executive to stop operating.<BR>
>><BR>
>> Also : for dynamic load balancing, I'd like to instruct several reader to<BR>
>> read the same piece - since the algorithm controls (for example) the<BR>
>> particles the algorithm can internally communicate information about what to<BR>
>> do amongst its processes, but it can't talk upstream to the readers and<BR>
>> fudge them.<BR>
>><BR>
>> I am wondering if there is any way of supporting this kind of thing using<BR>
>> the current information keys and my instinct says no. It seems like the<BR>
>> update pice and numpieces were really intended for streaming and we need two<BR>
>> kinds of 'pieces', one for streaming, another for splitting in _parallel_<BR>
>> because they aren't quite the same. (Please note that I haven't actually<BR>
>> tried changing piece requests in the algorithms yet, so I'm only guessing<BR>
>> that it won't work properly)<BR>
>><BR>
>> <cough><BR>
>> UPDATE_STREAM_PIECE<BR>
>> UPDATE_PARALLEL_PIECE<BR>
>> <\cough><BR>
>><BR>
>> Comments?<BR>
>><BR>
>> JB<BR>
>><BR>
>><BR>
>>><BR>
>>> I would have the reader (most parallel readers do this) generate empty<BR>
>>> data on all processes of id >= N. Then your filter can redistribute<BR>
>>> from those N processes to all M processes. I am pretty sure<BR>
>>> RedistributePolyData can do this for polydata as long as you set the<BR>
>>> weight to 1 on all processes. Ditto for D3.<BR>
>>><BR>
>>> -berk<BR>
>>><BR>
>>> On Fri, Dec 11, 2009 at 4:13 PM, Biddiscombe, John A. <<a href="biddisco@cscs.ch">biddisco@cscs.ch</a>><BR>
>>> wrote:<BR>
>>> > Berk<BR>
>>> ><BR>
>>> >> It sounds like M is equal to the number of processors (pipelines) and<BR>
>>> >> M >> N. Is that correct?<BR>
>>> ><BR>
>>> > Yes, That's the idea. N blocks, broken (in place) into M new blocks,<BR>
>>> > then<BR>
>>> fanned out to the M processes downstream where they can be processed<BR>
>>> separately . If it were on a single node, then each block could be a<BR>
>>> separate 'connection' to a downstream filter, but distributed, an<BR>
>>> explicit<BR>
>>> send is needed.<BR>
>>> ><BR>
>>> > JB<BR>
>>> ><BR>
>>> >><BR>
>>> >> -berk<BR>
>>> >><BR>
>>> >> On Fri, Dec 11, 2009 at 10:40 AM, Biddiscombe, John A.<BR>
>>> >> <<a href="biddisco@cscs.ch">biddisco@cscs.ch</a>><BR>
>>> >> wrote:<BR>
>>> >> > Berk<BR>
>>> >> ><BR>
>>> >> > The data will be UnstructuredGrid for now. Multiblock, but actually,<BR>
>>> >> > I<BR>
>>> >> don't really care what each block is, only that I accept one block on<BR>
>>> each<BR>
>>> >> of N processes, split it into more pieces, and the next filter accepts<BR>
>>> one<BR>
>>> >> (or more if the numbers don't match up nicely) blocks and process<BR>
>>> >> them.<BR>
>>> The<BR>
>>> >> redistribution shouldn't care what data types, only how many blocks in<BR>
>>> and<BR>
>>> >> out.<BR>
>>> >> ><BR>
>>> >> > Looking at RedistributePolyData makes me realize my initial idea is<BR>
>>> >> > no<BR>
>>> >> good. In my mind I had a pipeline where multiblock datasets are passed<BR>
>>> down<BR>
>>> >> the pipeline and simply the number of pieces is manipulated to achieve<BR>
>>> what<BR>
>>> >> I wanted - but I see now that if I have M pieces downstream mapped<BR>
>>> upstream<BR>
>>> >> to N pieces, what will happen is the readers will be effectively<BR>
>>> duplicated<BR>
>>> >> and M/N readers will read the same pieces. I don't want this to happen<BR>
>>> >> as<BR>
>>> IO<BR>
>>> >> will be a big problem if readers read the same blocks M/N times.<BR>
>>> >> > I was hoping there was a way of simply instructing the pipeline to<BR>
>>> manage<BR>
>>> >> the pieces, but I see now that this won't work, as there needs to be a<BR>
>>> >> specific Send from each N to their M/N receivers (because the data is<BR>
>>> >> physically in another process, so the pipeline can't see it). This is<BR>
>>> very<BR>
>>> >> annoying as there must be a class which already does this (block<BR>
>>> >> redistribution, rather than polygon level redistribution), and I would<BR>
>>> like<BR>
>>> >> it to be more 'pipeline integrated' so that the user doesn't have to<BR>
>>> >> explicitly send each time an algorithm needs it.<BR>
>>> >> ><BR>
>>> >> > I'll go through RedistributePolyData in depth and see what I can<BR>
>>> >> > pull<BR>
>>> out<BR>
>>> >> of it - please feel free to steer me towards another possibility :)<BR>
>>> >> ><BR>
>>> >> > JB<BR>
>>> >> ><BR>
>>> >> ><BR>
>>> >> >> -----Original Message-----<BR>
>>> >> >> From: Berk Geveci [<a href="mailto:berk.geveci@kitware.com">mailto:berk.geveci@kitware.com</a>]<BR>
>>> >> >> Sent: 11 December 2009 16:09<BR>
>>> >> >> To: Biddiscombe, John A.<BR>
>>> >> >> Cc: <a href="paraview@paraview.org">paraview@paraview.org</a><BR>
>>> >> >> Subject: Re: [Paraview] Parallel Data Redistribution<BR>
>>> >> >><BR>
>>> >> >> What is the data type? vtkRedistributePolyData and its subclasses<BR>
>>> >> >> do<BR>
>>> >> >> this for polydata. It can do load balancing (where you can specify<BR>
>>> >> >> a<BR>
>>> >> >> weight for each processor) as well.<BR>
>>> >> >><BR>
>>> >> >> -berk<BR>
>>> >> >><BR>
>>> >> >> On Fri, Dec 11, 2009 at 9:59 AM, Biddiscombe, John A.<BR>
>>> <<a href="biddisco@cscs.ch">biddisco@cscs.ch</a>><BR>
>>> >> >> wrote:<BR>
>>> >> >> > I have a filter pipeline which reads N blocks from disk, this<BR>
>>> >> >> > works<BR>
>>> >> fine<BR>
>>> >> >> on N processors.<BR>
>>> >> >> ><BR>
>>> >> >> > I now wish to subdivide those N blocks (using a custom filter) to<BR>
>>> >> produce<BR>
>>> >> >> new data which will consist of M blocks - where M >> N.<BR>
>>> >> >> ><BR>
>>> >> >> > I wish to run the algorithm on M processors and have the piece<BR>
>>> >> information<BR>
>>> >> >> transformed between the two filters (reader -> splitter), so that<BR>
>>> blocks<BR>
>>> >> are<BR>
>>> >> >> distributed correctly. The reader will Read N blocks (leaving M-N<BR>
>>> >> processes<BR>
>>> >> >> unoccupied), but the filter which splits them up needs to output a<BR>
>>> >> different<BR>
>>> >> >> number of pieces and have the full M processes receiving data.<BR>
>>> >> >> ><BR>
>>> >> >> > I have a reasonably good idea of how to implement this, but I'm<BR>
>>> >> wondering<BR>
>>> >> >> if any filters already do something similar. I will of course take<BR>
>>> apart<BR>
>>> >> the<BR>
>>> >> >> D3 filter for ideas, but I don't need to do a parallel spatial<BR>
>>> >> decomposition<BR>
>>> >> >> since my blocks are already discrete - I just want to redistribute<BR>
>>> >> >> the<BR>
>>> >> >> blocks around and more importantly change the numbers of them<BR>
>>> >> >> between<BR>
>>> >> >> filters.<BR>
>>> >> >> ><BR>
>>> >> >> > If anyone can suggest examples which do this already, please do<BR>
>>> >> >> ><BR>
>>> >> >> > Thanks<BR>
>>> >> >> ><BR>
>>> >> >> > JB<BR>
>>> >> >> ><BR>
>>> >> >> > --<BR>
>>> >> >> > John Biddiscombe, email:biddisco @<BR>
>>> cscs.ch<BR>
>>> >> >> > <a href="http://www.cscs.ch/">http://www.cscs.ch/</a><BR>
>>> >> >> > CSCS, Swiss National Supercomputing Centre | Tel: +41 (91)<BR>
>>> 610.82.07<BR>
>>> >> >> > Via Cantonale, 6928 Manno, Switzerland | Fax: +41 (91)<BR>
>>> 610.82.82<BR>
>>> >> >> ><BR>
>>> >> >> ><BR>
>>> >> >> > _______________________________________________<BR>
>>> >> >> > Powered by www.kitware.com<BR>
>>> >> >> ><BR>
>>> >> >> > Visit other Kitware open-source projects at<BR>
>>> >> >> <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><BR>
>>> >> >> ><BR>
>>> >> >> > Please keep messages on-topic and check the ParaView Wiki at:<BR>
>>> >> >> <a href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a><BR>
>>> >> >> ><BR>
>>> >> >> > Follow this link to subscribe/unsubscribe:<BR>
>>> >> >> > <a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><BR>
>>> >> >> ><BR>
>>> >> ><BR>
>>> ><BR>
>><BR>
> _______________________________________________<BR>
> Powered by www.kitware.com<BR>
><BR>
> Visit other Kitware open-source projects at<BR>
> <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><BR>
><BR>
> Please keep messages on-topic and check the ParaView Wiki at:<BR>
> <a href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a><BR>
><BR>
> Follow this link to subscribe/unsubscribe:<BR>
> <a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><BR>
><BR>
><BR>
><BR>
><BR>
> **** Kenneth Moreland<BR>
> *** Sandia National Laboratories<BR>
> ***********<BR>
> *** *** *** email: <a href="kmorel@sandia.gov">kmorel@sandia.gov</a><BR>
> ** *** ** phone: (505) 844-8919<BR>
> *** web: <a href="http://www.cs.unm.edu/~kmorel">http://www.cs.unm.edu/~kmorel</a><BR>
><BR>
><BR>
<BR>
<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
</SPAN></FONT><FONT SIZE="2"><FONT FACE="Consolas, Courier New, Courier"><SPAN STYLE='font-size:10pt'><BR>
**** Kenneth Moreland<BR>
*** Sandia National Laboratories<BR>
*********** <BR>
*** *** *** email: <a href="kmorel@sandia.gov">kmorel@sandia.gov</a><BR>
** *** ** phone: (505) 844-8919<BR>
*** web: <a href="http://www.cs.unm.edu/~kmorel">http://www.cs.unm.edu/~kmorel</a><BR>
</SPAN></FONT></FONT><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
</SPAN></FONT>
</BODY>
</HTML>