Hi Matthieu,<br><br>Please respond to the whole list so that everyone can share in the conversation.<br><br>When you start to get into parallel with the structured grids, I suggest you play around a bit with running pvserver in parallel, connect to it with the GUI, and play with some of the sources like Wavelet and Mandelbrot.<br>
<br>As for comparisons with VisIt's in-situ visualization. We are working on getting interaction through the GUI to process the data directly coming out of the simulation without hitting the disk but it's not ready yet. There would be 2 use cases for that. The first is where it stops the simulation and waits for the user to finish their viz/analysis, i.e. like VisIt. The second would be where the user changes the outputting pipeline while the simulation code is running. The simulation code keeps running while the user is setting up new pipelines and the changes don't take effect until the next opportunity after the user adds their new pipeline or changes parameters to their existing pipeline. Our design started working with the use case that the user would submit their simulation+coprocessing as a batch job. This makes sense if the simulation code is "fast". For example, one of the simulation codes that we've been working with, PHASTA led by Ken Jansen at UC Boulder, is able to do an implicit time step solve (I believe it's implicit but don't quote me on that) in less than a second for a problem with hundreds of millions of unknowns. We've done this with up to 160,000 cores. While ParaView can still process that efficiently for many filters, it may take the user minutes to actually examine the data. That's obviously going to be a major waste of computational power if the supercomputer is waiting on the user before computing the next time step. In this case, the predefined pipeline works very well in that it completes the requested coprocessing and immediately gives control back to the simulation code to continue its processing. Hopefully this makes sense. <br>
<br>I've inlined some comments below to make sure that I address your questions.<br><br>Andy<br><br><div class="gmail_quote">On Tue, Feb 7, 2012 at 8:52 AM, Matthieu Dorier <span dir="ltr"><<a href="mailto:matthieu.dorier@gmail.com" target="_blank">matthieu.dorier@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Andy,<div><br></div><div>Thank you a lot.</div><div><br></div><div>I'm not quiet sure to fully understand the SetWholeExtent stuff for parallel runs but I will certainly come back to you in the future when I effectively run in parallel.</div>
<div><br></div><div>For now I have another question: I always assumed that in-situ visualization with ParaView was done in a way similar than VisIt, with a client that attempts to connect to perform interactive visualization. But reading again all the Paraview tutorial, it seems I was wrong. </div>
<div><br></div><div>Can you please tell me if my understanding is right: ParaView only feeds a predefined pipeline (written in python or C++) with VTK objects, the "normal" end of a pipeline consists in writing output (images, for instance). </div>
</blockquote><div>Yes, this is the current situation where you should consider the pipelines that are created by the script generator gui plugin to be static and to only output data files and screenshot images. You can modify the scripts to add in better control though. We also are working on allowing the user to change the pipeline but I don't know when this will be ready.<br>
</div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Using data staging, the pipeline's output is redirected to the visualization cluster through sockets and the pvservers are used to interact with the user. So there is a "static" part from the simulation to staging where always the same coprocessing is performed, and an interactive part from staging to user where the user can request more specific visualization. Am I right?</div>
</blockquote><div>This is also in the works where the data set or some subset of it will be staged from the compute cluster to a viz cluster for the user to interact with the "current" result. This will get done over the network and once the data has been passed to the viz cluster, the compute cluster can continue on with simulation computations, if that's what the user desires.<br>
</div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><br></div><div>Thanks,</div><div><br></div><div>Matthieu<div><div><br><br><div class="gmail_quote">2012/2/6 Andy Bauer <span dir="ltr"><<a href="mailto:andy.bauer@kitware.com" target="_blank">andy.bauer@kitware.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Matthieu,<br><br>Your temperature array needs to be associated with your grid as point data. The temperature array will be stored as a derived class of vtkDataArray (I'll use a vtkDoubleArray). If you're storing your temperature data as a contiguous array with the same size as the number of points in your grid and you iterate through them such that you go through the x-direction fastest, then the y-direction next fastest, and finally z-direction, you'll probably want to use the SetArray() method to reuse the memory you're already using for your native storage. Then your wrapTemperature method would look like:<br>
<br>vtkDataArray* wrapTemperature(double* temperature, vtkIdType temperatureSize)<br>{<br> vtkDoubleArray* temperatureArray = vtkDoubleArray::New();<br> temperatureArray->SetName("Temperature");<br> temperatureArray->SetArray(temperature, temperatureSize, 1); // use 1 to tell VTK not to delete temperature when it doesn't need the array anymore<br>
return temperatureArray;<br>}<br><br>Alternatively, creating a new array would look like:<br>vtkDataArray* wrapTemperature(double* temperature, <more data>)<br>
{<br>
vtkDoubleArray* temperatureArray = vtkDoubleArray::New();<br> temperatureArray->SetNumberOfComponents(1); // assume temperature is a scalar tensor<br> temperatureArray->SetNumberOfTuples(grid->GetNumberOfPoints());<br>
temperatureArray->SetName("Temperature");<br> for(int zid=0;zid<PTZ;zid++)<br> for(int yid=0;yid<PTY;yid++)<br> for(int xid=0;xid<PTX;xid++)<br> temperatureArray->SetTupleValue(xid+yid*PTX+zid*PTX*PTY, temperature+<corresponding index in array>);<br>
<br>
return temperatureArray;<br>
}<br>
<br>Then your wrapMeshData() would look like:<br><div>vtkDataObject* wrapMeshData()</div><div><div>{</div><div> vtkFloatArray* xCoords, yCoords, zCoords;</div><div> xCoords = vtkFloatArray::New();</div>
<div> xCoords->setArray(mesh_x,PTX,1);</div><div><div> yCoords = vtkFloatArray::New();</div><div> yCoords->setArray(mesh_y,PTY,1);</div></div><div><div> zCoords = vtkFloatArray::New();</div><div> zCoords->setArray(mesh_z,PTZ,1);</div>
</div><div> vtkRectilinearGrid *grid = vtkRectilinearGrid::New();</div><div> grid->setDimensions(PTX,PTY,PTZ);</div><div> grid->setXCoordinates(xCoords);</div><div> grid->setYCoordinates(yCoords);</div></div>
<div>
grid->setZCoordinates(zCoords);<br> vtkDataArray* array = wrapTemperature();<br> grid->GetPointData()->AddArray(array);<br> array->Delete(); // decrement array reference counter<br></div>
<div> return (vtkDataObject*)grid;</div><div>}</div><br>Finally, don't forget to call "vtkCPDataDescription::SetWholeExtent(0, PTX-1, 0, PTY-1, 0, PTZ-1);". When running in parallel the input grid should be partitioned and wrapMeshData() would also call SetExtent() with that process's sub extent of the whole extent.<span><font color="#888888"><br>
<br>Andy</font></span><div><div><br><br><div class="gmail_quote">On Mon, Feb 6, 2012 at 5:49 AM, Matthieu Dorier <span dir="ltr"><<a href="mailto:matthieu.dorier@gmail.com" target="_blank">matthieu.dorier@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<div><br></div><div>I'm following the SC'10 tutorial with the help of the VTK doxygen, but I don't understand how to do the following:</div><div><br></div><div>I have a 3D rectilinear grid which coordinates are defined by mesh_x, mesh_y and mesh_z (float arrays of dimensions PX, PY and PZ). I have a variable "temperature" defined as a 3D array representing the temperature at each point of the grid. Currently I have followed the slides 24 and 25 from the SC'10 tutorial, at some point vtkDataObjects have to be created, here is how I created the grid object :</div>
<div><br></div><div><div>// This function is called to retrieve the mesh</div><div>vtkObject* wrapMeshData()</div><div>{</div><div> vtkFloatArray* xCoords, yCoords, zCoords;</div><div> xCoords = vtkFloatArray::New();</div>
<div> xCoords->setArray(mesh_x,PTX,1);</div><div><div> yCoords = vtkFloatArray::New();</div><div> yCoords->setArray(mesh_y,PTY,1);</div></div><div><div> zCoords = vtkFloatArray::New();</div><div> zCoords->setArray(mesh_z,PTZ,1);</div>
</div><div> vtkRectilinearGrid *grid = vtkRectilinearGrid::New();</div><div> grid->setDimensions(PTX,PTY,PTZ);</div><div> grid->setXCoordinates(xCoords);</div><div> grid->setYCoordinates(yCoords);</div><div>
grid->setZCoordinates(zCoords);</div>
<div> return (vtkObject*)grid;</div><div>}</div><div><br></div>Now I want to make a function that retrieves the temperature so that I can map each data value to each point of the mesh, something like :</div><div><br></div>
<div>vtkDataObject* wrapTemperature() {</div><div> // ???</div><div>}</div><div><br></div><div>How can I do that without copying the original array that I want to wrap?</div><div>Also what functions do I then call from the vtkCPDataDescription object? (on the tutorial they call SetGrid, I guess there is something else to call for the temperature field)</div>
<div><br></div><div>Thank you for your help,</div><div><br></div><div>Matthieu</div><div><br><div class="gmail_quote">2012/1/19 Berk Geveci <span dir="ltr"><<a href="mailto:berk.geveci@kitware.com" target="_blank">berk.geveci@kitware.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Please feel free to ask questions if anything is not clear or if you need some help. We will be working on better documentation in the coming few months.<div>
<br></div><div>Best,</div><div>-berk<div><div><br><br><div class="gmail_quote">
On Wed, Jan 18, 2012 at 12:08 PM, Matthieu Dorier <span dir="ltr"><<a href="mailto:matthieu.dorier@gmail.com" target="_blank">matthieu.dorier@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thank you, this tutorial will help a lot.<span><font color="#888888"><div><br></div></font></span><div><span><font color="#888888">Matthieu</font></span><div><div><br><br><div class="gmail_quote">
2012/1/18 Andy Bauer <span dir="ltr"><<a href="mailto:andy.bauer@kitware.com" target="_blank">andy.bauer@kitware.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The current main wiki page for coprocessing with ParaView is at <a href="http://paraview.org/Wiki/CoProcessing" target="_blank">http://paraview.org/Wiki/CoProcessing</a>. In there it has links to 2 examples, a C++ driven example and a python driven example. You were looking at the C++ example. There's also a powerpoint presentation and some more code examples at <a href="http://www.paraview.org/Wiki/SC10_Coprocessing_Tutorial" target="_blank">http://www.paraview.org/Wiki/SC10_Coprocessing_Tutorial</a>. We will but putting together a more extensive book but that probably won't be ready for a month or two.<br>
<br>My suggestion would be to first work on creating a vtkDataObject to represent your data. Look at the powerpoint presentation for information on doing that and then maybe the VTK doxygen (<a href="http://www.vtk.org/doc/nightly/html/classes.html" target="_blank">http://www.vtk.org/doc/nightly/html/classes.html</a>) for more specific API questions.<br>
<br>Andy<br><br><div class="gmail_quote"><div><div>On Wed, Jan 18, 2012 at 5:12 AM, Matthieu Dorier <span dir="ltr"><<a href="mailto:matthieu.dorier@gmail.com" target="_blank">matthieu.dorier@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div>
Hello,<div><br></div><div>I have difficulties to understand how to instrument a simulation with ParaView in order to perform in-situ visualization.</div><div>Is there any documentation other than the simple example provided at <a href="http://paraview.org/Wiki/Coprocessing_example" target="_blank">http://paraview.org/Wiki/Coprocessing_example</a> ?</div>
<div>Thank you,<span><font color="#888888"><br clear="all"><div><br></div>-- <br>Matthieu Dorier<br>ENS Cachan, Brittany (Computer Science dpt.)<br>IRISA Rennes, Office C113<br><a href="http://perso.eleves.bretagne.ens-cachan.fr/%7Emdori307" target="_blank">http://perso.eleves.bretagne.ens-cachan.fr/~mdori307</a><br>
</font></span></div>
<br></div></div>_______________________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">http://paraview.org/Wiki/ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br>
<br></blockquote></div><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Matthieu Dorier<br>ENS Cachan, Brittany (Computer Science dpt.)<br>IRISA Rennes, Office C113<br><a href="http://perso.eleves.bretagne.ens-cachan.fr/%7Emdori307" target="_blank">http://perso.eleves.bretagne.ens-cachan.fr/~mdori307</a><br>
</div></div></div>
<br>_______________________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">http://paraview.org/Wiki/ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br>
<br></blockquote></div><br></div></div></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br>Matthieu Dorier<br>ENS Cachan, Brittany (Computer Science dpt.)<br>IRISA Rennes, Office C113<br><a href="http://perso.eleves.bretagne.ens-cachan.fr/%7Emdori307" target="_blank">http://perso.eleves.bretagne.ens-cachan.fr/~mdori307</a><br>
</font></span></div>
</blockquote></div><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Matthieu Dorier<br>ENS Cachan, Brittany (Computer Science dpt.)<br>IRISA Rennes, Office C113<br><a href="http://perso.eleves.bretagne.ens-cachan.fr/%7Emdori307" target="_blank">http://perso.eleves.bretagne.ens-cachan.fr/~mdori307</a><br>
</div></div></div>
</blockquote></div><br>