<div dir="ltr">Thanks guys:<div><br></div><div><span style>> Just this week, I wrote a Python program that uses the numpy.fromfile function and ParaView's paraview.vtk.dataset_adapter.</span><u style></u><span style>n</span><span style>umpyTovtkDataArray and </span><a href="http://paraview.vtk.io/" target="_blank" style>paraview.vtk.io</a><span style>.</span><u style></u><span style>vtkXMLPolyData</span><span style>Writer functions to convert a large binary file to .vtp format. > It is reasonably fast. Maybe you can use something like that as a post-processing step.</span><br style>
<br style><span style>> This separates the tasks of producing data and converting it to VTK format.</span></div><div><font color="#222222" face="arial, sans-serif"><br></font></div><div><font color="#222222" face="arial, sans-serif">This was actually my first thought but the reason I didn't go with it is that it is dependent on the vtk API. I prefer to keep my code's dependency minimal. Also it rewrites basically the same information redundantly in a vtk file. Still I think that's the way to if one wants to stick with vtu for instance if you want to benefit from .pvtu format.</font></div>
<div><font color="#222222" face="arial, sans-serif"><br></font></div><div><font color="#222222" face="arial, sans-serif"><br></font></div><div><font color="#222222" face="arial, sans-serif">> </font><span style>If you know the structure of the binary file, you can use xdmf format file to read your binary data.</span></div>
<div><span style><br></span></div><div><span style><font color="#222222" face="arial, sans-serif">Actually I went ahead and learned it and now can write xdmf files to point to my binary files. I like and seems to be quite flexible; for example I can switch to HDF5 in the future if I want to and can keep my xmdf writer objects (with minor modifications).</font></span></div>
<div><span style><font color="#222222" face="arial, sans-serif"><br></font></span></div><div><div style>> if you grid does not change over time you can use XPath:</div><div style><font color="#222222" face="arial, sans-serif">> "</font><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-weight:bold">XPath, <span style="font-weight:normal">allows for elements in the XML document and the API to reference specific elements in a document"</span></span><div>
<div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word">
<span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word">
<span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="color:rgb(34,34,34);font-family:Helvetica;font-size:medium;word-wrap:break-word">
> <a href="http://www.xdmf.org/index.php/XDMF_Model_and_Format" target="_blank" style="color:rgb(17,85,204)">http://www.xdmf.org/index.php/XDMF_Model_and_Format</a></div><div style="color:rgb(34,34,34);font-family:Helvetica;font-size:medium;word-wrap:break-word">
<br></div>I actually did not understand when I was reading the tutorial. Can I use this feature to include other xdmf files as well? I am thinking on how to replicate .pvtu functionality with xdmf. Or maybe its possible to point to xdmf files in .pvtu directly? Could you please comment on this?</span></div>
<div style="word-wrap:break-word"><span style="border-collapse:separate"><br></span></div><div style="word-wrap:break-word"><span style="border-collapse:separate">Thanks,</span></div><div style="word-wrap:break-word"><span style="border-collapse:separate">Mohammad</span></div>
</span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></div></div></div><div><font color="#222222" face="arial, sans-serif"><br></font><br><div class="gmail_quote">
On Sun, May 20, 2012 at 7:43 AM, Felipe Bordeu Weldt <span dir="ltr"><<a href="mailto:felipe.bordeu@ec-nantes.fr" target="_blank">felipe.bordeu@ec-nantes.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Hi,<div><br></div><div>If you know the structure of the binary file, you can use xdmf format file to read your binary data.</div><div><br></div><div><div><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<div><br></div></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></div></div><div>for the binary part use something like this:</div>
<div><span style="white-space:pre-wrap"><DataItem ItemType="Uniform"</span></div><div><span style="font-family:monospace;white-space:pre-wrap"><font face="Helvetica"> Format="Binary"</font></span></div>
<div><span style="font-family:monospace;white-space:pre-wrap"><span style="font-family:Helvetica"> Dimensions="64 128 256"</span></span></div><div><span style="font-family:monospace;white-space:pre-wrap"><font face="Helvetica"> Seek="1234"</font></span><span style="font-family:monospace;white-space:pre-wrap"><span style="font-family:Helvetica">></span></span></div>
<div><span style="white-space:pre-wrap"> PressureFile.bin</span></div><div><span style="white-space:pre-wrap"></DataItem></span></div><div><br></div><div><div>if you grid does not change over time you can use XPath:</div>
<div>"<span style="font-weight:bold">XPath, <span style="font-weight:normal">allows for elements in the XML document and the API to reference specific elements in a document"</span></span><div><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<div><div><a href="http://www.xdmf.org/index.php/XDMF_Model_and_Format" target="_blank">http://www.xdmf.org/index.php/XDMF_Model_and_Format</a></div></div><div><br></div><div><br></div></div></span></div></span></div></span></div>
</span></div></span></div></span></div></span></div></span></div></span></div></span></div></span></div></div></div></div><div><br></div><div>I use xdmf all the time, I think is very flexible (ascii, binary, HDF5 heavy data storage), time, grids, extra data using "Information" tag. And all kinds of topologies and geometries. </div>
<div><br></div><div><br></div><div><div><div>Le 20 mai 12 ā 13:47, Hal Canary a écrit :</div><div><div class="h5"><br><blockquote type="cite"><div>On 05/19/2012 06:30 PM, Mohammad Mirzadeh wrote:<br><blockquote type="cite">
Hi guys,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I need a little bit of advice on how I'm outputting data. So far I have<br></blockquote><blockquote type="cite">been using a simple legacy VTK format and dumping my grid and all<br>
</blockquote><blockquote type="cite">sclaar/vectors into that single file. Basically I start with writing the<br></blockquote><blockquote type="cite">grid and then append as many as variables that I like to the file. This<br>
</blockquote><blockquote type="cite">has the problem that it was ascii and slow for large data sets,<br></blockquote><blockquote type="cite">especially fro a parallel code where the solver was taking only 5 sec<br></blockquote>
<blockquote type="cite">but the IO was around 20!!<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">So I want to turn into binary and I have learned how to output binary<br></blockquote><blockquote type="cite">
vtu files. The issue is, since the cluster I'm working with has kill<br></blockquote><blockquote type="cite">times (Max 24 hr) I also need to be able to save and resume my<br></blockquote><blockquote type="cite">simulation. So, what I do right now is dump my variables in binary form<br>
</blockquote><blockquote type="cite">into a single file that does not have any xml tag or anything -- just<br></blockquote><blockquote type="cite">raw binary data. This is a preferred method since when I want to resume,<br>
</blockquote><blockquote type="cite">I do not need to parse the xml code; I just read the binary file and<br></blockquote><blockquote type="cite">that's it. Besides, I save on the grid information I'm writing to disk.<br>
</blockquote><blockquote type="cite">For a 23M grid point, the grid information (which basically is a<br></blockquote><blockquote type="cite">quadtree saved as a general unstructured grid with<br></blockquote><blockquote type="cite">
cell connectivity information etc) file is about 1GB whereas my actual<br></blockquote><blockquote type="cite">data is only about 150MB. In most cases my grid does not change with<br></blockquote><blockquote type="cite">
time, so I really only need to save it once and that's enough.<br>
</blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Now, the problem is, I want to be able to use or refer to these binary<br></blockquote><blockquote type="cite">files using the xml tags. Here are the questions:<br>
</blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">1) Is this supported in .vtu format? How can I refer to a binary file<br></blockquote><blockquote type="cite">instead of dumping all the binary information in the file?<br>
</blockquote><blockquote type="cite">2) If this is not supported in .vtu, is it something xdmf can do? If so<br></blockquote><blockquote type="cite">how? Can you please point me to a tutorial?<br></blockquote><blockquote type="cite">
3) Is there any plan or strategy that would be better than what I have<br></blockquote><blockquote type="cite">in mind right now? Please remember that retaining the raw binary format<br></blockquote><blockquote type="cite">
of the files is a must for me (well I do not see convincing reasons to<br></blockquote><blockquote type="cite">give up on it, but if there is one, please let me know)<br></blockquote><blockquote type="cite"><br></blockquote>
<blockquote type="cite">Thanks<br></blockquote><blockquote type="cite">Mohammad<br></blockquote><br><br><br>Just this week, I wrote a Python program that uses the numpy.fromfile function and ParaView's paraview.vtk.dataset_adapter.numpyTovtkDataArray and paraview.vtk.io.vtkXMLPolyDataWriter functions to convert a large binary file to .vtp format. It is reasonably fast. Maybe you can use something like that as a post-processing step.<br>
<br>This separates the tasks of producing data and converting it to VTK format.<br><br><br>-- <br>Hal Canary<br>_______________________________________________<br>Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br><br>Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">http://paraview.org/Wiki/ParaView</a><br>
<br>Follow this link to subscribe/unsubscribe:<br><a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br></div></blockquote></div></div></div>
<br>
</div></div><br>_______________________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">http://paraview.org/Wiki/ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br>
<br></blockquote></div><br></div></div>