<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-GB;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Andy<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><b><span style="font-family:"Courier New";mso-fareast-language:EN-US"> obj = servermanager.misc.GlobalMapperProperties()</span></b><span style="font-family:"Courier New";mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><b><span style="font-family:"Courier New";mso-fareast-language:EN-US"> obj.GlobalImmediateModeRendering = 1</span></b><span style="font-family:"Courier New";mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I added this at the start of
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><b><span style="font-family:"Courier New";mso-fareast-language:EN-US">def DoCoProcessing(datadescription):</span></b><span style="font-family:"Courier New";mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">but it does not seem to have made any difference.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">JB<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Andy Bauer [mailto:andy.bauer@kitware.com]
<br>
<b>Sent:</b> 11 January 2012 02:39<br>
<b>To:</b> Takuya OSHIMA<br>
<b>Cc:</b> Biddiscombe, John A.; paraview@paraview.org<br>
<b>Subject:</b> Re: [Paraview] Unreclaimed memory in coprocessing? (PV 3.12.0 and git-master)<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">I did some more testing and determined that with my script that there aren't any vtkObjects that aren't cleaned up after each call to the coprocessing library with your surfaceRep.py script. Basicaly I kept
a counter in vtkObject that incremented for each constructor call and decremented for each destructor call. At the end of every call I had the same amount of vtkObjects.
<br>
<br>
After that I tried using valgrind to see if I could track it down any further. While there was some memory leaks reported, the amount didn't increase with the number of times the coprocessing pipeline was executed. Curiously though, valgrind did list an increase
in the amount of memory that was leaked. It seemed like there were quite a few PyObjects that weren't getting deleted properly though and maybe getting rid of them properly would help. It may also be that memory fragmentation is a problem since rather large
arrays are probably being allocated to for the field data as well as the opengl arrays (turning on immediate mode rendering would help with that but I'm not sure how to do that through the python but maybe someone else does).
<br>
<br>
Andy<o:p></o:p></p>
<div>
<p class="MsoNormal">On Tue, Jan 10, 2012 at 12:36 PM, Andy Bauer <<a href="mailto:andy.bauer@kitware.com">andy.bauer@kitware.com</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal">I ran the problem out for more time steps and am now seeing the same trend. Hopefully it won't take too much effort to track this down.<br>
<span style="color:#888888"><br>
Andy</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Tue, Jan 10, 2012 at 12:09 PM, Andy Bauer <<a href="mailto:andy.bauer@kitware.com" target="_blank">andy.bauer@kitware.com</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal">Hi,<br>
<br>
I'm trying to replicate it and with my scripts I'm getting the attached memory usage. I'm not seeing it though I may not be measuring memory very well or letting it play out long enough. I'll keep looking but it's possible that you're not deleting your created
data object that is getting passed into the coprocessing library. That would be my first recommendation to check. You may want to turn on debug leaks if you haven't in your paraview build.<br>
<br>
By the way, I generated that graph with the attached scripts.<br>
<span style="color:#888888"><br>
Andy</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">2012/1/10 Takuya OSHIMA <<a href="mailto:oshima@eng.niigata-u.ac.jp" target="_blank">oshima@eng.niigata-u.ac.jp</a>><o:p></o:p></p>
<p class="MsoNormal">Hi John,<br>
<br>
What I observed on a Mac and a Linux box with 8 processor parallel<br>
runs (around 400MB per process) were quite similar to your memory<br>
profiling but with a bit larger growth rate of 1-2% and 2-3% per<br>
coprocessing execution respectively (somehow it was bigger on Linux).<br>
I'll see if I can do similar profiling.<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Takuya OSHIMA, Ph.D.<br>
Faculty of Engineering, Niigata University<br>
8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN<o:p></o:p></p>
</div>
<p class="MsoNormal">From: "Biddiscombe, John A." <<a href="mailto:biddisco@cscs.ch" target="_blank">biddisco@cscs.ch</a>><br>
Subject: RE: [Paraview] Unreclaimed memory in coprocessing? (PV 3.12.0 and git-master)<br>
Date: Tue, 10 Jan 2012 14:51:41 +0000<o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
> Takuya<br>
><br>
> Running a test of coprocessing on a very small simulation using just 2 cores<br>
> on the desktop, I am seeing what looks like a very small leak. Over about 30<br>
> iterations, it’s of the order of 30MB, so might not be real (need to test on a<br>
> bigger domain to see if it grows faster), but the trend looks credible. I’ll<br>
> see if I can pinpoint where the loss is occurring.<br>
><br>
> JB<br>
><o:p></o:p></p>
</div>
<p class="MsoNormal">> [cid]<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">><br>
> -----Original Message-----<br>
> From: <a href="mailto:paraview-bounces@paraview.org" target="_blank">paraview-bounces@paraview.org</a> [mailto:<a href="mailto:paraview-bounces@paraview.org" target="_blank">paraview-bounces@paraview.org</a>] On<br>
> Behalf Of Takuya OSHIMA<br>
> Sent: 10 January 2012 15:11<br>
> To: <a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a><br>
> Subject: Re: [Paraview] Unreclaimed memory in coprocessing? (PV 3.12.0 and<br>
> git-master)<br>
><br>
> Attached in the previous mail is a sample Python coprocessing script that<br>
> exhibits the problem (written by the coprocessing script generator of<br>
> git-master as of today).<br>
><br>
> Takuya<br>
><br>
> Takuya OSHIMA, Ph.D.<br>
><br>
> Faculty of Engineering, Niigata University<br>
><br>
> 8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN<br>
><br>
> From: Takuya OSHIMA <<a href="mailto:oshima@eng.niigata-u.ac.jp" target="_blank">oshima@eng.niigata-u.ac.jp</a>><br>
><br>
> Subject: Unreclaimed memory in coprocessing? (PV 3.12.0 and git-master)<br>
><br>
> Date: Tue, 10 Jan 2012 23:08:21 +0900 (JST)<br>
><br>
> > Hi,<br>
><br>
> ><br>
><br>
> > I am having a problem where the memory usage of my simulation program<br>
><br>
> > which links with the PV coprocessing library grows until it gets<br>
><br>
> > killed by the operating system.<br>
><br>
> ><br>
><br>
> > I have the following piece of code for triggering coprocessing and if<br>
><br>
> > I watch the memory usage with the ps/top commands it grows only at the<br>
><br>
> > timestep where this->Processor->CoProcess() is called. My simulation<br>
><br>
> > program uses vtkImageData as underlying data structure so no data<br>
><br>
> > format adaptation (that may involve allocating extra memory for data<br>
><br>
> > copy) is performed.<br>
><br>
> ><br>
><br>
> > this->DataDescription->SetTimeData(this->TimeInfo->GetTimeValue(),<br>
><br>
> > this->TimeInfo->GetTIter());<br>
><br>
> > if (this->Processor->RequestDataDescription(this->DataDescription))<br>
><br>
> > {<br>
><br>
> > vtkCPInputDataDescription *idd<br>
><br>
> > = this->DataDescription->GetInputDescriptionByName("input");<br>
><br>
> > // obtain vtkImageData to coprocess from my simulation program<br>
><br>
> > vtkImageData *grid = this->TimeInfo->GetGrid();<br>
><br>
> > idd->SetWholeExtent(grid->GetWholeExtent());<br>
><br>
> > idd->SetGrid(grid);<br>
><br>
> > // now call the coprocessing library<br>
><br>
> > this->Processor->CoProcess(this->DataDescription);<br>
><br>
> > }<br>
><br>
> ><br>
><br>
> > Besides, the Apple's "leaks" tool shows no meaningful leaks which<br>
><br>
> > makes me wonder if e.g. there are un-unregistered proxies than simply<br>
><br>
> > unmatched New()/Delete() or new/delete pairs.<br>
><br>
> ><br>
><br>
> > The problem is same for PV 3.12.0 and the git master as of today.<br>
><br>
> > Is anybody else having the problem? Any ideas?<br>
><br>
> ><br>
><br>
> > Thanks,<br>
><br>
> > Takuya<br>
><br>
> ><br>
><br>
> > Takuya OSHIMA, Ph.D.<br>
><br>
> > Faculty of Engineering, Niigata University<br>
><br>
> > 8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN<br>
><br>
> _______________________________________________<br>
><br>
> Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
><br>
> Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/" target="_blank">
http://www.kitware.com/opensource/</a><br>
> opensource.html<br>
><br>
> Please keep messages on-topic and check the ParaView Wiki at: <a href="http://">
http://</a><br>
> <a href="http://paraview.org/Wiki/ParaView" target="_blank">paraview.org/Wiki/ParaView</a><br>
><br>
> Follow this link to subscribe/unsubscribe:<br>
><br>
> <a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br>
><br>
> *<br>
><br>
_______________________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">
http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">
http://paraview.org/Wiki/ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>