Hi Frank,<br><br>I'm glad you got a workaround in place. We'll put this in the bug tracker so it can get fixed. ParaView should catch the exception or warning and report it properly without hanging!<br><br>For those following along, here is a simple way to trigger the hang:<br>
<br>import numpy<br>numpy.ones(1)/0<br><br>In a terminal, numpy will print "RuntimeWarning: divide by zero encountered in divide", but in ParaView the program will hang. The actual hang is coming from a tricky interaction between ParaView and Python's C-API. In fact, depending on your version of numpy, or whether it was built with thread support, this might not hang. Last month I sent an email to the paraview-developers mailing list with the subject line "Scipy with ParaView is broken" where I describe the issue in more detail. Unfortunately, the hang that Frank encountered is another case of the same bug sneaking up and biting someone!<br>
<br>Pat<br><br><br><div class="gmail_quote">On Mon, Sep 10, 2012 at 1:42 PM, Frank Horowitz <span dir="ltr"><<a href="mailto:frank.horowitz@cornell.edu" target="_blank">frank.horowitz@cornell.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi All,<br>
<br>
OK, attaching the debugger found the problem.<br>
<br>
My data were underflowing from the squaring operation, and the numpy exception was not being caught or explicitly ignored.<br>
<br>
My workaround in the programmable filter is to call numpy's seterr function and it's square function.<br>
<br>
The relevant code snippet looks like:<br>
<br>
seterr(all='ignore')<br>
<br>
m = sqrt(square(x) + square(y) + square(z))<br>
<br>
<br>
Strangely enough, calling seterr(under='ignore') still hangs with an uncaught numpy underflow exception, but "all='ignore'" works.<br>
<br>
This kind of problem is probably old hat to experienced paraview users, but perhaps paraview can be setup to catch and report such exceptions to avoid the kind of surprises I faced as a new user? (I'll leave it up to you to decide whether this should be considered a bug report or a feature request.)<br>
<br>
In any case, for completeness in case it helps someone else via Googling a similar problem, the full code that works for me is:<br>
<br>
--- code start ---<br>
x = inputs[0].PointData['Scalars_'] # replace the literal string 'Scalars_' with whatever your data is named up the pipeline<br>
y = inputs[1].PointData['Scalars_'] # Ditto<br>
z = inputs[2].PointData['Scalars_'] # Ditto<br>
<br>
seterr(all='ignore') # Instruct numpy to ignore exceptions.<br>
m = sqrt(square(x)+square(y)+square(z))<br>
<br>
output.PointData.append(m,'magnitude')<br>
<br>
--- code end ---<br>
<br>
Thanks again for all of the help!<br>
<br>
Cheers,<br>
Frank Horowitz<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
On 06/09/2012, at 1:47 PM, Berk Geveci wrote:<br>
<br>
> Just to make sure that I understand this: you have some code that reads<br>
> some files in a proprietary format and spits out 1 vtu file per component?<br>
> What is that format like? What is the script written in? It should be<br>
> pretty easy to change that script to combine components together and write<br>
> them out as a vector. You could also calculate derived quantities there and<br>
> write them to the vtu file. Or you could still calculate that using the<br>
> Programmable Filter.<br>
><br>
> On Thu, Sep 6, 2012 at 1:19 PM, Frank Horowitz<br>
> <<a href="mailto:frank.horowitz@cornell.edu">frank.horowitz@cornell.edu</a>>wrote:<br>
><br>
>> Thanks for the replies!<br>
>><br>
>> At the moment, I need to finish a poster for a presentation on Saturday,<br>
>> but promise to get to this Sunday or Monday. (I'll need to re-build a<br>
>> debugging version of paraview. Any hints on how to accomplish this within<br>
>> ccmake/cmake will be gratefully received, since that system is new to me<br>
>> too. Is it as simple as setting a CCFLAGS environment variable before<br>
>> invoking make -- or something similar??)<br>
>><br>
>> I also found that the 200**3 wavelet worked fine on my<br>
>> paraview-compiled-from-source Linux installation. Let's ignore the problem<br>
>> I found on the OSX box with the binary installer for the moment. (One<br>
>> problem at a time!)<br>
>><br>
>> Because of that, I'm now suspicious of the upstream code that writes .vtu<br>
>> files.<br>
>><br>
>> Also, responding here to your later comment from your next email:<br>
>><br>
>>> I am going to disagree with Andy about this being a better solution. I<br>
>>> don't even think that it addresses what Frank is trying to do. But beyond<br>
>>> that Array Calculator is cumbersome and slow compared to the Python<br>
>>> Calculator or Python Programmable Filter.<br>
>>><br>
>>> As far as I understand, Frank is not trying to combine components from<br>
>>> different files. He is trying to combine components written as different<br>
>>> arrays in the same file hence the need for sqrt(x*x + y*y + z*z). I think<br>
>>> that Andy is confused by inputs[0] - that's simple the way to access the<br>
>>> input (even if there is only one) of the Python Programmable Filter.<br>
>><br>
>> Actually, there is one file per component per timeslice.<br>
>><br>
>> I've had a quick look at the file format written by my upstream simulator.<br>
>> It's very simple. Maybe I should just write the .vtu files from within my<br>
>> own script that reads the upstream meshes to avoid any other potential<br>
>> problems with the upstream codes.<br>
>><br>
>> Does paraview.simple have an easy way to construct a field vector at the<br>
>> location of arbitrary points? If so, I think the Python script to do *that*<br>
>> will be faster than me trying to chase all of this down…<br>
>><br>
>> Thanks again for the advice!<br>
>><br>
>> Cheers,<br>
>> Frank Horowitz<br>
>><br>
>><br>
>><br>
>> On 06/09/2012, at 1:00 PM, Berk Geveci wrote:<br>
>><br>
>>> I just ran this with a 500^3 wavelet on my machine without any problems.<br>
>>> The whole thing (including a slice filter) takes 20-30 seconds but the<br>
>>> programmable filter is actually pretty much instantaneous (<0.1 seconds)<br>
>>> using a debug build from Git master. Is there any way you can attach to<br>
>> the<br>
>>> stuck ParaView with a debugger and look at the stack trace?<br>
>>><br>
>>> On Wed, Sep 5, 2012 at 9:13 PM, Frank Horowitz<br>
>>> <<a href="mailto:frank.horowitz@cornell.edu">frank.horowitz@cornell.edu</a>>wrote:<br>
>>><br>
>>>> Still more checking things out. A problem (THE problem???) can be<br>
>>>> demonstrated with your code snippet from a few messages back if you<br>
>>>> construct the Wavelet like so rather than using the default constructor:<br>
>>>><br>
>>>> Wavelet(WholeExtent=(-100,100,-100,100,-100,100))<br>
>>>><br>
>>>> N.B. this is on a different machine, with a stock Mac OSX binary<br>
>> version<br>
>>>> 3.14.1 installed.<br>
>>>><br>
>>>><br>
>>>> On 05/09/2012, at 5:49 PM, Frank Horowitz wrote:<br>
>>>><br>
>>>>> OK, more tracking the problem:<br>
>>>>><br>
>>>>> The assumption underlying my "close up" description below is not<br>
>>>> correct. The problem appears to be the actual _size_ of the dataset I'm<br>
>>>> feeding in. Even a single time step fed in to the filter (with some<br>
>> 866,000<br>
>>>> points) hangs at the x*x computation, even though that appeared to work<br>
>>>> perfectly fine with your 3D Wavelet as a source (only some 9,000<br>
>> points).<br>
>>>>><br>
>>>>> I'm guessing a memory leak somewhere? When was the last time paraview<br>
>>>> was run through valgrind?<br>
>>>>><br>
>>>>> Cheers (and sorry about the red herring earlier),<br>
>>>>> Frank<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> On 05/09/2012, at 4:42 PM, Frank Horowitz wrote:<br>
>>>>><br>
>>>>>> OK, the 30,000 foot view:<br>
>>>>>><br>
>>>>>> I have a time series of about 20 slices (currently) of values of each<br>
>>>> component (3 components total) of a vector displacement 3d spatial field<br>
>>>> from an elastic wave propagation simulation. At the moment, I'm simply<br>
>>>> trying to visualise the vector length to see the propagating phases of<br>
>> the<br>
>>>> wavefield. Because my upstream codes and scripts only write individual<br>
>>>> components of the vector field, I need to combine the components and<br>
>> then<br>
>>>> calculate the vector lengths *somewhere*. The programmable filter<br>
>> seemed<br>
>>>> like a good place to try that, since once I am past the learning curve<br>
>> for<br>
>>>> it, I'll have the full flexibilty of numpy for doing almost anything<br>
>> else I<br>
>>>> want with these and similar data from my upstream simulation codes.<br>
>>>>>><br>
>>>>>> Now the 10,000 foot view:<br>
>>>>>> I've been trying to simply calculate the sqrt(x*x + y*y + z*z) scalar<br>
>>>> field (where x, y, and z are the respective components as a function of<br>
>>>> position) and let that flow on down the pipeline as a new spatial<br>
>> field to<br>
>>>> be visualised. I assumed (correctly if I'm interpreting your answer<br>
>>>> correctly) that if I built the filter to deal with the spatial aspects<br>
>> of<br>
>>>> the calculation, the time series would be taken care of by paraview<br>
>>>> iterating over all files pointed to by inputs[0] (a set of files<br>
>> containing<br>
>>>> timesteps of the x components), inputs[1] (ditto for the y components),<br>
>> and<br>
>>>> inputs[2] (ditto for the z components). The set of *.vtu files for each<br>
>>>> component have filenames specifying component and time step (e.g.: x001,<br>
>>>> x002, x003, etc., y001, y002, y003, etc., and z001, z002, z003 etc.).<br>
>>>>>><br>
>>>>>> Now the close-up:<br>
>>>>>> As far as I can tell, after compiling from source from the git<br>
>>>> "release" branch (on an up-to-date Linux Mint distro -- which tracks<br>
>> Ubuntu<br>
>>>> Precise), my approach will work for a set of 3 single timestep arrays<br>
>> fed<br>
>>>> into the "ports" (or whatever paraview's terminology is for the inputs<br>
>> to a<br>
>>>> filter in the pipeline). In other words, a single timeslice<br>
>>>> (x001,y001,z001) appears on inputs[0:3] (numpy slice notation) and is<br>
>>>> processed correctly. (I'll go away and verify that shortly.) I *think*<br>
>> the<br>
>>>> "hangs/race-conditions/whatever" start when there are multiple timesteps<br>
>>>> presented to each input port. I was unsure whether the 'inputs[0],<br>
>>>> inputs[1], inputs[2]' syntax needed to be extended to deal with time<br>
>> series<br>
>>>> as above, and could not find documentation on the inputs[] array after<br>
>>>> extensive searching -- just a few snippets of example code that were all<br>
>>>> consistent with paraview dealing with the time series on its own.<br>
>>>>>><br>
>>>>>> I'm inexperienced enough with paraview to not know if I have a<br>
>>>> conceptual error with the way I'm approaching this problem, or if I have<br>
>>>> simply stumbled over a bug. That's why I've posted to the mailing list!<br>
>>>>>><br>
>>>>>> As always, your (and the rest of the list's) help is greatly<br>
>>>> appreciated!<br>
>>>>>><br>
>>>>>> Cheers,<br>
>>>>>> Frank<br>
>>>>>><br>
>>>>>><br>
>>>>>> On 05/09/2012, at 4:01 PM, Andy Bauer wrote:<br>
>>>>>><br>
>>>>>>> I'm not sure I completely understand about your inputs[0]. As for<br>
>>>> iterating<br>
>>>>>>> over time with the python programmable filter, inside the filter it<br>
>>>> will<br>
>>>>>>> only have access to values from a single time step at each invocation<br>
>>>> of<br>
>>>>>>> RequestData(). You can think of the order of operations if you have 5<br>
>>>> time<br>
>>>>>>> steps as:<br>
>>>>>>> 1) iterate over each of the 5 time steps (i.e. load a file and update<br>
>>>> the<br>
>>>>>>> pipeline with that data set)<br>
>>>>>>> 2) update the python programmable filter output for the current data<br>
>>>> set<br>
>>>>>>> resulting from the loaded file<br>
>>>>>>><br>
>>>>>>> If you want time statistics for your variables I think you probably<br>
>>>> want to<br>
>>>>>>> use the calculator filter to square the value at each point and then<br>
>>>> use<br>
>>>>>>> the temporal statistics filter to get an average of those values.<br>
>> After<br>
>>>>>>> that you can just multiply the results by the number of time steps.<br>
>>>>>>><br>
>>>>>>> If I'm misunderstanding what you're trying to do, I'd suggest giving<br>
>> a<br>
>>>>>>> description of what you're trying to do and maybe there's a simpler<br>
>>>> way of<br>
>>>>>>> doing it.<br>
>>>>>>><br>
>>>>>>> Andy<br>
>>>>>>><br>
>>>>>>> On Wed, Sep 5, 2012 at 2:56 PM, Frank Horowitz<br>
>>>>>>> <<a href="mailto:frank.horowitz@cornell.edu">frank.horowitz@cornell.edu</a>>wrote:<br>
>>>>>>><br>
>>>>>>>> OK Andy,<br>
>>>>>>>><br>
>>>>>>>> Thanks for that!<br>
>>>>>>>><br>
>>>>>>>> I've compiled and installed Paraview from git's release branch, and<br>
>> (a<br>
>>>>>>>> slight modification) of your code runs correctly. I think that means<br>
>>>> that<br>
>>>>>>>> my paraview/python integration is semi-sane.<br>
>>>>>>>><br>
>>>>>>>> However, my problem still occurs. Recall that I have a time series<br>
>> of<br>
>>>> .vtu<br>
>>>>>>>> files (indexed by integers built-in to their filenames) being fed<br>
>> into<br>
>>>>>>>> inputs[0] (at least as I understand it).<br>
>>>>>>>><br>
>>>>>>>> In my original code snippet, I assumed that the "m = x*x" statement<br>
>>>> would<br>
>>>>>>>> iterate not only over the point values of the spatial array, but<br>
>> also<br>
>>>> over<br>
>>>>>>>> all timeslices. Is that a correct assumption? Is that the source of<br>
>>>> my<br>
>>>>>>>> problems???<br>
>>>>>>>><br>
>>>>>>>> Thanks again for your help,<br>
>>>>>>>> Frank Horowitz<br>
>>>>>>>><br>
>>>>>>>> ------------------------------<br>
>>>>>>>> *From:* Andy Bauer [<a href="mailto:andy.bauer@kitware.com">andy.bauer@kitware.com</a>]<br>
>>>>>>>> *Sent:* Tuesday, September 04, 2012 7:57 PM<br>
>>>>>>>> *To:* Frank Horowitz<br>
>>>>>>>> *Cc:* <a href="mailto:paraview@paraview.org">paraview@paraview.org</a><br>
>>>>>>>> *Subject:* Re: [Paraview] Newbie question on Python Programmable<br>
>>>> Filters<br>
>>>>>>>><br>
>>>>>>>> Hi Frank,<br>
>>>>>>>><br>
>>>>>>>> I didn't have a problem with the following on my windows machine<br>
>>>> using the<br>
>>>>>>>> ParaView 3.14.1 installer.<br>
>>>>>>>> =======<br>
>>>>>>>> try: paraview.simple<br>
>>>>>>>><br>
>>>>>>>> except: from paraview.simple import *<br>
>>>>>>>><br>
>>>>>>>> paraview.simple._DisableFirstRenderCameraReset()<br>
>>>>>>>><br>
>>>>>>>> Wavelet1 = Wavelet()<br>
>>>>>>>><br>
>>>>>>>> RenderView1 = GetRenderView()<br>
>>>>>>>><br>
>>>>>>>> ProgrammableFilter1 = ProgrammableFilter()<br>
>>>>>>>><br>
>>>>>>>> ProgrammableFilter1.PythonPath = ''<br>
>>>>>>>><br>
>>>>>>>> ProgrammableFilter1.RequestInformationScript = ''<br>
>>>>>>>><br>
>>>>>>>> ProgrammableFilter1.Script = 'x =<br>
>>>> inputs[0].PointData[\'RTData\']\nprint<br>
>>>>>>>> "x shape =",x.shape\n\nm = x*x # I _thought_ this should run at<br>
>> numpy<br>
>>>>>>>> speeds?\nprint "m shape=",m.shape\n\n# more debugging code to finish<br>
>>>>>>>> computation of the norm<br>
>>>> omitted\n\noutput.PointData.append(m,"Displacement<br>
>>>>>>>> Norm")'<br>
>>>>>>>><br>
>>>>>>>> Show()<br>
>>>>>>>><br>
>>>>>>>> Render()<br>
>>>>>>>><br>
>>>>>>>> =============<br>
>>>>>>>><br>
>>>>>>>> Does this work for you? If it doesn't, I wonder if it's an issue<br>
>> with<br>
>>>>>>>> having multiple pythons and numpy or system environments causing<br>
>>>> problems.<br>
>>>>>>>> What version of paraview are you using and how did you get it<br>
>>>>>>>> built/installed on your machine?<br>
>>>>>>>><br>
>>>>>>>> Andy<br>
>>>>>>>><br>
>>>>>>>> On Tue, Sep 4, 2012 at 6:04 PM, Frank Horowitz <<br>
>>>> <a href="mailto:frank.horowitz@cornell.edu">frank.horowitz@cornell.edu</a><br>
>>>>>>>>> wrote:<br>
>>>>>>>><br>
>>>>>>>>> Hal Canary wrote on Tue Sep 4 15:22:01 EDT 2012:<br>
>>>>>>>>>> On 09/04/2012 02:20 PM, Frank Horowitz wrote:<br>
>>>>>>>>>>><br>
>>>>>>>>>> x = inputs[0].PointData['Scalars_']<br>
>>>>>>>>>><br>
>>>>>>>>>><br>
>>>>>>>>>><br>
>>>>>>>>>> I thought one needs to convert a vtkarray to a numpy array with<br>
>>>>>>>>>><br>
>>>>>>>>>> x = numpy.array(inputs[0].PointData['Scalars_'])<br>
>>>>>>>>>><br>
>>>>>>>>>> before doing anything with it.<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> Computations after that style of construction hang too. To my eye,<br>
>>>> the<br>
>>>>>>>>> results of such an expression should be 100% standard numpy,<br>
>> obeying<br>
>>>> 100%<br>
>>>>>>>>> standard numpy semantics.<br>
>>>>>>>>><br>
>>>>>>>>> It appears that there is a bug in numpy integration to the Python<br>
>>>>>>>>> Programmable Filter.<br>
>>>>>>>>><br>
>>>>>>>>> I'll re-compile from source and report back on the results. Is the<br>
>>>> bug<br>
>>>>>>>>> tracking system evident from the homepage?<br>
>>>>>>>>><br>
>>>>>>>>> Cheers,<br>
>>>>>>>>> Frank Horowitz<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> _______________________________________________<br>
>>>>>>>>> Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
>>>>>>>>><br>
>>>>>>>>> Visit other Kitware open-source projects at<br>
>>>>>>>>> <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
>>>>>>>>><br>
>>>>>>>>> Please keep messages on-topic and check the ParaView Wiki at:<br>
>>>>>>>>> <a href="http://paraview.org/Wiki/ParaView" target="_blank">http://paraview.org/Wiki/ParaView</a><br>
>>>>>>>>><br>
>>>>>>>>> Follow this link to subscribe/unsubscribe:<br>
>>>>>>>>> <a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br>
>>>>>>>>><br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>><br>
>>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
>>>><br>
>>>> Visit other Kitware open-source projects at<br>
>>>> <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
>>>><br>
>>>> Please keep messages on-topic and check the ParaView Wiki at:<br>
>>>> <a href="http://paraview.org/Wiki/ParaView" target="_blank">http://paraview.org/Wiki/ParaView</a><br>
>>>><br>
>>>> Follow this link to subscribe/unsubscribe:<br>
>>>> <a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br>
>>>><br>
>><br>
>><br>
<br>
_______________________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">http://paraview.org/Wiki/ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.paraview.org/mailman/listinfo/paraview" target="_blank">http://www.paraview.org/mailman/listinfo/paraview</a><br>
</div></div></blockquote></div><br>