View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0009808ParaViewFeaturepublic2009-10-30 14:032015-01-11 08:16
ReporterAlan Scott 
Assigned ToUtkarsh Ayachit 
PriorityhighSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version4.2Fixed in Version4.2 
Summary0009808: Ability to automatically select min, max, or average cell or point at a time step.
DescriptionWe need to add the functionality to have ParaView automatically select the cell or point that represents a variable's min or max or average value. This should work for all time steps - thus the selected cell or point would move around with time.
TagsNo tags attached.
ProjectSandia
Topic Nameremove_duplicate_tiff_readers
Typeincorrect functionality
Attached Files

 Relationships
related to 0010958closedUtkarsh Ayachit Edit/Find data command - find and calculate min and max variable for each time step of an animation 

  Notes
(0018304)
Alan Scott (manager)
2009-10-30 14:10

 A stretch goal would be to use the output of this filter as input to the plot selection over time filter.
(0022711)
Alan Scott (manager)
2010-10-26 17:10

 A user of mine just asked for this "stretch goal". We really do need to include my note above as part of this feature.
(0022712)
Alan Scott (manager)
2010-10-26 17:10

 A user of mine just asked for this "stretch goal". We really do need to include my note above as part of this feature.
(0029664)
Alan Scott (manager)
2012-11-07 21:33

 Not sure if this belongs assigned to Robert, but issuing a Todo. The Find Data dialog is important.
(0032481)
David C. Lonie (developer)
2014-03-18 08:40

 This bug is quite old, and seems to already be implemented:

1) Open can.ex2, load all arrays
2) Open find data dialog
3) Find Cells of can.ex2, set query to 'EQPS == max(EQPS)'
4) Walk through timesteps. The selection will updated.

Feel free to reopen if there is more work needed.
(0032483)
Utkarsh Ayachit (administrator)
2014-03-18 13:11

 (developers should never close bugs directly without going through gatekeeper review/customer review). I've also change workflow permissions to reflect the same.
(0032486)
Alan Scott (manager)
2014-03-19 20:03

 From an e-mail thread between Dave, Utkarsh and me:

From Alan:
> What I want is to have some very basic options such as min and max
> available as a button, or pull down menus, etc. Think of a very
> bright person that has never programmed in their life (such as a PhD
> in Geology or an MD doing medical research). If I were king, and i
> realize this is harder, I would like to be able to select the max and
> min cell per time step, and the max and min over all time.
>
> At the minimum - total minimum, we should have it trivial to find good
> quality documentation - with examples - of how this works.
>
> My/our rule of thumb is that if it doesn't work correctly in parallel,
> it is broken. And, misrepresenting data is significantly worse than
> not giving any data at all. And thanks for testing all the
> combinations (which is what I will do when I close it).

Regarding the query on the Find Data:

From Dave:
>
> BTW, the Query option is in a drop-down menu on the find data dialog.
> It's the first combo box in the second section of the dialog.

From Alan
>
> Ah, I found it. Big picture - I don't like
> it. We shouldn't have a list of variable names intermixed with a
> command. It is confusing. Could we move query to be its own line, or
> its own button, or something? At the least, could we create a space
> above Query and bold any commands (such as Query)?


From Dave:
>
> I'm still tracking down the issues with doing this on a distributed dataset.
> I got the calculations to run without crashing, but got different
> results depending on the data distribution (e.g. multiblock vs. single
> data set, with/without D3, etc). I'll hold off on the GUI elements
> until this is behaving properly.
>
>
(0032487)
Alan Scott (manager)
2014-03-19 20:04

 Nope, doesn't work. See notes from the e-mail thread below.
(0032551)
David C. Lonie (developer)
2014-04-08 10:55

 Fixed problems with the "arrayname == global_max(arrayname)" query on datasets with unevenly distributed data arrays by syncing array names and ensuring that all processors have the same names available in their scope.

This does not address the usability concerns raised by Alan. Further GUI work will be needed to expose the functionality in a user-friendly way, but now the underpinnings should be ready.
(0032579)
Utkarsh Ayachit (administrator)
2014-04-09 09:55

 SUMMARY
---------------------------------------------
Topics merged into master:
        14220-hide-decoration-button
(VTK) 14646-scalar-bar-borders
        14663_timesteps_indicator
        9808-distributed-query-selection
        catalyst-rgbz-plugin
(VTK) view-settings


---------------------------------------------
Topics reverted from next:
  expand_fragment_integration
(0032580)
Utkarsh Ayachit (administrator)
2014-04-09 09:55

 leaving this one open so I can test it before passing on to Alan.
(0032589)
David C. Lonie (developer)
2014-04-14 09:34

 9808-bugfix should fix the dashboard issues on blight.
(0032645)
Utkarsh Ayachit (administrator)
2014-05-17 22:23

 Argh, still doesn't work fully. The following query fails:

+ builtin mode
+ open can.ex2, move to timestep > 0 (e.g. 15)
+ run query for Points, VEL[:,1] == global_min(VEL[:,1]) -- we get two points selected -- min for each block is selected.
+ if you run Python calculator with expression "global_min(VEL[:,1])", it works fine.

+ It's curious that cell query, EQPS == global_min(EQPS) works just fine
(0032646)
Utkarsh Ayachit (administrator)
2014-05-17 22:29

 Some more insight into the problem:

Attached is a simple dataset with 2 spheres as 2 blocks with "Result" array set to 1 and 2 respectively.

The following query fails:
  Result[:,0] == global_max(Result[:,0])

The following succeeds:
  Result == global_max(Result)

The problem stems from the fact that Result[:,0] is a new array and hence the code doesn't treat it as a composite-array. I think we may have to give up on Python implementation for this and just and custom extract selection support for max/min/mean.
(0033034)
Utkarsh Ayachit (administrator)
2014-07-18 10:07

 Alright, resolving this again, fingers crossed.

The FindData dialog now has "is min", "is max", "is mean", "is mean within threshold" options that will help you find elements that min, max etc.

And it should work in parallel too.

Note, it uses NumPy.

Also another caveat: FindData by just IDs is currently not working and it will raise warnings. I'm working on it. Feel free to leave this bug open till that's resolved. I'm marking this "customer review" so that you can test the min/max functionality.
(0033039)
Alan Scott (manager)
2014-07-18 20:46

 Yes!! OK, I will test.

OK, tested! It died. I did the following:

master, Linux, remote server (8 pvservers).
Numpy support Client side: On (and using my numpy), version 1.7.1
Numpy support Server side: On (and using my numpy), version 1.6.1 (Could this be the problem?)
Python version matches (and is mine), 2.7.5.

  
Opened g1s1 (ask me, Do Not Release).
Find Data/ EQPS/ find max
Died. Here is what the client put out:

paraview: /.../ParaView4/ParaViewCore/ClientServerCore/Rendering/vtkSpreadSheetView.cxx:83: bool<unnamed>::OrderByNames::operator()(vtkAbstractArray*, vtkAbstractArray*): Assertion `a1Index == static_cast< int >(~0u >> 1) && a2Index == static_cast< int >(~0u >> 1)' failed.

Server side just grumbled that the socket had died.
(0033042)
Alan Scott (manager)
2014-07-18 21:11

 Tested remote server, can.exo, and disk_out_ref.exo. Worked great!
(0033045)
Utkarsh Ayachit (administrator)
2014-07-21 14:23

 Grrr...this seems to work for me. Can you try again please? Also make sure you start with a clean build on client and server. The python code build/install logic is a bug wacky.
(0033049)
Alan Scott (manager)
2014-07-21 21:23

 Still asserts. I replicated it as follows:

Deleted all .config/ParaView/* files. Master as of today, clean build, Linux, 8 pvservers, client debug, client running under totalview. Assert is from the client.

Open g1s1. There will be 16 files. All variables on. Apply.
Click on the Find Data icon (I didn't use the Edit/ Find Data menu.)
Change Global Id to EQPS. Change "is" to "Is Max". Click on Run Selection Query. Assert.

We hit an assert in vtkSpreadSheetView.cxx, line 83. Basically, a1->GetName() and a2->GetName() don't equal anything in the order[] array. I believe that a1 and a2 GetName() is just null. Thus, a result is never found, and the assert asserts.

Furthermore, in vtkSpreadSheetView.cxx, vtkSpreadSheetView::GetNumberOfColumns(), I believe that this->Internals->ActiveRepresentation and MostRecentlyAccessedBlock may be bad. Sending Utkarsh a screenshot at this point. Note that I am showing this, then what is in *(this), then *(this->Internals).

Next, showing the call stack, and what is in FetchBlock. Notice that block index == 0. Not sure where else to go from here... :-)
(0033077)
Utkarsh Ayachit (administrator)
2014-07-24 16:12

 commit f6dc4055de7af65df0e1bde427ca09416611bf42
Author: Utkarsh Ayachit <utkarsh.ayachit@kitware.com>
Date: Thu Jul 24 16:11:40 2014 -0400

    Fix segfault seen in BUG 0009808.
    
    On certain platforms, the comparison operator is called for sorting with
    same arguments. We skipped that case in our "assert". Fixed that.
    
    Change-Id: I8fdc496d2b078878dca21eca64b8b8ce3547285
(0033092)
Utkarsh Ayachit (administrator)
2014-07-25 09:35

 SUMMARY
---------------------------------------------
Topics merged into master:
        13575_PV_PLUGIN_DEBUG
        14771_fix_global_annotation_filter
        14885_python_view_crash
        fix_apply_pqProxyWidgetDialog
        fix_get_proxies
        fix-int64_t
(VTK) remove_duplicate_tiff_readers
        remove_matplotlib_utilities
(0033112)
Alan Scott (manager)
2014-07-28 18:17

 Spectacular!!!!!

Tested g1s1, max EQPS, remote server, Linux, master.

 Issue History
Date Modified Username Field Change
2009-10-30 14:03 Alan Scott New Issue
2009-10-30 14:10 Alan Scott Note Added: 0018304
2010-08-30 22:22 Alan Scott Relationship added related to 0010958
2010-09-01 11:32 Utkarsh Ayachit Target Version => 3.10.shortlist
2010-10-26 17:10 Alan Scott Note Added: 0022711
2010-10-26 17:10 Alan Scott Note Added: 0022712
2010-12-01 16:53 Utkarsh Ayachit Assigned To => Robert Maynard
2010-12-01 16:53 Utkarsh Ayachit Status backlog => tabled
2010-12-01 16:53 Utkarsh Ayachit Priority normal => high
2011-06-16 13:10 Zack Galbreath Category Feature Request => Feature
2011-06-24 14:24 Alan Scott Status expired => backlog
2011-06-24 14:24 Alan Scott Resolution open => reopened
2011-06-24 14:24 Alan Scott Status backlog => todo
2011-06-27 08:32 Utkarsh Ayachit Project => Sandia
2012-05-31 10:23 Utkarsh Ayachit Status todo => backlog
2012-11-07 21:33 Alan Scott Note Added: 0029664
2012-11-07 21:33 Alan Scott Status backlog => todo
2012-12-11 09:47 Sebastien Jourdain Assigned To Robert Maynard => David Thompson
2014-01-16 17:00 Utkarsh Ayachit Type => incorrect functionality
2014-01-16 17:00 Utkarsh Ayachit Assigned To David Thompson => David C. Lonie
2014-01-16 17:00 Utkarsh Ayachit Target Version => 4.2
2014-03-18 08:40 David C. Lonie Note Added: 0032481
2014-03-18 08:40 David C. Lonie Status todo => closed
2014-03-18 08:40 David C. Lonie Resolution reopened => fixed
2014-03-18 08:40 David C. Lonie Fixed in Version => git-master
2014-03-18 13:10 Utkarsh Ayachit Status closed => customer review
2014-03-18 13:11 Utkarsh Ayachit Note Added: 0032483
2014-03-19 20:03 Alan Scott Note Added: 0032486
2014-03-19 20:04 Alan Scott Note Added: 0032487
2014-03-19 20:04 Alan Scott Status customer review => todo
2014-04-08 10:55 David C. Lonie Topic Name => 9808-distributed-query-selection
2014-04-08 10:55 David C. Lonie Note Added: 0032551
2014-04-08 10:55 David C. Lonie Status todo => gatekeeper review
2014-04-08 10:55 David C. Lonie Resolution fixed => open
2014-04-09 09:55 Utkarsh Ayachit Status gatekeeper review => customer review
2014-04-09 09:55 Utkarsh Ayachit Note Added: 0032579
2014-04-09 09:55 Utkarsh Ayachit Assigned To David C. Lonie => Utkarsh Ayachit
2014-04-09 09:55 Utkarsh Ayachit Note Added: 0032580
2014-04-09 09:55 Utkarsh Ayachit Status customer review => gatekeeper review
2014-04-09 09:55 Utkarsh Ayachit Resolution open => fixed
2014-04-14 09:33 David C. Lonie Status gatekeeper review => active development
2014-04-14 09:34 David C. Lonie Topic Name 9808-distributed-query-selection => 9808-distributed-query-selection, 9808-bugfix
2014-04-14 09:34 David C. Lonie Note Added: 0032589
2014-04-14 09:34 David C. Lonie Status active development => gatekeeper review
2014-05-17 22:23 Utkarsh Ayachit Note Added: 0032645
2014-05-17 22:23 Utkarsh Ayachit Status gatekeeper review => todo
2014-05-17 22:29 Utkarsh Ayachit Note Added: 0032646
2014-07-18 10:05 Utkarsh Ayachit Topic Name 9808-distributed-query-selection, 9808-bugfix => fix_find_data
2014-07-18 10:05 Utkarsh Ayachit Status todo => gatekeeper review
2014-07-18 10:07 Utkarsh Ayachit Note Added: 0033034
2014-07-18 10:07 Utkarsh Ayachit Status gatekeeper review => customer review
2014-07-18 20:46 Alan Scott Note Added: 0033039
2014-07-18 21:11 Alan Scott Note Added: 0033042
2014-07-21 14:23 Utkarsh Ayachit Note Added: 0033045
2014-07-21 21:23 Alan Scott Note Added: 0033049
2014-07-24 16:12 Utkarsh Ayachit Note Added: 0033077
2014-07-24 16:13 Utkarsh Ayachit Topic Name fix_find_data => remove_duplicate_tiff_readers
2014-07-24 16:13 Utkarsh Ayachit Status customer review => gatekeeper review
2014-07-25 09:35 Utkarsh Ayachit Status gatekeeper review => backlog
2014-07-25 09:35 Utkarsh Ayachit Note Added: 0033092
2014-07-25 09:38 Utkarsh Ayachit Status backlog => customer review
2014-07-28 18:17 Alan Scott Note Added: 0033112
2014-07-28 18:17 Alan Scott Status customer review => closed
2014-08-29 10:59 Utkarsh Ayachit Fixed in Version git-master => 4.2
2015-01-11 08:15 Utkarsh Ayachit Source_changeset_attached => ParaView master 879dbb3d
2015-01-11 08:16 Utkarsh Ayachit Source_changeset_attached => ParaView master f6dc4055

Copyright © 2000 - 2018 MantisBT Team