MantisBT - ParaView
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0010958ParaViewFeaturepublic2010-07-08 12:312014-08-29 10:59
ReporterAlan Scott 
Assigned ToUtkarsh Ayachit 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version4.2Fixed in Version4.2 
ProjectSandia
Topic Name10958_query_seln_over_time
Typeincorrect functionality
Summary0010958: Edit/Find data command - find and calculate min and max variable for each time step of an animation
DescriptionThe Edit/ Find data command is a fabulous tool. However, it really needs a few more features.

* The ability to find the min and/or max variable, along with cell number, at a time step. Mean would be really nice to have also.
* Be able to take this min and/or max and display it in tabular format. Allow it to be 2d plotted. Allow it to be written out as a CSV file.

Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0013150closed Utkarsh Ayachit Simplify "Find Data" dialog 
related to 0010968closed David Thompson Freeze Selection action in Find Data query dialog 
related to 0009808closed Utkarsh Ayachit Ability to automatically select min, max, or average cell or point at a time step. 
related to 0013794closed David C. Lonie Remote server, find max error 
child of 0010627closed David Thompson Additional functionality for find data 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2010-07-08 12:31Alan ScottNew Issue
2010-07-08 12:35Alan ScottRelationship addedchild of 0010627
2010-08-30 22:22Alan ScottRelationship addedrelated to 0009808
2010-09-01 11:25Utkarsh AyachitTarget Version3.10 => 3.10.shortlist
2011-06-16 13:10Zack GalbreathCategoryFeature Request => Feature
2012-05-22 19:27Alan ScottRelationship addedrelated to 0013150
2012-05-22 22:38Utkarsh AyachitAssigned To => Utkarsh Ayachit
2012-05-22 22:38Utkarsh AyachitStatusexpired => backlog
2012-05-22 22:38Utkarsh AyachitResolutionopen => reopened
2012-11-07 21:31Alan ScottNote Added: 0029663
2012-11-07 21:31Alan ScottStatusbacklog => todo
2012-11-07 21:32Alan ScottNote Edited: 0029663bug_revision_view_page.php?bugnote_id=29663#r523
2012-11-07 21:41Alan ScottProject => Sandia
2012-11-07 21:41Alan ScottType => incorrect functionality
2012-12-11 09:50Sebastien JourdainAssigned ToUtkarsh Ayachit => David Thompson
2012-12-11 12:36David ThompsonNote Added: 0029867
2012-12-12 10:24Ken MorelandNote Added: 0029871
2012-12-17 18:41David ThompsonNote Added: 0029898
2012-12-17 18:42David ThompsonRelationship addedrelated to 0010968
2012-12-31 18:02David ThompsonNote Added: 0029966
2013-01-09 12:39David ThompsonTopic Name => 10958_query_seln_over_time
2013-01-09 12:39David ThompsonNote Added: 0030042
2013-01-09 12:39David ThompsonStatustodo => gatekeeper review
2013-01-09 12:39David ThompsonResolutionreopened => fixed
2013-01-11 14:42Utkarsh AyachitFixed in Version => git-master
2013-01-11 14:43Utkarsh AyachitStatusgatekeeper review => customer review
2013-01-11 14:43Utkarsh AyachitNote Added: 0030106
2013-01-30 10:40Utkarsh AyachitFixed in Versiongit-master => 3.98.1
2013-02-13 12:51Alan ScottRelationship addedrelated to 0013794
2013-02-13 12:55Alan ScottNote Added: 0030341
2013-12-24 19:42Alan ScottNote Added: 0032067
2013-12-24 19:42Alan ScottAssigned ToDavid Thompson => Utkarsh Ayachit
2013-12-24 19:42Alan ScottStatuscustomer review => todo
2014-01-16 13:46Utkarsh AyachitAssigned ToUtkarsh Ayachit => David C. Lonie
2014-03-07 09:12David C. LonieStatustodo => active development
2014-07-25 07:49David C. LonieNote Added: 0033087
2014-07-25 07:49David C. LonieStatusactive development => todo
2014-07-25 09:38Utkarsh AyachitAssigned ToDavid C. Lonie =>
2014-07-25 09:39Utkarsh AyachitTarget Version => 4.2
2014-07-29 16:27Alan ScottNote Added: 0033132
2014-07-29 16:27Alan ScottStatustodo => closed
2014-07-29 16:27Alan ScottAssigned To => Utkarsh Ayachit
2014-07-29 16:27Alan ScottFixed in Version3.98.1 => 4.2
2014-07-31 09:21Utkarsh AyachitFixed in Version4.2 => git-master
2014-08-29 10:59Utkarsh AyachitFixed in Versiongit-master => 4.2

Notes
(0029663)
Alan Scott   
2012-11-07 21:31   
(edited on: 2012-11-07 21:32)
Find data needs to be cleaned up. This is extremely important functionality that we have been asking for for a long time - needs to be implemented.
(0029867)
David Thompson   
2012-12-11 12:36   
The request says "Be able to take this min and/or max and display it in tabular format. Allow it to be 2d plotted. Allow it to be written out as a CSV file." The spreadsheet view now in the selection inspector should display the matching cells or points at a single time step.

1. What would the axes for a 2-d plot include besides the scalar value? Time?

2. Using Extract Selection from the selection inspector should allow min or max values to be written to a CSV now. Or is this also intended to mean "over time"?
(0029871)
Ken Moreland   
2012-12-12 10:24   
The functionality of Find Data has changed since this bug was originally written. Most of the implementation is there. That is, you can ask for the min or max of a field and see the values in the spreadsheet contained in the Find Data dialog. However, the plotting over time functionality is broken. To replicate, do the following:

1. Load can.ex2. Go to the last time step.
2. Open Find Data.
3. Construct the query to Find Cell from can.ex2 where EQPS is max.
4. Click Run Selection Query. The results show up in the spreadsheet (so far so good).
5. Click Plot Selection Over Time. Apply the filter.

Fail. I get a bunch of errors like this:

ERROR: In /Users/kmorel/src/ParaView/VTK/Filters/Extraction/vtkExtractSelection.cxx, line 365
vtkExtractSelection (0x1160d79a0): Unrecognized CONTENT_TYPE: 9
(0029898)
David Thompson   
2012-12-17 18:41   
The "Unrecognized CONTENT_TYPE" errors exist because vtkExtractSelection (in VTK, and used by vtkExtractArraysOverTime, also in VTK) does not know about QUERY selections; they are implemented in ParaView as a new type of vtkSelectionNode. This is also the cause of issue 10968 (no way to "freeze" query selections) as vtkExtractSelection is used to obtain IDs from the query.

There are 2 strategies to deal with this: (1) migrate query-selection code from PV into VTK or (2) subclass things in VTK for use in PV.

(1) Migrating query-selection code to VTK would also require vtkPythonProgrammableFilter be moved, which in turn would require vtkPVPythonInterpretor be broken into a subclass that could be moved to VTK. In order to keep VTK/Filters/Extraction from depending on Python, this would likely also involve some kind of "vtkInterpreterFactory" in Common/Core that could produce a vtkPythonInterpreter when linked to the proper kits.

(2) All of the filters in VTK that use vtkExtractSelection would have to be subclassed for use by ParaView, including vtkExtractSelection, vtkExtractArraysOverTime, vtkPExtractArraysOverTime, and probably vtkConvertSelection (for bug 10968). The methods in these classes that use vtkExtractSelection are generally pretty hefty -- so either we add virtual methods to the VTK classes to construct selection extractors or duplicate a fair amount of code.
(0029966)
David Thompson   
2012-12-31 18:02   
A workaround for query selections (see 2012-12-17 note above) eliminates the "Unrecognized CONTENT_TYPE" error. However, there is still an issue with the use case in Ken's 2012-12-12 note: query selections such as the maximum value can return different points/cells and different numbers of points/cells at each time step.

In the face of multiple points/cells per timestep, how should the array extractor work? Is it necessary for the time-history plot to accommodate multiple values per time step? Would reporting a single (arbitrary) attribute value from each time step be adequate? If not, then would a min/max/mean/variance be adequate (this won't always make sense -- e.g., GlobalID or other index attribute)?

Also, vtkExtractArraysOverTime currently outputs 1 table per (block,cell)-tuple in the selection (except for LOCATION selections). Should this be changed for QUERY selections? For simple queries, it makes sense to have one output table per queried attribute (e.g., a "global max" query on an attribute produces 1 output table, a ">=" query produces one table for each unique attribute value >= the user-specified value). However, for arbitrary query strings entered by the user, there would have to be one output table for each combination of "active" query clause and allowed value for that clause. For example, the query "(P < 10) & ((T > 323) | (V < 5))" has 3 leaf clauses: P < 10, T > 323, and S < 5. In a dataset where P is in {8,9,10}, T is in {320,330,340}, and P is in {3,4,5}, there might be output tables labeled "P=8,T=330", "P=8,T=340", "P=9,T=330", "P=9,T=330", "P=8,V=3", "P=8,V=3", "P=9,V=4", and "P=9,V=4". Furthermore, tuples that match both the T and V clauses would be repeated in multiple tables.

The simplest solution seems to be either a single output table per query, but in that case we must report some statistic(s) across all tuples at a given timestep.
(0030042)
David Thompson   
2013-01-09 12:39   
The 10958_query_seln_over_time branch fixes "Plot arrays over time" to support query selections (based on attribute values rather than ID or viewport selections). The underlying VTK classes have also been modified to compute statistics on the selection and report those instead of creating a vtkTable time-history for each point or cell in the selection. Statistics are always reported for query selections and an option is provided in the ParaView UI to obtain statistics for other selections.

The statistics include the interquartile boundaries (min, q1, median, q3, and max) and -- for numeric arrays -- the mean and standard deviation of all selected points/cells within a given time step. These values are reported for each time step.
(0030106)
Utkarsh Ayachit   
2013-01-11 14:43   
merged into master, where applicable.
(0030341)
Alan Scott   
2013-02-13 12:55   
This bug is waiting for 13794 to be resolved. since this is really a superset of that bug, I am keeping both open. When 13794 closes, retest this one.
(0032067)
Alan Scott   
2013-12-24 19:42   
Moving to "todo". We rolled this functionality out - since it was not working in parallel. Very important to get done.
(0033087)
David C. Lonie   
2014-07-25 07:49   
No longer active on this project.
(0033132)
Alan Scott   
2014-07-29 16:27   
This bug has been fixed in other bug reports. Lets close this one off, let users actually use this new code, and write new bugs/ feature requests at that time. Closing.