Scripting:Requirements

Overview

Users, developers, and managers (oh my) have requested a scripting interface to ParaQ. The ParaQ Scripting Group is responsible for capturing the requirements for scripting, and developing a design document to guide ParaQ development.

We must strike a balance between supporting scripting, and limiting the scope of work required by scripting.

Goals

The goals of ParaQ scripting are:

• To simplify developer testing through scripting.
• To enable users to automate repetative offline tasks.

Requirements

• Ability to apply a script to multiple different input files and save an image/movie of the visualization from each input file
• Scripting language is backwards compatible. Scripts written in a previous version of ParaQ must work in newer versions.
• Script files saved from ParaQ must work in the batch/command line program.
• Script files must be understandable by humans without needing 2 Tylenol.
• Either the scripting language or the pvCommandLine application must provide looping and flow-of-control.
• (Optional?) When a user determines that they like a set of scripted operations, they can attach the script to a button in ParaQ.

Use Cases

• A user makes a visualization in ParaQ and saves out a script file. The user then open the script file in pvCommandLine, twiddles some values and saves the image/movie of the visualization. Some values include filenames, variable names, values of variables.
• A user opens a toy dataset that models an extremely large dataset. The user creates a visualization and saves a script. The user then opens pvCommandLine and applies the script to the extremely large dataset
• An expert user can copy and paste from their scripts to new scripts to add steps.
• An expert user can set some reader parameters for one or possibly several pipeline elements, then re-run the script for different filenmanes ending up with an interactive interface to continue investigating the data (e.g. setting the metadatafile name or loading each sideset in a different pipeline element).
• A user makes a visualization in ParaQ and saves out a script file. The user then repeats the script for different input files (note: this case could be implemented by state files instead of editable script files.
• An expert user can edit a script to add mathematical (calculator) functions to create new variables and either save a new data file or proceed to an interactive visualization.
• (Low priority) A user opens a saved script file with ParaQ and steps through, visualizing what each command actually does to their data.

Current Capabilities

ParaView

ParaView currently uses or enables scripting in the following ways:

1. Wrapping: ParaView classes can be automatically wrapped in a variety of scripting languages. Tcl is the default scripting language. Note that both GUI and server manager (SM) classes are wrapped and can be scripted. Tests and demos use this scripting language to test GUI and SM.
2. Trace file: ParaView saves out a trace file, which is a Tcl/Tk script, that records every action taken by the user. The trace script manipulates both GUI and SM objects but it is mostly at GUI level. This trace file can be 'replayed' in ParaView, restoring the application to the final state of the script. In addition, this script can be hand-edited by expert users, who can make the script do just about anything. Hand-editing the scripts is difficult because they access internal GUI API that is not documented well. Backwards compatibility is poor since GUI API changes impact scripts. Trace file is loaded by the full application.
3. State file: This is a Tcl/Tk script that captures the current state of the application. This is very similar to trace files (with all the same pros and cons) but does not store intermediate steps. State file is loaded by the full application
4. Batch file: This is a Tcl (no Tk) script that is SM level only. Batch file is loaded by a separate application (pvbatch). pvbatch is very similar to the python SM application we have. pvbatch tends to be better at backwards compatibility (i.e. older scripts can be loaded by newer pvbatch). Some SM changes (specially render modules) have broken compatibility but we should be able to avoid these. Batch files are written out by the GUI classes not the SM.
5. SM state file: Server manager is capable of dumping out it's entire state as an XML file. Currently, this file is only used by ParaView Enterprise Edition (PVEE). It is almost identical to the Tcl batch file but we have not had time to merge the two together. This file is very verbose and contains the entire SM state.

Ensight

Ensight currently uses or enables scripting in the following ways:

1. External API: Ensight has an API that can be called by a variety of scripting languages. Metadata support scripts use this API to support metadata.
2. Context file: Ensight saves out a context file, that records the current state of ensight. This context file can be loaded in EnSight, restoring the application to the final state of the file. In addition, this scipt can be hand-edited by expert users, who can make the script do just about anything. Hand-editing the scripts is difficult because they access internal GUI API that is not documented well. Backwards compatibility is good.
3. Archive file: This is a file script that captures the current state of the application. This is very similar to context files (with all the same pros and cons) but does not store intermediate steps. Archive file is loaded by the full application. Archive files are not compatible between versions.
4. Command file: This is a file that records all the commands entered into the application in this session. It can be "replayed" to repeat all the commands. Expert users can edit this file to do anything they want. It is somewhat easier to edit than a context file and is even more compatible between versions. It is slower however as it does all the commands in the session even if they have no impact on the final result.
5. Scenario file: EnSight cna save a scenario file, which is much like a context file except that it is used only by the standalone EnLiten application to view the created scene.

Cubit

Cubit currently uses or enables scripting in the following ways:

1. Wrapping: SWIG is used to wrap a set of query functions in Cubit (mesh & geometry queries) and the Cubit command interface. SWIG is also used to wrap helper functions that interface with journaling of commands, playback of journals, loading/managing Qt extension modules, etc...
2. Extension Modules:PyQt can optionally be loaded to add to the GUI. C++ extensions wrapped with SWIG can optionally be loaded to add features.
3. Interactive Python prompt:A python prompt is provided to general functionality. In python, Cubit can be queried for objects and state. String based commands can be fed through Cubit to do desired operations. Step-by-Step playback through a python script and breakpoints are supported.

VisIt