VisIt avt Integration

From ParaQ Wiki
Jump to navigationJump to search

TODO


Introduction

This document describes the Visit Database Bridge ParaView plugin. Any references to "plugin" or "bridge" herein refer to this plugin. The motivation for the plugin is simple to allow ParaView to make use of some subset of VisIt's IO components. In ParaView IO components are called "readers" but in VisIt they are called "databases". Besides the nomenclature, there are a number of subtle differences between the two application's implementation of IO components. First VisIt provides a consistent interface both for database implementors and for programmers constructing pipelines using avt. Avt is waht VisIt's visualization pipeline library is called. VisIt database plugin authors implement one of 4 interfaces derived from avtFileFormat. We thus had the option of tying into the VisIt code base here. The file format interface provides a consistent albeit low level interface to any given database. We could simply rip out the various database source code and manipulate the code directly explicitly using this interface to programatically manipulate individual database classes. In this approach we'd likely end up writing a VTK pipeline for each VisIt database implementation. This alone is a lot of work as there are 84 databases currently available and would require a major restructuring of VisIt's source tree, resulting in the direct inclusion of the requisite source in ParaView. It very quickly became apparent that if we were to do this we'd reinvent a large amount of code that was already available in VisIt. Pulling source code from VisIt would introduce a maintainability nightmare as we would then have our own version of the VisIt database and various avt sources which would make upgrading versions of VisIt cumbersome to the point that such upgrades would likely never happen. Fortunately there was a second option, namely to manipulate the database plugins at the using the plugin interface. This turns out to be quite a bit more efficient. The database plugins are used with out alteration in essence using VisIt avt classes as a shared library to programatically manipulate the plugins in a general way. This approach should have additional benefit of making the process of upgrading VisIt as simple as installing the new version and reconfiguring the bridge. Unfortunately a number aspects of VisIt's design complicate this approach. The first and largest complication is that VisIt has forked VTK at release 5.0 circa 2006. was not designed as a library but rather ahas not (to our knowledge)

General Information and Notes

VisItSlideScreenShot.png

This file is a site-config file that I used to build VisIt as a library for use with ParaView. To do so VisIt's makefiles had to be patched so that its GUI is excluded and ParaView's VTK can be used. The top section of the file is a set of notes as to how I configured VisIt's dependencies and VisIt itself. The bottom of this file is a bash script that VisIt uses to configure itself. You will have to modify the variables to reflect locations of the dependencies on your system. If you are only interested in a specific database plugin then you can skip many of the dependencies. You can't skip szip, silo, or hdf5. Be sure to point ParaView to the version of HDF5 you are using here when you build ParaView.

Notes:

VisIt suggests to use static libraries for its dependencies. This isn't always a good idea. HDF5 for instance should be shared when using with ParaView. Theses notes need to be updated to reflect this. TODO

Windows has a completely different build process because VisIt used visual studio. Mac is as of yet uncharted territory.

In theory one can build all of VisIt's dependencies using the "build_visit" script available from VisIt's web site. If you go that route, then you still should use this file to configure the patched VisIt with the ./configure line below.

You'll need to have Qt 3 development packge around somewhere because VisIt's ./configure script has a Qt 3 dependency.

It's best to try ot use the versions of the libraries I have used here, if not then second best is to use the ones recomended by VisIt. If you do niether you're on your own, but things will probably be OK as long as you stick to the recomended configuration.

Which of VisIt's databases are available?

This depends largely on how VisIt is built because databases only are built if their dependencies are found. Our plugin handles this by scanning the VisIt build at compile time, and only generating server manager XML for those which are present. This list reflects what is available in a full VisIt 1.10.0 build. A number of these however are intentionally skipped since ParaView already has support for them. The databases are skipped based on the contents of a text file included in the plugin sources.

  • ANALYZE
  • ANSYS
  • AUXFile
  • AugDecomp
  • BOV
  • BOW
  • Boxlib2D
  • Boxlib3D
  • CEAucd
  • CGNS
  • CMAT
  • CTRL
  • Cale
  • Chombo
  • Claw
  • Cosmos
  • CosmosPP
  • Curve2D
  • DDCMD
  • Dune
  • Dyna3D
  • EnSight
  • Enzo
  • ExtrudedVol
  • FITS
  • FLASH
  • Fluent
  • GDAL
  • GGCM
  • GTC
  • H5Nimrod
  • H5Part
  • Hex
  • Image
  • KullLite
  • Lines
  • M3D
  • MFIX
  • MM5
  • Miranda
  • NASTRAN
  • NETCDF
  • Nek3D
  • OVERFLOW
  • OpenFOAM
  • PATRAN
  • PDB
  • PFLOTRAN
  • PLOT2D
  • PLOT3D
  • Pixie
  • PlainText
  • Point3D
  • ProteinDataBank
  • RAW
  • Rect
  • S3D
  • SAMI
  • SAMRAI
  • SAR
  • SAS
  • STL
  • Shapefile
  • Silo
  • SimV1
  • SimV1Writer
  • Spheral
  • StreamGhostTest
  • TFT
  • TSurf
  • Tecplot
  • Tetrad
  • UNIC
  • VASP
  • VLI
  • Vis5D
  • Vista
  • WavefrontOBJ
  • XDMF
  • XYZ
  • Xmdv
  • ZeusMP
  • ZipWrapper

Step 1: Build VisIt's dependencies

szip

LIBS=-lm CFLAGS=-fPIC CXXFLAGS=-fPIC ./configure --prefix=/opt/szip-2.1 --disable-shared
make -j 8
sudo make install

HDF4

LIBS=-lm CFLAGS=-fPIC CXXFLAGS=-fPIC ./configure --prefix=/opt/hdf4-4.2r3 --disable-fortran --with-szlib=/opt/szip-2.1/ 
make -j 8
sudo make install

HDF5

for gcc 4.3 you'll have to edit perform/zip_perf.c change line 549 to "output = open(filename, O_RDWR | O_CREAT, S_IRWXU);" 

CFLAGS=-fPIC CXXFLAGS=-fPIC ./configure --prefix=/opt/hdf5-1.6.8_ser --disable-shared --disable-fortran --disable-parallel --with-szlib=/opt/szip-2.1 
make -j 8
sudo make install
sudo ln -s /opt/szip-2.1/lib/libsz.a /opt/hdf5-1.6.8_ser/lib/libsz.a

BoxLib

For gcc-4.3 add #include <cstdlib> to ParallelDescriptor.cpp Usiing gfortran instead of g77 works well. 

cd into boxlib directory in the CCSEApps
edit GNUMakefile, set USE_MPI=false, COMP=g++
mv std std.old
chmod 644 *.H
CXXFLAGS=-fPIC CFLAGS=-fPIC FFLAGS=-fPIC make -j 8
sudo  mkdir -p /opt/boxlib/{include/2D,include/3D,lib}
sudo cp libbox3d.Linux.g++.f77.DEBUG.a /opt/boxlib/lib/libbox3D.a
sudo cp *.H /opt/boxlib/include/3D/
edit GNUMakefile,set DIM=2
CXXFLAGS=-fPIC CFLAGS=-fPIC FFLAGS=-fPIC make -j 8
sudo cp libbox2d.Linux.g++.f77.DEBUG.a /opt/boxlib/lib/libbox2D.a
sudo cp *.H /opt/boxlib/include/2D/

NetCDF

For gcc-4.3 add #include <cstring> to ./cxx/ncvalues.cpp 

CXXFLAGS=-fPIC CFLAGS=-fPIC ./configure --prefix=/opt/netcdf-3.6.0-p1
make -j 8
sudo mkdir /opt/netcdf-3.6.0-p1
sudo make install

Silo

4.6.2 doesn't work with VisIt and gcc 4.3 as of this writing(2009-02-25). Some sort of link issue, may be libtool.? 

./configure --prefix=/opt/silo-4.6.1 --without-readline --with-hdf5=/opt/hdf5-1.6.8_ser/include/,/opt/hdf5-1.6.8_ser/lib/ --without-exodus --with-szlib=/opt/szip-2.1 --disable-fortran --disable-browser --disable-shared --disable-silex
make -j 8
sudo make install
sudo ln -s /opt/silo-4.6.2/lib/libsiloh5.a /opt/silo-4.6.2/lib/libsilo.a

CGNS

Only use hdf5 if you need it.
CXXFLAGS=-fPIC CFLAGS=-fPIC ./configure --prefix=/opt/cgns-2.4 --with-szip=/opt/szip-2.1/lib/libsz.a --with-hdf5=/opt/hdf5-1.6.8_ser/
make -j 8
sudo mkdir -p /opt/cgns-2.4/{include,lib}
sudo make install

CFITSIO

./configure --prefix=/opt/cfitsio
make -j 8
sudo make install

H5Part

CFLAGS=-fPIC ./configure --prefix=/opt/h5part-1.3.3 --with-hdf5path=/opt/hdf5-1.6.8_ser/
make -j 8
sudo make install

CCMIO

Didn't work & looks like it uses qmake ?!?. If some one complains we'll get it working.

GDAL

For gcc-4.3 download gdal-1.6.0, the following config works with both version 

CFLAGS=-fPIC CXXFLAGS=-fPIC ./configure --prefix=/opt/gdal-1.6.0 --enable-static --disable-shared --with-libtiff=internal --with-gif=internal --with-png=internal --with-jpeg=internal --with-libz=internal --with-netcdf=no --without-jasper --without-python
make -j 8
sudo make install

Qt 3

We don't have to link against Qt but VisIt plugin system requires Qt to build. 

export QTDIR=`pwd`
export LD_LIBRARY_PATH=$QTDIR/lib
./configure --prefix=/opt/qt-3.3.8
make -j 8
sudo make install

Step 2: Build and install VisIt

Once some subset of VisIt's dependencies ahave been built on the target system it's time to build VisIt itself.


NOTE for now set VTK_USE_64BIT_IDS to OFF NOTE should we force the use of static libs when both shared and static are available? -Bstatic ??

patch -p6 < /home/burlen/ext/kitware_cvs/VisitPluginTool/VisItPV3Build.in.patch 

./configure --prefix=/opt/VisIt-1.10.0 --with-config=/home/burlen/ext/kitware_cvs/VisitPluginTool/building_visit/vtkVisitDatabaseBridge.conf --with-hdf5=/opt/hdf5-1.6.8_ser/include,/opt/hdf5-1.6.8_ser/lib --enable-parallel --disable-scripting --disable-visitmodule --disable-viewer-mesa-stub --disable-icet --disable-bilib --disable-glew --disable-bzip2 --with-dbs=all

Step 3: Build the ParaView plugin

Building the database plugin for ParaView is a two step process, namely: 

cd bin;
ccmake ../ ===> Set BOOTSTRAP to ON
make
ccmake ../ ===> Set BOOSTRAP to OFF
make

Step 1: 

  BootstrapConfigure.
     During this pass a local copy of the VisIt libraries and database
     plugins are made. On Windows a copy of the "third party dependencies"
     is also made. This is done to facilitate the crossplatform build
     and configuration which is necessary due the large differences between
     the VisIt build system accross platforms, and to mitigate differneces
     in how various platforms handle shared libraries at run time. The final
     step of this pass is to generate the CMake files, ServerManager and 
     Pq xml for the vtkVisItDatabaseBridge plugin upon the VisIt configuration
     found.

Step 2: 

  vtkVisItDatabaseBridge ParaView Plugin.
    During this pass the paraview plugin is built from the configuration
    generated during the first pass.

Windows

Configure PV for shared libs, using system HDF5. Point both ZLIB and HDF5 include and lib to HDF5 of VisIt/windowsbuild.

C:\VisitPluginTool\plugin\bin\Debug\MSVC8.Net\Debug

set PATH=%PATH%;C:\VisitPluginTool\plugin\bin\Debug;C:\VisitPluginTool\plugin\bin\Debug\MSVC8.Net\Debug;C:\VisitPluginTool\plugin\bin\Debug\MSVC8.Net\Debug\databases;C:\VisitPluginTool\plugin\bin\Debug\MSVC8.Net\ThirdParty


What's Not Done

  1. We need to map VisIt ghost cells to VTK ghost cells.
  2. Finish MIR. Part of the MIR code has been written (it's commented out). Basically we need to inquire with the VisIt devs about what is the right way to do this.
  3. Finish Experssions.Part of the expression code has been written (it's commented out). Expressions will be expected by VisIt users.They can be trivially added. See the avtExpresssionEvaluationFilter.
  4. Leaks! Run the plugin through valgrind and you'll find all kinds of leaks. These are certainly coming from within VisIt, however we have to

be certain it's not ourfault before asking them for help. I've been careful but now need to go over thoroughly.

Retrieved from "http://www.paraview.org/ParaView/index.php?title=VisIt_avt_Integration&oldid=6268"