ParaQ:Multiview

Overview

Ideally, multiview rendering in ParaQ will enable a client to request multiple render windows from the server manager. Each window could then be placed in its own QVTKWidget and be placed like any other widget in the ParaQ client.

In order to support multiple render contexts, the render server must somehow support the requests for rendering different contexts. Opening multiple render windows on the server is problematic since overlapping windows can cause undesired behavior by graphics hardware. Sharing a render window can be achieved by either multiplexing the renderers or laying the renderers out. The latter option was chosen because this will work well in tile display mode and the client should easily be able to provide the layout inevitably created for its own GUI.

Implementing multiview will require changes in IceT, the server manager's render modules, and other parts of VTK and ParaView.

Multiple Renderers in IceT

As of ParaView 2.4, vtkIceTRenderManager only supports one renderer, which is expected to be a vtkIceTRenderer. As a hack, vtkIceTRenderManager will ignore all but the first renderer, allowing ParaView to overlay annotation in other renderers that is drawn after the first renderer is composited. The other renderers are not vtkIceTRenderer objects so that they are not composited.

New Support in vtkIceTRenderManager

vtkIceTRenderManager has been changed to support multiple vtkIceTRenderers, all of which will be composited correctly so long as they do not overlap.

Renderers that are not vtkIceTRenderers are also supported. They can overlap each other or even the vtkIceTRenderers and everything will work just fine. The one caveat is that all the non vtkIceTRenderers must be on a level that is higher than any vtkIceTRenderer. That is, the rendering must be set up such that all the vtkIceTRenderers get rendered before any non-vtkIceTRenderer.

Technical Details

Much of the implementation was achieved simply by iterating over all the renderers and finding all of the vtkIceTRenderers (as opposed to just looking at the first one). Some of the state held in vtkIceTRenderManager (such as the IceT context) had to be different for each vtkIceTRenderer, so that state was moved into the vtkIceTRenderer class.

When a renderer's viewport did not cover the full render window, the tile layout given to IceT had to be modified such that it covered the part of the global display related to the viewport. This is done simply by intersecting the renderer's viewport (in global coordinates) with each tile's viewport, culling out empty intersections. In places where the viewport divided a tile, this often lead to tiles of varying size. IceT handles this just fine so long as all rendering contexts can render full tile images. vtkIceTRenderer assures this by calling glViewport with the entire rendering context and disabling the scissor test. Having a tile split into renderers also requires the pieces to be cached and reconstructed, as the composite of each renderer obliterates the image from the previous one. An end render callback on all vtkIceTRenderers grabs the composited image and copies it to the appropriate place in the reduced image buffer.

In order to support non-composited overlays, the composited image needed to be drawn back into the render window before any overlays are rendered. This is done by attaching an observer to the start render event on all the non-composited renderers. In this callback, the composited image is drawn back to the render window. An ivar makes sure it only happens once. Assuming that all non-composited renderers are rendered after all composited vtkIceTRenderers (as is listed in a requirement above), the composited image will be drawn correctly.

Ripple Effect

For technical reasons, changes in the VTK/IceT rendering classes caused changes in other classes. This section documents some of the changes that occurred.

Renderer Synchronization

As part of its operation, vtkParallelRenderManager (the superclass of vtkIceTRenderManager) synchronizes some of its state in the "root" process with that on other processes. Previously, only the state of the first renderer was synchronized. In order for multiple renderers to be supported in vtkIceTRenderManager, all the renderers had to by synchronized.

Synchronizing all of the renderers requires the renderers to match exactly on all of the render windows. This is not always desirable, or even possible. For example, right now the client has extra renderers in its render window for annotations that the server does not have.

To get around this issue, you can now specify a subset of renderers that vtkParallelRenderManager will synchronize. This is what is used for the vtkDesktopDeliveryClient/Server between the client and server.

Server Side Annotation Stripping

vtkParallelRenderManager has methods that return the rendered/composited image. These methods will return a buffer in memory if it exists or read the image from the OpenGL buffer if not. The intent is to allow reuse of a composited buffer in memory to avoid multiple OpenGL buffer reads, which can be expensive.

This mechanism is supported in vtkIceTRenderManager, but there is a catch. The composited image in memory only has images generated from vtkIceTRenderers, not from any of the non-composited renderers. As far as ParaView is concerned, this means that the annotation is stripped off of the image.

This is actually a happy accident since we want the annotation stripped off so that it can be drawn on the client side. However, if we need the annotation available on server-generated images (for example, for an animation), then we need some extra logic to ensure that we read the images from the OpenGL frame buffer.

Known Bugs

Although there may be many bugs hidden in the new render manager code, here are the known issues or potential problems.

• When switching the image reduction factor (as is often done when switching between interactive and still renders), the image for any renderer that is not re-rendered may be clobbered for a tile display mode.
• vtkIceTRenderManager does not support node 0 having multiple render windows with the rest having one render window. This means that ParaQ will not support parallel jobs without client server.

Disabling Renderers

When calling render on a render window, it calls render on all of its renderers. In our client/server mode this is not always desirable. When calling render on one of the render windows on the client, this would lead to re-rendering the image for all render windows.

To get around this, a flag called Draw has been added to vtkRenderer. When Draw is off, the Render method does nothing. Thus, the render module can render the data for a single render window on the client side by turning off the rendering on renderers associated with all other client-side render windows.

Desktop Delivery

The original vtkDesktopDeliveryClient/Server pair of objects assumes that there is one render window on each side of the socket. We need them to use multiple render windows on the client side that share a single render window on the server side. To make migration easier, a new set of classes called vtkPVDesktopDeliveryClient/Server have been created. The new classes are mostly a copy of the old classes. The changes are documented here.

Window Identification

Because there are multiple client objects that connect to the same server object, the client objects must be identified. vtkPVDesktopDeliveryClient has an Id ivar. The id should be set to a number identifying it. The numbers used do not matter so long as no id is used twice.

vtkPVDesktopDeliveryServer has a method AddRenderer(int id, vtkRenderer *ren) that is used to associate renderers on the server side to the render window on the client side.

Render Window Layout

The client must specify how the render windows are lain out in a parent GUI window (or maybe the desktop). There are two functions to do this. First, specify the size of the parent GUI window with GUISize. Second, specify the location of the render window in the parent GUI window with WindowPosition. The size of the render window is taken from the render window itself.

WindowPosition uses a coordinate system where the pixel in the upper left hand corner is <math>(0, 0)</math>. This is flipped in the y coordinate from what OpenGL and VTK use for pixel coordinates. So, for a window in the upper left corner, its window position should be <math>(0, 0)</math>. For a window in the lower left corner, its window position should be <math>(0, h_{GUI} - h_{rw})</math> where <math>h_{GUI}</math> is the height of the parent GUI window and <math>h_{rw}</math> is the height of the render window.

MultiView in Server Manager

Server manager API now support multiple views. At VTK level, each view consists of a render window + renderers (client) and renderers (server) pair. There is only one render window on the server side. The render modules were modified to support this. Each view is managed by one render module and all render modules are created and initialized by a multi-view render module (note that only LODRenderModule, IceTRenderModule and IceTDesktopRenderModule support multiple views). The first thing to do is to create multi-view render module an initialize it:

multiViewRenderModule = 
  vtkSMMultiViewRenderModuleProxy::SafeDownCast(
    proxy_manager->NewProxy("rendermodules", "MultiViewRenderModule"));
const char* renderModuleName = options->GetRenderModuleName();
  
if (!renderModuleName)
  {
  // User didn't specify the render module to use.
  if (options->GetTileDimensions()[0])
    {
    // Server/client says we are rendering for a Tile Display.
    // Now decide if we must use IceT or not.
    renderModuleName = "IceTRenderModule";
    }
  else if (options->GetClientMode())
    {
    renderModuleName = "IceTDesktopRenderModule";
    }
  else
    {
    // Not running in Client Server Mode.
    // Use local information to choose render module.
    renderModuleName = "LODRenderModule";
    }
  }
multiViewRenderModule->SetRenderModuleName(renderModuleName);

Next, create render modules (one per view):

vtkSMProxy* view1 = multiViewRenderModule->NewRenderModule();
view1->UpdateVTKObjects();
vtkSMProxy* view2 = multiViewRenderModule->NewRenderModule();
view2->UpdateVTKObjects();
vtkSMProxy* view3 = multiViewRenderModule->NewRenderModule();
view3->UpdateVTKObjects();
vtkSMProxy* view4 = multiViewRenderModule->NewRenderModule();
view4->UpdateVTKObjects();

Now display proxies can be created and added to the render modules. Note that we still have to tell the render modules the size and the position of the view. On the client side, this is done by inserting the render window of each render module in a QVTKWidget:

window1 = new QVTKWidget(this->mainGrid);
vtkRenderWindow* renderWindow = 
   view1->GetRenderWindow();
window1->SetRenderWindow(renderWindow);
window1->update();

However, this is not enough. The server objects have to be notified of the size and position changes. This can't be done automatically because VTK and the server manager do not know anything about the position of the views inside the GUI. In the small example I wrote, I first added an event filter to the QVTKWidget:

class pqResizeEventWatcher : public QObject
{
public:
  pqMainWindow* MainWindow;
  pqResizeEventWatcher() : MainWindow(0) {}
protected:
  bool eventFilter(QObject* object, QEvent* event)
    {
      if (event->type() == QEvent::Resize)
        {
        if (this->MainWindow)
          {
          this->MainWindow->PositionViews();
          }
        }
      return false;
    }
};
this->EventWatcher = new pqResizeEventWatcher;
window1->installEventFilter(this->EventWatcher);

In the callback, I resized the views as follows:

void pqMainWindow::PositionViews()
{
  vtkSMIntVectorProperty* prop = 0;
For each view:
  // Set the gui size. Here MainWindow is the window that contains
  // all QVTKWidgets. This is the size of the server side render window.
  // Make sure to set the GUISize on ALL render modules. Here this->Views
  // is the vector of all render modules.
  prop = vtkSMIntVectorProperty::SafeDownCast(
    this->Views[n]->GetProperty("GUISize"));
  if (prop)
    {
    prop->SetElements2(this->MainWindow->width(), this->MainWindow->height());
    }
  // Set the position of Nth view. This is the position relative to
  // the MainWindow (upper left).
  prop = vtkSMIntVectorProperty::SafeDownCast(
    this->Views[n]->GetProperty("WindowPosition"));
  if (prop)
    {
    prop->SetElements2(this->QVTKWidgets[n]->pos().x(), 
                       this->QVTKWidgets->pos().y());
    }
  
}

I put together this example by copying, pasting and modifying from a test program that I created (by modified the old Hello World). If something does not work as expected, please contact me. I might have missed something. You can find more information in the header files of vtkSMMultiViewRenderModuleProxy and vtkSMIceTDesktopRenderModuleProxy.