MantisBT - ParaView
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0006698ParaView(No Category)public2008-04-01 15:272011-01-13 17:00
ReporterBerk Geveci 
Assigned ToUtkarsh Ayachit 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionunable to reproduce 
PlatformOSOS Version
Product Version 
Target Version3.6Fixed in Version 
Project
Topic Name
Type
Summary0006698: Multi-block writer does not work in parallel
DescriptionvtkXMLCompositeDataWriter does not work in parallel. Each processor writes its blocks using a serial writer. If multiple processors share the same block, they all write to the same file, causing corruption. Maybe we should use parallel writers?
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0006779closed Utkarsh Ayachit Cannot save Exodus as anything 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2008-04-01 15:27Berk GeveciNew Issue
2008-04-01 16:20Berk GeveciNote Added: 0011020
2008-04-10 17:59Ken MorelandStatusbacklog => tabled
2008-04-10 17:59Ken MorelandAssigned To => Berk Geveci
2008-06-03 11:06Utkarsh AyachitAssigned ToBerk Geveci => Burlen
2008-08-07 13:17Utkarsh AyachitAssigned ToBurlen => Berk Geveci
2008-08-19 15:18Utkarsh AyachitRelationship addedrelated to 0006779
2008-08-27 15:37Berk GeveciAssigned ToBerk Geveci => Utkarsh Ayachit
2008-08-29 08:35Utkarsh AyachitStatustabled => @80@
2008-08-29 08:35Utkarsh AyachitResolutionopen => fixed
2008-08-29 08:35Utkarsh AyachitNote Added: 0013225
2008-09-08 17:08Alan ScottStatus@80@ => @20@
2008-09-08 17:08Alan ScottResolutionfixed => reopened
2008-09-08 17:08Alan ScottNote Added: 0013365
2009-02-17 14:09Ken MorelandCategory3.4 => 3.6
2009-02-20 10:46Utkarsh AyachitNote Added: 0015251
2009-02-20 10:46Utkarsh AyachitStatus@20@ => @80@
2009-02-20 10:46Utkarsh AyachitResolutionreopened => unable to reproduce
2009-03-02 21:34Alan ScottNote Added: 0015502
2009-03-02 21:34Alan ScottStatus@80@ => closed
2009-05-13 13:46Utkarsh AyachitTarget VersionMulti-block support => 3.6
2011-01-13 17:00Source_changeset_attached => VTK master a2bd8391
2011-01-13 17:00Source_changeset_attached => VTK master 020ef709
2011-06-16 13:10Zack GalbreathCategory => (No Category)

Notes
(0011020)
Berk Geveci   
2008-04-01 16:20   
Instead of using parallel writer (which is difficult due to the possibility of null or empty nodes), we could have the composite reader/writer handle this case. Every piece would make sure to assign a different filename and when more than one node have a block for the same node, the file would like the following:
<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0" byte_order="LittleEndian" compressor="vtkZLibDataCompressor">
  <vtkMultiBlockDataSet>
    <DataSet index="0" file="mb/mb_0.vtp"/>
    <Block index="1">
      <DataSet index="0" file="mb/mb_1_0.vtp"/>
      <DataSet index="0" file="mb/mb_1_1.vtp"/>
      <DataSet index="1" file="mb/mb_2.vtp"/>
    </Block>
  </vtkMultiBlockDataSet>
</VTKFile>

The reader would deal with appending datasets with the same index. Only datasets that can be appended would be supported (i.e. no structured datasets)
(0013225)
Utkarsh Ayachit   
2008-08-29 08:35   
vtkXMLCompositeDataWriter now works in parallel. Each
process now always writes out files with process number encoded into the
filename thus assuring that they are unique. Then when writing out the meta-file
the root process locates all leaf nodes that have non-null datasets on more than
1 process and inserts a multi-piece at that location.

eg. writing out can.ex2 on 3 processes results in following XML

<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0" byte_order="LittleEndian" compressor="vtkZLibDataCompressor">
  <vtkMultiBlockDataSet>
    <Block index="0">
      <Piece index="0">
        <DataSet index="0" file="can/can_0_0.vtu"/>
        <DataSet index="1" file="can/can_1_0.vtu"/>
        <DataSet index="2" file="can/can_2_0.vtu"/>
      </Piece>
      <Piece index="1">
        <DataSet index="0" file="can/can_0_1.vtu"/>
        <DataSet index="1" file="can/can_1_1.vtu"/>
        <DataSet index="2" file="can/can_2_1.vtu"/>
      </Piece>
    </Block>
    <Block index="1"/>
    <Block index="2"/>
    <Block index="3"/>
    <Block index="4"/>
    <Block index="5"/>
    <Block index="6"/>
    <Block index="7"/>
  </vtkMultiBlockDataSet>
</VTKFile>


/cvsroot/ParaView3/ParaView3/VTK/IO/vtkXMLCompositeDataWriter.cxx,v <-- VTK/IO/vtkXMLCompositeDataWriter.cxx
new revision: 1.3; previous revision: 1.2
/cvsroot/ParaView3/ParaView3/VTK/IO/vtkXMLCompositeDataWriter.h,v <-- VTK/IO/vtkXMLCompositeDataWriter.h
new revision: 1.2; previous revision: 1.1
/cvsroot/ParaView3/ParaView3/VTK/Parallel/vtkXMLPMultiBlockDataWriter.cxx,v <-- VTK/Parallel/vtkXMLPMultiBlockDataWriter.cxx
new revision: 1.2; previous revision: 1.1
/cvsroot/ParaView3/ParaView3/VTK/Parallel/vtkXMLPMultiBlockDataWriter.h,v <-- VTK/Parallel/vtkXMLPMultiBlockDataWriter.h
new revision: 1.2; previous revision: 1.1
(0013365)
Alan Scott   
2008-09-08 17:08   
Pushed back for feedback. I sent comments in a separate e-mail, along with an example bad dataset.
(0015251)
Utkarsh Ayachit   
2009-02-20 10:46   
Alan,

Can you retest this? I cannot reproduce the problem anymore.

Thanks.
(0015502)
Alan Scott   
2009-03-02 21:34   
Tested client/ remote server.