View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0014541ParaView(No Category)public2014-01-25 14:322016-08-12 09:59
Reporterj 
Assigned ToKitware Robot 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionmoved 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0014541: Possibly faullty data offset counting for Base64 encoded data in AppendData-section
DescriptionDear All,

Paraview 13.12.0 (32bit) seems to read VTK AppendedData in a
peculiar and possibly faulty way.

According to the documentation of VTK XML format, the data
(later the raw data) of each

  <DataArray .... format="appended" offset="N"/>
  <DataArray .... format="appended" offset="N2"/>
  <DataArray .... format="appended" offset="N3"/>

section is stored in <AppendedData> section as follows: It
starts with the total length of the data expressed as int32
(MMMM). It is followed by the data from each of the
DataArray sections in a continuous manner (AAAA,BBBB,CCCCC..):
 
  <AppendedData encoding="raw">
  _MMMMAAAAAAAAAAAAAAAABBBBCCCCCCCCCCCCCCCCCCCC
  </DataArray>

The offset expresses the location of each section; N bytes
offset from the beginning of the first data section (N=0 ~
33th bit). Here N_A=0,N_B=16,N_C=20, M=40.

However, if the data is base64 encoded,

  <AppendedData encoding="Base64">
  _Base64[MMMMAAAAAAAAAAAAAAAABBBBCCCCCCCCCCCCCCCCCCCC]
  </DataArray>

every byte (8bits) in the encoded string correspond to six
bytes of the raw data. Again the offset is based again at
the start of the actual data (N=0 is now the 45th bit).
However, the offset N correspond to N chars of encoded
array, that is 6N bit or 3N/4 bytes of raw data.

As a result the offset does not in general correspond to any
byte-boundary of the raw data and never corresponds to a
char-boundary of the encoded data.

In addition, to allow reference to the start of a data
section, previous data has to be padded to a multiple of 3
bytes; or as float32 corresponds to 4 bytes, the data has to
padded to a multiple of 12 bytes (3xfloats32).


Questions:

 1) Is this true for VTK+base64 at large or only paraview

 2) If not for which versions of VTK / paraview

 3) Is this a stable arrangement - should I write my code to adhere?

I attach an example of the bit alignment and a vts-file for
testing (without debugger, change the alignment of rho-data
and save-data in csv). This is data is for debugging reasons
stored as UInt8, but the phenomenon is independent of the
data type.

 Paraview 13.12.0 (32bit)
 Linux 3.2.0-58-generic-pae #88-Ubuntu SMP
 2013 i686 i686 i386 GNU/Linux

With preciation for any comments.

J.

EXAMPLE OF THE ALIGNMENT
========================================
|running bit
      |C = byte.bit encoded data
             |D = byte.bit orginal data
                   |comment
========================================
---- byte 0 in base64 code -------
  0 0.8
  1 0.7
  2 0.6 0.8 data length, byte 0
  3 0.5 0.7
  4 0.4 0.6
  5 0.3 0.5
  6 0.2 0.4
  7 0.1 0.3
---- byte 1 in base64 code -------
  8 1.8
  9 1.7
 10 1.6 0.2
 11 1.5 0.1
 12 1.4 1.8 data length, byte 1
 13 1.3 1.7
 14 1.2 1.6
 15 1.1 1.5
---- byte 2 in base64 code -------
 16 2.8
 17 2.7
 18 2.6 1.4
 19 2.5 1.3
 20 2.4 1.2
 21 2.3 1.1
 22 2.2 2.8 data length, byte 2
 23 2.1 2.7
---- byte 3 in base64 code -------
 24 3.8
 25 3.7
 26 3.6 2.6
 27 3.5 2.5
 28 3.4 2.4
 29 3.3 2.3
 30 3.2 2.2
 31 3.1 2.1
---- byte 4 in base64 code -------
 32 4.8
 33 4.7
 34 4.6 3.8 data length, byte 3
 35 4.5 3.7
 36 4.4 3.6
 37 4.3 3.5
 38 4.2 3.4
 39 4.1 3.3
---- byte 5 in base64 code -------
 40 5.8
 41 5.7
 42 5.6 3.2
 43 5.5 3.1
 44 5.4 4.8 offset=0; data byte 0;
 45 5.3 4.7
 46 5.2 4.6
 47 5.1 4.5
---- byte 6 in base64 code -------
 48 6.8
 49 6.7
 50 6.6 4.4
 51 6.5 4.3
 52 6.4 4.2 offset=1;
 53 6.3 4.1
 54 6.2 5.8 data byte 1;
 55 6.1 5.7
---- byte 7 in base64 code -------
 56 7.8
 57 7.7
 58 7.6 5.6
 59 7.5 5.5
 60 7.4 5.4 offset=2;
 61 7.3 5.3
 62 7.2 5.2
 63 7.1 5.1
---- byte 8 in base64 code -------
 64 8.8
 65 8.7
 66 8.6 6.8 data byte 2;
 67 8.5 6.7
 68 8.4 6.6 offset=3;
 69 8.3 6.5
 70 8.2 6.4
 71 8.1 6.3
---- byte 9 in base64 code -------
 72 9.8
 73 9.7
 74 9.6 6.2
 75 9.5 6.1
 76 9.4 7.8 offset=4; data byte 3;
 77 9.3 7.7
 78 9.2 7.6
 79 9.1 7.5
---- byte 10 in base64 code -------
 80 10.8
 81 10.7
 82 10.6 7.4
 83 10.5 7.3
 84 10.4 7.2 offset=5;
 85 10.3 7.1
 86 10.2 8.8 data byte 4;
 87 10.1 8.7
---- byte 11 in base64 code -------
 88 11.8
 89 11.7
 90 11.6 8.6
 91 11.5 8.5
 92 11.4 8.4 offset=6;
 93 11.3 8.3
 94 11.2 8.2
 95 11.1 8.1
---- byte 12 in base64 code -------
 96 12.8
 97 12.7
 98 12.6 9.8 data byte 5;
 99 12.5 9.7
100 12.4 9.6 offset=7;
101 12.3 9.5
102 12.2 9.4
103 12.1 9.3
---- byte 13 in base64 code -------
104 13.8
105 13.7
106 13.6 9.2
107 13.5 9.1
108 13.4 10.8 offset=8; data byte 6;
109 13.3 10.7
110 13.2 10.6
111 13.1 10.5
112 13.0 10.4







TagsBase64, VTK, XML
ProjectTBD
Topic Name
Typeincorrect functionality
Attached Files? file icon test1.vts [^] (581 bytes) 2014-01-25 14:32
? file icon test.vts [^] (2,225 bytes) 2014-01-25 21:17

 Relationships

  Notes
(0032217)
j (reporter)
2014-01-25 15:18
edited on: 2014-01-25 21:16

 Correction

The attached correct testfile test.vts, the data section contains numbers
1 to 1250 modulo 87. The offset=11 gives the coordinates thecorrect values and the "rho" variable obtains the following values from the original data array:

|00100100|00101000|00101100|00110000|00110100 = bits
|36______|40______|44______|48______|52______ = rho
_______|10______|11______|12______|13______|_ = original data
(0038502)
Kitware Robot (administrator)
2016-08-12 09:59

 Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current ParaView Issues page linked in the banner at the top of this page.

 Issue History
Date Modified Username Field Change
2014-01-25 14:32 j New Issue
2014-01-25 14:32 j File Added: test1.vts
2014-01-25 14:38 j Tag Attached: Base64
2014-01-25 14:38 j Tag Attached: VTK
2014-01-25 14:38 j Tag Attached: XML
2014-01-25 15:18 j Note Added: 0032217
2014-01-25 15:21 j Note Added: 0032218
2014-01-25 21:16 j Note Edited: 0032217
2014-01-25 21:16 j Note Deleted: 0032218
2014-01-25 21:17 j File Added: test.vts
2016-08-12 09:59 Kitware Robot Note Added: 0038502
2016-08-12 09:59 Kitware Robot Status backlog => closed
2016-08-12 09:59 Kitware Robot Resolution open => moved
2016-08-12 09:59 Kitware Robot Assigned To => Kitware Robot

Copyright © 2000 - 2018 MantisBT Team