git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: PyArrow and Parquet DELTA_BINARY_PACKED


Hi Uwe,

I'm quite confused by the findings, Im attaching a bunch of files corresponding to the version and library generating the files.

On the first topic of DELTA_BINARY_PACKED. It seems it's something not well supported on the Java side as well or my implementation is off, but I just copied over the "CsvParquetWriter.java". I created a sample encoder based on parquet-mr and it seems if dictionary is used, it falls back to PLAIN instead of DELTA when the dict gets too big. Regardless what I do I can't make it use DELTA for the attached schema.

In terms of size difference I see the issue in the resulting metadata, but not the root cause. You will see the code is identical with just the addition of version="2.0". This results in changing the output file metadata from "ENC:PLAIN_DICTIONARY,PLAIN,RLE" to "ENC:RLE,PLAIN" hence increasing the size quite substantially.

Let me know if there's anything else I can provide to help debug this one. The second part is not critical since I can just use v1 for now, but good to figure out why the output changes. The first part is a bit more pressing for me since I really want to assess the difference between RLE and DELTA_BINARY_PACKED on monotonically increasing values like a timestamp, ticking at a constant rate.

On Sun, May 13, 2018 at 11:58 AM, Uwe L. Korn <uwelk@xxxxxxxxxx> wrote:
Hello Feras,

`DELTA_BINARY_PACKED` is at the moment only implemented in parquet-cpp on the read path. The necessary encoder implementation for this code is missing at the moment.

The change in file size is something I also don't understand. The only difference between the two versions is that with version 2, we encode uint32 columns in version 1 as INT64 whereas in version two, we can encode them as UINT32. This type was not available in version 1. It would be nice, if you could narrow down the issue to e.g. the column which causes the increase in size. You might also use the Java parquet-tools or parquet-cli to inspect the size statistics of the parts of the individual Parquet file.

Uwe

On Fri, May 11, 2018, at 3:07 AM, Feras Salim wrote:
> Hi, I was wondering if I'm missing something or currently the
> `DELTA_BINARY_PACKED` is only available for reading when it comes to
> parquet files, I can't find a way for the writer to encode timestamp data
> with `DELTA_BINARY_PACKED`, furthermore I seem to get about 10% increase in
> final file size when I change from ver 1 to ver 2 without changing anything
> else about the schema or data.

file:        file:/Users/feras/Desktop/SPY_nodict_java.parquet
creator:     parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a)

file schema: m
--------------------------------------------------------------------------------
date:        REQUIRED INT64 O:TIMESTAMP_MILLIS R:0 D:0
type:        REQUIRED BINARY R:0 D:0
ticker:      REQUIRED BINARY R:0 D:0
price:       REQUIRED INT32 R:0 D:0
size:        REQUIRED INT32 R:0 D:0
exchange:    REQUIRED BINARY R:0 D:0
conditions:  REQUIRED BINARY R:0 D:0

row group 1: RC:2340100 TS:134648137 OFFSET:4
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:4 SZ:18722040/18722040/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
type:         BINARY UNCOMPRESSED DO:0 FPO:18722044 SZ:31300350/31300350/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
ticker:       BINARY UNCOMPRESSED DO:0 FPO:50022394 SZ:16381482/16381482/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
price:        INT32 UNCOMPRESSED DO:0 FPO:66403876 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
size:         INT32 UNCOMPRESSED DO:0 FPO:75764751 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
exchange:     BINARY UNCOMPRESSED DO:0 FPO:85125626 SZ:21439994/21439994/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
conditions:   BINARY UNCOMPRESSED DO:0 FPO:106565620 SZ:28082521/28082521/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN

row group 2: RC:2340100 TS:134572689 OFFSET:134648141
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:134648141 SZ:18722040/18722040/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
type:         BINARY UNCOMPRESSED DO:0 FPO:153370181 SZ:31248705/31248705/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
ticker:       BINARY UNCOMPRESSED DO:0 FPO:184618886 SZ:16381482/16381482/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
price:        INT32 UNCOMPRESSED DO:0 FPO:201000368 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
size:         INT32 UNCOMPRESSED DO:0 FPO:210361243 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
exchange:     BINARY UNCOMPRESSED DO:0 FPO:219722118 SZ:21416191/21416191/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
conditions:   BINARY UNCOMPRESSED DO:0 FPO:241138309 SZ:28082521/28082521/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN

row group 3: RC:2340100 TS:134566071 OFFSET:269220830
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:269220830 SZ:18722040/18722040/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
type:         BINARY UNCOMPRESSED DO:0 FPO:287942870 SZ:31298597/31298597/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
ticker:       BINARY UNCOMPRESSED DO:0 FPO:319241467 SZ:16381482/16381482/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
price:        INT32 UNCOMPRESSED DO:0 FPO:335622949 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
size:         INT32 UNCOMPRESSED DO:0 FPO:344983824 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
exchange:     BINARY UNCOMPRESSED DO:0 FPO:354344699 SZ:21359681/21359681/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
conditions:   BINARY UNCOMPRESSED DO:0 FPO:375704380 SZ:28082521/28082521/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN

row group 4: RC:2340100 TS:134625251 OFFSET:403786901
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:403786901 SZ:18722040/18722040/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
type:         BINARY UNCOMPRESSED DO:0 FPO:422508941 SZ:31331647/31331647/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
ticker:       BINARY UNCOMPRESSED DO:0 FPO:453840588 SZ:16381482/16381482/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
price:        INT32 UNCOMPRESSED DO:0 FPO:470222070 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
size:         INT32 UNCOMPRESSED DO:0 FPO:479582945 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
exchange:     BINARY UNCOMPRESSED DO:0 FPO:488943820 SZ:21385811/21385811/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
conditions:   BINARY UNCOMPRESSED DO:0 FPO:510329631 SZ:28082521/28082521/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN

row group 5: RC:2340100 TS:134609900 OFFSET:538412152
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:538412152 SZ:18722040/18722040/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
type:         BINARY UNCOMPRESSED DO:0 FPO:557134192 SZ:31303657/31303657/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
ticker:       BINARY UNCOMPRESSED DO:0 FPO:588437849 SZ:16381482/16381482/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
price:        INT32 UNCOMPRESSED DO:0 FPO:604819331 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
size:         INT32 UNCOMPRESSED DO:0 FPO:614180206 SZ:9360875/9360875/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
exchange:     BINARY UNCOMPRESSED DO:0 FPO:623541081 SZ:21398450/21398450/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN
conditions:   BINARY UNCOMPRESSED DO:0 FPO:644939531 SZ:28082521/28082521/1.00 VC:2340100 ENC:BIT_PACKED,PLAIN

row group 6: RC:863879 TS:49763046 OFFSET:673022052
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:673022052 SZ:6911513/6911513/1.00 VC:863879 ENC:BIT_PACKED,PLAIN
type:         BINARY UNCOMPRESSED DO:0 FPO:679933565 SZ:11570893/11570893/1.00 VC:863879 ENC:BIT_PACKED,PLAIN
ticker:       BINARY UNCOMPRESSED DO:0 FPO:691504458 SZ:6047445/6047445/1.00 VC:863879 ENC:BIT_PACKED,PLAIN
price:        INT32 UNCOMPRESSED DO:0 FPO:697551903 SZ:3455726/3455726/1.00 VC:863879 ENC:BIT_PACKED,PLAIN
size:         INT32 UNCOMPRESSED DO:0 FPO:701007629 SZ:3455726/3455726/1.00 VC:863879 ENC:BIT_PACKED,PLAIN
exchange:     BINARY UNCOMPRESSED DO:0 FPO:704463355 SZ:7954707/7954707/1.00 VC:863879 ENC:BIT_PACKED,PLAIN
conditions:   BINARY UNCOMPRESSED DO:0 FPO:712418062 SZ:10367036/10367036/1.00 VC:863879 ENC:BIT_PACKED,PLAIN
file:           file:/Users/feras/Desktop/SPY.gz.v2.parquet
creator:        parquet-cpp version 1.4.0
extra:          pandas = {"index_columns": ["Date_Timestamp"], "column_indexes": [{"name": null, "pandas_type": "unicode", "metadata": {"encoding": "UTF-8"}, "numpy_type": "object", "field_name": null}], "pandas_version": "0.22.0", "columns": [{"name": "EventType", "pandas_type": "categorical", "metadata": {"ordered": false, "num_categories": 7}, "numpy_type": "int8", "field_name": "EventType"}, {"name": "Price", "pandas_type": "float64", "metadata": null, "numpy_type": "float64", "field_name": "Price"}, {"name": "Quantity", "pandas_type": "int32", "metadata": null, "numpy_type": "int32", "field_name": "Quantity"}, {"name": "Exchange", "pandas_type": "categorical", "metadata": {"ordered": false, "num_categories": 13}, "numpy_type": "int8", "field_name": "Exchange"}, {"name": "Conditions", "pandas_type": "categorical", "metadata": {"ordered": false, "num_categories": 44}, "numpy_type": "int8", "field_name": "Conditions"}, {"name": "Date_Timestamp", "pandas_type": "datetime", "metadata": null, "numpy_type": "datetime64[ns]", "field_name": "Date_Timestamp"}]}

file schema:    schema
--------------------------------------------------------------------------------
EventType:      OPTIONAL BINARY O:UTF8 R:0 D:1
Price:          OPTIONAL DOUBLE R:0 D:1
Quantity:       OPTIONAL INT32 R:0 D:1
Exchange:       OPTIONAL BINARY O:UTF8 R:0 D:1
Conditions:     OPTIONAL BINARY O:UTF8 R:0 D:1
Date_Timestamp: OPTIONAL INT64 O:TIMESTAMP_MILLIS R:0 D:1

row group 1:    RC:12564380 TS:26780923 OFFSET:4
--------------------------------------------------------------------------------
EventType:       BINARY GZIP DO:0 FPO:4 SZ:1958899/168054889/85.79 VC:12564380 ENC:RLE,PLAIN
Price:           DOUBLE GZIP DO:0 FPO:1958976 SZ:2803854/100520415/35.85 VC:12564380 ENC:RLE,PLAIN
Quantity:        INT32 GZIP DO:0 FPO:4762920 SZ:12627390/50259823/3.98 VC:12564380 ENC:RLE,PLAIN
Exchange:        BINARY GZIP DO:0 FPO:17390387 SZ:6256981/114955481/18.37 VC:12564380 ENC:RLE,PLAIN
Conditions:      BINARY GZIP DO:0 FPO:23647436 SZ:550474/150780424/273.91 VC:12564380 ENC:RLE,PLAIN
Date_Timestamp:  INT64 GZIP DO:0 FPO:24197985 SZ:2583325/100520415/38.91 VC:12564380 ENC:RLE,PLAIN
file:           file:/Users/feras/Desktop/SPY.gz.v1.parquet
creator:        parquet-cpp version 1.4.0
extra:          pandas = {"index_columns": ["Date_Timestamp"], "column_indexes": [{"name": null, "pandas_type": "unicode", "metadata": {"encoding": "UTF-8"}, "numpy_type": "object", "field_name": null}], "pandas_version": "0.22.0", "columns": [{"name": "EventType", "pandas_type": "categorical", "metadata": {"ordered": false, "num_categories": 7}, "numpy_type": "int8", "field_name": "EventType"}, {"name": "Price", "pandas_type": "float64", "metadata": null, "numpy_type": "float64", "field_name": "Price"}, {"name": "Quantity", "pandas_type": "int32", "metadata": null, "numpy_type": "int32", "field_name": "Quantity"}, {"name": "Exchange", "pandas_type": "categorical", "metadata": {"ordered": false, "num_categories": 13}, "numpy_type": "int8", "field_name": "Exchange"}, {"name": "Conditions", "pandas_type": "categorical", "metadata": {"ordered": false, "num_categories": 44}, "numpy_type": "int8", "field_name": "Conditions"}, {"name": "Date_Timestamp", "pandas_type": "datetime", "metadata": null, "numpy_type": "datetime64[ns]", "field_name": "Date_Timestamp"}]}

file schema:    schema
--------------------------------------------------------------------------------
EventType:      OPTIONAL BINARY O:UTF8 R:0 D:1
Price:          OPTIONAL DOUBLE R:0 D:1
Quantity:       OPTIONAL INT32 R:0 D:1
Exchange:       OPTIONAL BINARY O:UTF8 R:0 D:1
Conditions:     OPTIONAL BINARY O:UTF8 R:0 D:1
Date_Timestamp: OPTIONAL INT64 O:TIMESTAMP_MILLIS R:0 D:1

row group 1:    RC:12564380 TS:22128787 OFFSET:4
--------------------------------------------------------------------------------
EventType:       BINARY GZIP DO:4 FPO:93 SZ:1037494/4727557/4.56 VC:12564380 ENC:PLAIN_DICTIONARY,PLAIN,RLE
Price:           DOUBLE GZIP DO:1037572 FPO:1038252 SZ:2144124/12263463/5.72 VC:12564380 ENC:PLAIN_DICTIONARY,PLAIN,RLE
Quantity:        INT32 GZIP DO:3181790 FPO:3191180 SZ:11836165/18356783/1.55 VC:12564380 ENC:PLAIN_DICTIONARY,PLAIN,RLE
Exchange:        BINARY GZIP DO:15018038 FPO:15018157 SZ:3761153/6211788/1.65 VC:12564380 ENC:PLAIN_DICTIONARY,PLAIN,RLE
Conditions:      BINARY GZIP DO:18779265 FPO:18779435 SZ:181943/480623/2.64 VC:12564380 ENC:PLAIN_DICTIONARY,PLAIN,RLE
Date_Timestamp:  INT64 GZIP DO:18961287 FPO:19175098 SZ:3167908/88781159/28.03 VC:12564380 ENC:PLAIN_DICTIONARY,PLAIN,RLE
file:        file:/Users/feras/Desktop/SPY_dict_java.parquet
creator:     parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a)

file schema: m
--------------------------------------------------------------------------------
date:        REQUIRED INT64 O:TIMESTAMP_MILLIS R:0 D:0
type:        REQUIRED BINARY R:0 D:0
ticker:      REQUIRED BINARY R:0 D:0
price:       REQUIRED INT32 R:0 D:0
size:        REQUIRED INT32 R:0 D:0
exchange:    REQUIRED BINARY R:0 D:0
conditions:  REQUIRED BINARY R:0 D:0

row group 1: RC:2340100 TS:134648137 OFFSET:4
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:4 SZ:18722040/18722040/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
type:         BINARY UNCOMPRESSED DO:0 FPO:18722044 SZ:31300350/31300350/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
ticker:       BINARY UNCOMPRESSED DO:0 FPO:50022394 SZ:16381482/16381482/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
price:        INT32 UNCOMPRESSED DO:0 FPO:66403876 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
size:         INT32 UNCOMPRESSED DO:0 FPO:75764751 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
exchange:     BINARY UNCOMPRESSED DO:0 FPO:85125626 SZ:21439994/21439994/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
conditions:   BINARY UNCOMPRESSED DO:0 FPO:106565620 SZ:28082521/28082521/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED

row group 2: RC:2340100 TS:134572689 OFFSET:134648141
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:134648141 SZ:18722040/18722040/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
type:         BINARY UNCOMPRESSED DO:0 FPO:153370181 SZ:31248705/31248705/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
ticker:       BINARY UNCOMPRESSED DO:0 FPO:184618886 SZ:16381482/16381482/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
price:        INT32 UNCOMPRESSED DO:0 FPO:201000368 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
size:         INT32 UNCOMPRESSED DO:0 FPO:210361243 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
exchange:     BINARY UNCOMPRESSED DO:0 FPO:219722118 SZ:21416191/21416191/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
conditions:   BINARY UNCOMPRESSED DO:0 FPO:241138309 SZ:28082521/28082521/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED

row group 3: RC:2340100 TS:134566071 OFFSET:269220830
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:269220830 SZ:18722040/18722040/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
type:         BINARY UNCOMPRESSED DO:0 FPO:287942870 SZ:31298597/31298597/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
ticker:       BINARY UNCOMPRESSED DO:0 FPO:319241467 SZ:16381482/16381482/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
price:        INT32 UNCOMPRESSED DO:0 FPO:335622949 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
size:         INT32 UNCOMPRESSED DO:0 FPO:344983824 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
exchange:     BINARY UNCOMPRESSED DO:0 FPO:354344699 SZ:21359681/21359681/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
conditions:   BINARY UNCOMPRESSED DO:0 FPO:375704380 SZ:28082521/28082521/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED

row group 4: RC:2340100 TS:134625251 OFFSET:403786901
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:403786901 SZ:18722040/18722040/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
type:         BINARY UNCOMPRESSED DO:0 FPO:422508941 SZ:31331647/31331647/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
ticker:       BINARY UNCOMPRESSED DO:0 FPO:453840588 SZ:16381482/16381482/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
price:        INT32 UNCOMPRESSED DO:0 FPO:470222070 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
size:         INT32 UNCOMPRESSED DO:0 FPO:479582945 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
exchange:     BINARY UNCOMPRESSED DO:0 FPO:488943820 SZ:21385811/21385811/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
conditions:   BINARY UNCOMPRESSED DO:0 FPO:510329631 SZ:28082521/28082521/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED

row group 5: RC:2340100 TS:134609900 OFFSET:538412152
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:538412152 SZ:18722040/18722040/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
type:         BINARY UNCOMPRESSED DO:0 FPO:557134192 SZ:31303657/31303657/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
ticker:       BINARY UNCOMPRESSED DO:0 FPO:588437849 SZ:16381482/16381482/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
price:        INT32 UNCOMPRESSED DO:0 FPO:604819331 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
size:         INT32 UNCOMPRESSED DO:0 FPO:614180206 SZ:9360875/9360875/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
exchange:     BINARY UNCOMPRESSED DO:0 FPO:623541081 SZ:21398450/21398450/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED
conditions:   BINARY UNCOMPRESSED DO:0 FPO:644939531 SZ:28082521/28082521/1.00 VC:2340100 ENC:PLAIN,BIT_PACKED

row group 6: RC:863879 TS:49763046 OFFSET:673022052
--------------------------------------------------------------------------------
date:         INT64 UNCOMPRESSED DO:0 FPO:673022052 SZ:6911513/6911513/1.00 VC:863879 ENC:PLAIN,BIT_PACKED
type:         BINARY UNCOMPRESSED DO:0 FPO:679933565 SZ:11570893/11570893/1.00 VC:863879 ENC:PLAIN,BIT_PACKED
ticker:       BINARY UNCOMPRESSED DO:0 FPO:691504458 SZ:6047445/6047445/1.00 VC:863879 ENC:PLAIN,BIT_PACKED
price:        INT32 UNCOMPRESSED DO:0 FPO:697551903 SZ:3455726/3455726/1.00 VC:863879 ENC:PLAIN,BIT_PACKED
size:         INT32 UNCOMPRESSED DO:0 FPO:701007629 SZ:3455726/3455726/1.00 VC:863879 ENC:PLAIN,BIT_PACKED
exchange:     BINARY UNCOMPRESSED DO:0 FPO:704463355 SZ:7954707/7954707/1.00 VC:863879 ENC:PLAIN,BIT_PACKED
conditions:   BINARY UNCOMPRESSED DO:0 FPO:712418062 SZ:10367036/10367036/1.00 VC:863879 ENC:PLAIN,BIT_PACKED