Skip to content

Commit

Permalink
Remove check for Nulls buffer when null_count = 0
Browse files Browse the repository at this point in the history
Summary:
In PyArrow 6.0, it's legit for an arrow array to have non-null NullBuffer while null_count is zero (see also pytorch/torcharrow#109) :
```
import pyarrow as pa
>>> pa.__version__
'6.0.0'
>>> a = pa.array([1, 2, None, 3])
>>> a = a.fill_null(12)
>>> a.buffers()
[<pyarrow.lib.Buffer object at 0x7fe688222330>, <pyarrow.lib.Buffer object at 0x7fe6680c2ef0>]
```

User also reports similiar issue when converting arrow array reading from Parquet, see pytorch/torcharrow#146 (comment)

Differential Revision: D33836988

fbshipit-source-id: bdb7277671eaf4fc25dfad46ed5bbe3272569ace
  • Loading branch information
wenleix authored and facebook-github-bot committed Jan 28, 2022
1 parent 37a3e4a commit f978977
Showing 1 changed file with 0 additions and 4 deletions.
4 changes: 0 additions & 4 deletions velox/vector/arrow/Bridge.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -676,10 +676,6 @@ VectorPtr importFromArrowImpl(
"Nulls buffer can't be null unless null_count is zero.");
nulls = wrapInBufferView(
arrowArray.buffers[0], bits::nbytes(arrowArray.length));
} else {
VELOX_USER_CHECK_NULL(
arrowArray.buffers[0],
"Nulls buffer must be nullptr when null_count is zero.");
}

// String data types (VARCHAR and VARBINARY).
Expand Down

0 comments on commit f978977

Please sign in to comment.