[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Format] Pointer types / span types

Is there a way to model the span type in a normal Arrow schema? I
would need to look a bit more closely to understand it better

On Mon, Apr 30, 2018 at 5:42 PM, Antoine Pitrou <antoine@xxxxxxxxxx> wrote:
> Le 30/04/2018 à 23:39, Brian Hulette a écrit :
>> Yes my first reaction to both of these requests is
>> - would dictionary-encoding work?
>> - would a List<T> work?
>> I think for the former the analogy is more clear, for the latter,
>> technically a List encodes start and stop indices with an offset array
>> rather than separate arrays for start and stop indices. Is there a
>> reason an offset array wouldn't work for the OAMap use-case though?
> With an offsets array, spans (lists) are contiguous: span N + 1 starts
> off where span N stops.  With separate start/stops array, they needn't
> be: the logical array can "walk" the physical array in any order.
> Regards
> Antoine.