[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Format] Pointer types / span types

If this were accomplished at the application level, how would it work with the IPC formats? I'd think you'd need to have two separate files (or streams), since array 1 and array 2 will be different lengths. Perhaps that could be an argument for making span a core logical type?


On 05/02/2018 03:34 PM, Antoine Pitrou wrote:
On Wed, 2 May 2018 10:12:37 -0400
Wes McKinney <wesmckinn@xxxxxxxxx> wrote:
It sounds like the "span" type could be implemented as a composite of
multiple Arrow arrays / schemas:

array 1 (data)
any schema

array 2 (view)
struct <
   start: int64,
   stop: int64
Unless I'm missing something, this feels like an application-level
concern rather than something that needs to be addressed in the
columnar format / metadata.
Well, couldn't the same theoretically be said about list arrays?
In the end, I suppose it all depends whether there's enough demand to
make it a core logical type inside Arrow, rather than something people
write custom code for in their application.