git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (ARROW-3850) Support MapType and StructType for PySpark integration


Florian Wilhelm created ARROW-3850:
--------------------------------------

             Summary: Support MapType and StructType for PySpark integration
                 Key: ARROW-3850
                 URL: https://issues.apache.org/jira/browse/ARROW-3850
             Project: Apache Arrow
          Issue Type: New Feature
    Affects Versions: 0.11.1
            Reporter: Florian Wilhelm


It would be great to support MapType and StructType in Arrow so that PySpark can make use of it.

 
Quite often as in my use-case in Hive table cells are also complex types saved. Currently it's not possible to user the new {{pandas_udf}} decorator which internally uses Arrow to generate a UDF an columns with complex types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)