git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Support Higher-order functions in Flink sql


Hi Wenhui,

Thanks for bringing the topics up. Both make sense to me. For higher-order functions, I'd suggest you come up with a list of things you'd like to add. Overall, Flink SQL is weak in handling complex types. Ideally we should have a doc covering the gaps and provide a roadmap for enhancement. It would be great if you can broaden the topic a bit.

Thanks,
Xuefu 


------------------------------------------------------------------
Sender:winifred.wenhui.tang@xxxxxxxxx <winifred.wenhui.tang@xxxxxxxxx>
Sent at:2018 Dec 3 (Mon) 16:13
Recipient:dev <dev@xxxxxxxxxxxxxxxx>
Subject:[DISCUSS] Support Higher-order functions in Flink sql

Hello all,

Spark 2.4.0 was released last month. I noticed that Spark 2.4 
“Add a lot of new built-in functions, including higher-order functions, to deal with complex data types easier.”[1]
I wonder if it's necessary for Flink to add higher-order functions to enhance it's ability.

By the way, I found that if we wants to enhance the functionality of Flink sql, we often need to modify Calcite. It may be a little inconvenient,so may be we can extend Calcite core parser in Flink to deal with some non-standard SQL syntax, as mentioned in Flink SQL DDL Design[2].

Look forward to your feedback.

Best,
Wen-hui Tang

[1] https://issues.apache.org/jira/browse/SPARK-23899
[2] https://docs.google.com/document/d/1TTP-GCC8wSsibJaSUyFZ_5NBAHYEB1FVmPpP7RgDGBA/edit#



Winifred-wenhui Tang