SQL Filter Pushdowns in Apache Beam SQL
We are currently playing with Apache Beam’s SQL extension on top of Flink. One of the features that we were interested is the SQL Predicate Pushdown feature that Spark provides. Does Beam support that?
I have an unbounded dataset that I want to join with some static reference data stored in a database. Will beam perform the logic of figuring out all the unique keys in the window and push it down to the jdbc source or will it bring all the data from the jdbc source into memory and then perform the join?