git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

withPartitioner() vs calling partitionCustom() beforehand


Hi all,

I have a question regarding partitioning.

Does calling the withPartitioner() method on a coGroup operation has the same effect as performing partitionCustom on both datasets beforehand?
i.e.

Is

  1.  a.coGroup(b).where(…).equalTo(…).withPartitioner(…).with(…)

equivalent to:


  1.  DataSet a = aa.partitionCustom(…)
  2.  DataSet b = bb.partitionCustom(…)
  3.  a.coGroup(b).where(…).equalTo(…).with(…)


Do both snippets perform the same low-level physical partitioning?

Thank you,
Giannis