Updates on Calcite benchmark development
My colleagues and I are completing the implementation of the first
benchmark suite for Calcite, with the expectation to have something in
place in the next 2-4 weeks, and available for testing.
The plan is to make it available for your review and testing, and then
Current status is:
We have 80+ out of 99 TPC-DS queries made compatible to Calcite for
Postgres backend. (Calcite-178)
It will be compatible with most of the backends, but the queries will need
testing and validation.
We need to complete following two tasks to complete the initial benchmark
1) an accurate method to measure and instrument the Calcite performance
since the time measurement for Calcite may involve overhead associated with
2) fix queries that require syntax modification for Calcite syntax or to
account for some lack of support
We are not sure yet how bad the 2 part might be. It might involve some bug
patches or workarounds, which also might vary from backend to backend.
Big thanks to my R&D team members, Dr. Seung-Hwan Lim and Ashwin Vajantri
for working on this, and specially on tedious TPC-DS query syntax fixing to
make it work with Calcite.