git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Approximate query processing in Calcite


However, we are interested in exploring ways that a deeper integration
might be beneficial especially in the case of federated query processing
across multiple database backends.


Can you elaborate a bit more on this? I can understand that there are common functionalities can be shared and make sense to share (parsing, relational algebras, etc). But I am more curious about what new scenarios and/or performance benefits can be achieved through the integration.

I personally work on the infrastructure side, and am not the direct user of data system. But some of our customers do express interests in approximate query support. I’d like to talk to them and do some more survey. I would be happy to share with you my findings after that. Thanks.


On May 7, 2018, at 11:15 AM, Michael Mior <mmior@xxxxxxxxxxxx> wrote:

Xiening,

You are correct that VerdictDB is currently completely DBMS independent.
However, we are interested in exploring ways that a deeper integration
might be beneficial especially in the case of federated query processing
across multiple database backends. The first step would be to simply allow
Calcite and VerdictDB to work together. That is potentially highly useful
in itself since then it should be possible to perform AQP over any
Calcite-supported backend.

If you'd be willing to discuss potential use cases further, I'd love to try
to schedule a call with you.

--
Michael Mior
mmior@xxxxxxxxxxxx


Le lun. 7 mai 2018 à 13:30, Xiening Dai <xndai.git@xxxxxxxx> a écrit :

Hi Michael,

AQP is valuable in our business scenarios. Often time our data scientist
would issue exploratory queries to get a basic sense of the data (means,
aggregation on certain groupings, etc). But from my understanding, Verdict
DB is independent to the database system and the query planner/optimizer. I
wonder what you want to achieve specifically through Calcite integration?


On May 7, 2018, at 10:05 AM, Michael Mior <mmior@xxxxxxxxxxxx> wrote:

Edmon (and others),

I'd be curious to hear more about your specific use cases if you're able
to
share. Especially those who have companies which may benefit from using
AQP
with Calcite to lower costs.

--
Michael Mior
mmior@xxxxxxxxxxxx


Le jeu. 3 mai 2018 à 18:58, Edmon Begoli <ebegoli@xxxxxxxxx> a écrit :

I am excited that you are considering taking Calcite in this direction.

Approximate querying and probabilistic databases are of great interest
to
me, and I might be able to provide some applied research scenarios.

One domain that comes to mind where we had some use cases is a sensor
data
analysis.

Thank you,
Edmon

On Thu, May 3, 2018 at 6:54 PM, Michael Mior <mmior@xxxxxxxxxxxx>
wrote:

Hi all,

I recently had a chat with the VerdictDB (http://verdictdb.org/) team
about
possible integration with Calcite. VerdictDB sits between an
application
and a database to enable the approximation of query results which are
expected to be highly accurate while consuming significantly fewer
resources on the backend.

I'm curious to talk to anyone who might have a use case for this.
Particularly those using Calcite to power analytics systems that can
tolerate approximate results. We'll likely be looking at putting
together a
proof of concept in the next few weeks if there's any interest. Let me
know!

--
Michael Mior
mmior@xxxxxxxxxxxx







( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-calcite-development/msg03434.html on line 167
Call Stack
#TimeMemoryFunctionLocation
10.0007368760{main}( ).../msg03434.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-calcite-development/msg03434.html on line 167
Call Stack
#TimeMemoryFunctionLocation
10.0007368760{main}( ).../msg03434.html:0