git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table


Unfortunately I know very little about ES so I'm not in a great position to
asses the impact of these changes. I will say that that legacy
compatibility is great, but maintaining two sets of logic is always a
challenge. A few follow up questions:

1) What's the time horizon for the current adapter no longer working with
these changes to ES?

2) Any guess how complicated it would be to maintain code paths for both
behaviours? I know this is probably really challenging to estimate, but I
really have no idea of the scope of these changes. Would it mean two
different ES adapters?

3) Do we really need compatibility with the current version of the adapter?
IMO this depends on what versions of ES we would lose support for and how
complex it would be for users of the current ES adapter to make updates for
any Calcite API changes.

Thanks for your continued work on the ES adapter Andrei!

--
Michael Mior
mmior@xxxxxxxxxx



Le jeu. 28 juin 2018 à 12:57, Andrei Sereda <andrei@xxxxxxxxx> a écrit :

> Hello,
>
> Elastic announced
> <
> https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html
> >
> that they will be deprecating mapping types in ES6 and indexes will be
> single-typed only.
>
> Historical analogy <https://www.elastic.co/blog/index-vs-type> between
> RDBMS and elastic was that index is equivalent to a database and type
> corresponds to table in that database. In a couple of releases (ES6-8) this
> shall not longer be true.
>
> Recent SQL addition
> <https://www.elastic.co/blog/elasticsearch-6-3-0-released> to elastic
> confirms
> this trend
> <
> https://www.elastic.co/guide/en/elasticsearch/reference/current/_mapping_concepts_across_sql_and_elasticsearch.html
> >.
> Index is equivalent to a table and there are no more ES types.
>
> I would like to propose to include this logic in Calcite ES adapter. IE,
> expose each ES single-typed index as a separate table inside calcite
> schema. This is in contrast to  current integration where schema can only
> have a single index. Current approach forces you to create multiple schemas
> to query single-typed indexes (on the same ES cluster).
>
> Legacy compatibility can always be controlled with configuration
> parameters.
>
> Do you agree with such changes ? If yes, would you consider a PR ?
>
> Regards,
> Andrei.
>


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-calcite-development/msg03908.html on line 133
Call Stack
#TimeMemoryFunctionLocation
10.0022368824{main}( ).../msg03908.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-calcite-development/msg03908.html on line 133
Call Stack
#TimeMemoryFunctionLocation
10.0022368824{main}( ).../msg03908.html:0