Re: [druid-dev] Load rule doesn't honor intervals properly
That sounds like a bug to me - a patch would be welcome!
Btw, since we are trying to migrate the dev mailing list to Apache, please
cross post this sort of thing with dev@xxxxxxxxxxxxxxxxxxxxxxxxxx, or even
only post to that list.
On Thu, Mar 29, 2018 at 5:43 PM, 'Pala Muthiah' via Druid Development <
> Hello folks,
> Anybody have insight on the below? Curious to know if there would be
> unforeseen side effects if we count even partial overlap as valid.
> On Mon, Mar 19, 2018 at 10:54 AM, pala.muthiah via Druid Development <
> druid-development@xxxxxxxxxxxxxxxx> wrote:
>> In our deployment, we enabled background segment merging and found that
>> some of the data within the load period was actually getting dropped.
>> My suspicion was that when a merged segment only partially overlaps with
>> a period (e.g: Rule says keep data from Jan 1st onwards, and i have a
>> segment that spans Dec 25th - Jan 2nd), for correctness that segment should
>> be kept but current implementation seems to drop it.
>> I checked the code and found indeed Rules.eligibleForLoad() only keeps
>> segments that overlap fully.
>> Is this a bug, or is there other reason behind this? In our case, we do
>> have data sources that are highly aggregated and therefore a single segment
>> could span a month for example.
>> I can submit a patch but wanted to get proper context.
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Druid Development" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> To unsubscribe from this group and all its topics, send an email to
>> To post to this group, send email to druid-development@xxxxxxxxxxxxxxxx.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> For more options, visit https://groups.google.com/d/optout.
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscribe@xxxxxxxxxxxxxxxx.
> To post to this group, send email to druid-development@xxxxxxxxxxxxxxxx.
> To view this discussion on the web visit https://groups.google.com/d/ms
> For more options, visit https://groups.google.com/d/optout.