git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[GitHub] RestfulBlue opened a new issue #5990: fromIndex(0) to toIndex(-1) exception in kafka indexing task


RestfulBlue opened a new issue #5990: fromIndex(0) to toIndex(-1) exception in kafka indexing task
URL: https://github.com/apache/incubator-druid/issues/5990
 
 
   Currently in our dev environment kafka indexing task throw exception like this:
   ```
   java.lang.IllegalArgumentException: fromIndex(0) > toIndex(-1)
   	at java.util.ArrayList.subListRangeCheck(ArrayList.java:1006) ~[?:1.8.0_121]
   	at java.util.ArrayList.subList(ArrayList.java:996) ~[?:1.8.0_121]
   	at io.druid.segment.realtime.appenderator.AppenderatorImpl.persistAll(AppenderatorImpl.java:408) ~[druid-server-0.12.1.jar:0.12.1]
   	at io.druid.segment.realtime.appenderator.AppenderatorImpl.push(AppenderatorImpl.java:519) ~[druid-server-0.12.1.jar:0.12.1]
   	at io.druid.segment.realtime.appenderator.BaseAppenderatorDriver.pushInBackground(BaseAppenderatorDriver.java:351) ~[druid-server-0.12.1.jar:0.12.1]
   	at io.druid.segment.realtime.appenderator.StreamAppenderatorDriver.publish(StreamAppenderatorDriver.java:268) ~[druid-server-0.12.1.jar:0.12.1]
   	at io.druid.indexing.kafka.KafkaIndexTask.lambda$createAndStartPublishExecutor$1(KafkaIndexTask.java:364) ~[?:?]
   	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_121]
   	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
   	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
   ```
   If we look into this exception, the source is [AppenderatorImpl ](https://github.com/apache/incubator-druid/blob/master/server/src/main/java/io/druid/segment/realtime/appenderator/AppenderatorImpl.java) , function persistAll . 
   
   ```
         final int limit = sink.isWritable() ? hydrants.size() - 1 : hydrants.size();
   
         for (FireHydrant hydrant : hydrants.subList(0, limit)) {
           if (!hydrant.hasSwapped()) {
             log.info("Hydrant[%s] hasn't persisted yet, persisting. Segment[%s]", hydrant, identifier);
             indexesToPersist.add(Pair.of(hydrant, identifier));
           }
         }
   ```
   here subList throw this exception. As i understand, when sink is writable and hydrants.size == 0 ( for some reason) -> this exception is thrown. Will adding check to zero fix this problem or it will broke something?=)
   
   for example 
   ```
    final int limit = sink.isWritable() && hydrants.size() > 0 ? hydrants.size() - 1 : hydrants.size();
   ```
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@xxxxxxxxxxxxxxxx


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxx