git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[GitHub] mbeckerle commented on a change in pull request #77: Modifications to IO layer to support streaming input data


mbeckerle commented on a change in pull request #77: Modifications to IO layer to support streaming input data
URL: https://github.com/apache/incubator-daffodil/pull/77#discussion_r201046173
 
 

 ##########
 File path: daffodil-io/src/main/scala/org/apache/daffodil/io/InputSource.scala
 ##########
 @@ -0,0 +1,472 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.io
+
+import java.nio.ByteBuffer
+
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.daffodil.exceptions.Assert
+
+
+abstract class InputSource {
+
+  /**
+   * Determine whether the underlying data has the specified number of bytes
+   * available starting at the current byte position. This function must block
+   * until either nBytes are known to be available or end-of-file is reached.
+   * This does not advance the current position.
+   *
+   * @param nBytes the number of bytes to determine if are available
+   * @return true if nBytes are available, false otherwise
+   */
+  def areBytesAvailable(nBytes: Long): Boolean
+
+  /**
+   * Return the number of currently available bytes.
+   *
+   * This should not be used to determine the length of the data, as more bytes
+   * may become available in the future. This should really only be used for
+   * debug purposes.
+   */
+  def bytesAvailable(): Long
+
+  /**
+   * Return a single byte at the current byte position with a value in the
+   * range of 0 to 255.
+   *
+   * Increments the current byte position if successful.
+   *
+   * @return the byte at the current byte position if it exists, or -1 if EOF is reached.
+   */
+  def get(): Int
+
+  /**
+   * Return a byte array with data from the current byte position.
+   *
+   * Stores the next len bytes of data in dest starting at index off. In len
+   * bytes are not available or len bytes cannot fit in the dest array starting
+   * at the given offset, the dest array is not modified and false is returned.
+   *
+   * @return true if len bytes are available and written to the dest array,
+   *         false otherwise
+   */
+  def get(dest: Array[Byte], off: Int, len: Int): Boolean
+
+  /**
+   * Get the current byte position, using zero-based indexing
+   *
+   * @return the current byte position
+   */
+  def position(): Long
+
+  /**
+   * Set the current byte position, using zero-based indexing
+   *
+   * bytPos0b cannot be greater than the most recent read data. In other words,
+   * this can only be used to move backwards in data.
+   *
+   * @param bytePos0b the new current byte position
+   */
+  def position(bytePos0b: Long): Unit
+
+  /**
+   * Set the specified byte position as a location that that one may want to
+   * call setPosition in the future. This is essentially setting a mark in the
+   * data that can be reset back to later. Implementations are allowed to free
+   * any bytes before a locked byte position. It is implied tha the current
+   * byte position is locked, and thus any data that might exist after it
+   * cannot be freed.
+   *
+   * @param bytePos0b the byte position to lock
+   */
+  def lockPosition(bytePos0b: Long): Unit
+
+  /**
+   * Release a previously locked byte position, allowing the implementation to
+   * free any unlocked bytes.
+   *
+   * @param bytePos0b the byte position to release
+   */
+  def releasePosition(bytePos0b: Long): Unit
+
+  /**
+   * Alerts the implementation to attempt to free data that is no longer used,
+   * if possible. If possible, this should free any unlocked bytes.
+   */
+  def compact(): Unit
+
+  private var _debugging = false
+
+  final def areDebugging: Boolean = _debugging
+
+  final def setDebugging(setting: Boolean): Unit = _debugging = setting
+}
+
+/**
+ * Implements the InputSource interface, reading data from a
+ * generic java.io.InputStream and storing the data in buckets of a defined
+ * size. Buckets are freed when no locks exist inside the bucket to minimize
 
 Review comment:
   Mixed terminology "lock" vs. "reference count".  Add comment explaining this has nothing to do with synchronization locks. The term "lock" here just means we've incremented the ref-count so that the block will not be released until ref count goes to zero. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@xxxxxxxxxxxxxxxx


With regards,
Apache Git Services