git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Scanning the list of entries present on a bookie


IIRC, InterleavedLedgerStorage has for each ledger an index file
mapping the entries to entry logger offsets, you could probably scan
that directly (particularly if you included a lower bound -- probably
the client's current idea of the LAC).
-Sam

On Thu, Apr 12, 2018 at 12:31 AM, Enrico Olivelli <eolivelli@xxxxxxxxx> wrote:
> Hi BookKeepers,
> during implementation of BP-14 I am facing a problem so I am asking for
> suggestions.
>
> My need is to be able to know the list of all entries stored on a
> LedgerStorage given a ledgerId.
>
> Scanning from 0 to LedgerStorage#getLastAddConfirmed() does not seem to
> work because we have to deal with WriteAdvHandle, so there can be temporary
> "gaps" in the sequence of entries.
>
> I can have a writer which writes entries 0,1,5,6,7. Its LAC will be at most
> 1 as entries 2,3,4 are not written yet.
> I need on the bookie to able to know that entries 0, 1, 5, 6, 7 are stored
> on LedgerStorage.
>
> I cannot issue a scan from 0 to Long.MAX_VALUE, my current 'solution' it to
> make the client (writer) send the 'maximum entry id' and perform a scan
> from 0 to maxEntryId.
> In the example the writer will send a forceLedger RPC with maxEntryId = 7.
>
> This is need only for recoveries are bookie restart because I have to
> reconstruct the knowledge about which entries have been persisted durably
> on the Bookie.
>
> I am not very expert about LedgerStorage implementations, and I don't know
> if it would be feasible to have such 'scan all entries' method.
>
> This is the code I am talking about
> https://github.com/apache/bookkeeper/pull/1317/files#diff-3b81b1c90d1f51017627b3c032676168R1210
>
> Any help is really appreciated
> Enrico