git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: plasma tutorial which version?


I managed to make it work with the example listed below. I had to comment some includes unused to which I failed to link at compile time. I would appreciate some hints on how to create/compile/build clients better with arrow/plasma.

Thanks,
Ovidiu


+++++ create.cc  ++++++
Compiled with: g++ create.cc `pkg-config --cflags --libs plasma arrow` --std=c++11 -o create

#include <assert.h>
#include <signal.h>
#include <stdlib.h>
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>

#include <random>

//#include "arrow/test-util.h"

#include "plasma/client.h"
//#include "plasma/common.h"
//#include "plasma/plasma.h"
//#include "plasma/protocol.h"


using namespace plasma;

int main(int argc, char** argv) {
  // Start up and connect a Plasma client.
  PlasmaClient client;
  ARROW_CHECK_OK(client.Connect("/tmp/plasma", "", PLASMA_DEFAULT_RELEASE_DELAY));

   ObjectID object_id = ObjectID::from_random();
   std::vector<uint8_t> data(100, 0);
  uint8_t metadata[] = {5};
  int64_t metadata_size = sizeof(metadata);
  bool has_object;

    std::shared_ptr<Buffer> data_buffer;
    ARROW_CHECK_OK(client.Create(object_id, data.size(), &metadata[0], metadata_size,
                                 &data_buffer));
    for (size_t i = 0; i < data.size(); i++) {
      data_buffer->mutable_data()[i] = data[i] + 1;
    }


  // Seal the object.
  ARROW_CHECK_OK(client.Seal(object_id));

 std::vector<ObjectID> object_ids = {object_id};
 std::vector<ObjectBuffer> object_buffers;
 ARROW_CHECK_OK(client.Get(object_ids, -1, &object_buffers));

 fprintf(stdout, "%d\n", object_buffers[0].data->data()[0]);

 ARROW_CHECK_OK(client.Contains(object_id, &has_object));
 fprintf(stdout, "contains object:%d\n", has_object);
  // Disconnect the client.
  ARROW_CHECK_OK(client.Disconnect());
}


> On 02 May 2018, at 14:20, Ovidiu-Cristian MARCU <ovidiu-cristian.marcu@xxxxxxxx> wrote:
> 
> It seems the client Create signature changed (see below tutorial error).  However, I’ve looked into client_tests and I see it passes ok all 9 tests. So I’m trying to modify the example to create an object. I still have some issues compiling, see below, maybe you can provide some help as I just started looking into apache arrow/plasma.
> 
> I’m using create.cc <http://create.cc/>, however trying to compile fails with:
> g++ create.cc `pkg-config --cflags --libs plasma arrow` --std=c++11 -o create
> In file included from /usr/local/include/plasma/plasma.h:40:0,
>                 from create.cc:15:
> /usr/local/include/plasma/common_generated.h:7:37: fatal error: flatbuffers/flatbuffers.h: No such file or directory
> #include "flatbuffers/flatbuffers.h”
> 
> Before, I managed to make && make install after 
> cmake .. -DCMAKE_BUILD_TYPE=Release -DARROW_PLASMA=on
> and run
> /release/plasma_store -m 1000000000 -s /tmp/plasma
> Allowing the Plasma store to use up to 1GB of memory.
> Starting object store with directory /dev/shm and huge page support disabled
> 
> ++++ create object example modified ++++
> create.cc
> #include <assert.h>
> #include <signal.h>
> #include <stdlib.h>
> #include <sys/time.h>
> #include <sys/types.h>
> #include <unistd.h>
> 
> #include <random>
> 
> #include "plasma/client.h"
> #include "plasma/common.h"
> #include "plasma/plasma.h"
> #include "plasma/protocol.h"
> 
> using namespace plasma;
> 
> int main(int argc, char** argv) {
>  // Start up and connect a Plasma client.
>  PlasmaClient client;
>  ARROW_CHECK_OK(client.Connect("/tmp/plasma", "", PLASMA_DEFAULT_RELEASE_DELAY));
> 
>   ObjectID object_id = ObjectID::from_random();
>   std::vector<uint8_t> data(100, 0);
>  uint8_t metadata[] = {1};
>  int64_t metadata_size = sizeof(metadata);
> 
>    std::shared_ptr<Buffer> data_buffer;
>    ARROW_CHECK_OK(client.Create(object_id, data.size(), &metadata[0], metadata_size,
>                                 &data_buffer));
>    for (size_t i = 0; i < data.size(); i++) {
>      data_buffer->mutable_data()[i] = data[i];
>    }
> 
>  // Seal the object.
>  ARROW_CHECK_OK(client.Seal(object_id));
>  // Disconnect the client.
>  ARROW_CHECK_OK(client.Disconnect());
> }
> 
> +++++++++ tutorial error +++++++
> with https://arrow.apache.org/docs/cpp/md_tutorials_plasma.html <https://arrow.apache.org/docs/cpp/md_tutorials_plasma.html> create.cc
> g++ create2.cc `pkg-config --cflags --libs plasma` --std=c++11 -o create
> In file included from /usr/local/include/plasma/common.h:32:0,
>                 from /usr/local/include/plasma/client.h:30,
>                 from create2.cc:1:
> create2.cc: In function ‘int main(int, char**)’:
> create2.cc:12:104: error: no matching function for call to ‘plasma::PlasmaClient::Create(plasma::ObjectID&, int64_t&, uint8_t*, std::basic_string<char>::size_type, uint8_t**)’
>   ARROW_CHECK_OK(client.Create(object_id, data_size, (uint8_t*) metadata.data(), metadata.size(), &data));
>                                                                                                        ^
> create2.cc:12:104: note: candidate is:
> In file included from create2.cc:1:0:
> /usr/local/include/plasma/client.h:92:10: note: arrow::Status plasma::PlasmaClient::Create(const ObjectID&, int64_t, const uint8_t*, int64_t, std::shared_ptr<arrow::Buffer>*, int)
>   Status Create(const ObjectID& object_id, int64_t data_size, const uint8_t* metadata,
>          ^
> /usr/local/include/plasma/client.h:92:10: note:   no known conversion for argument 5 from ‘uint8_t** {aka unsigned char**}’ to ‘std::shared_ptr<arrow::Buffer>*’
> 
> Thanks,
> Ovidiu
> 
>> On 02 May 2018, at 14:08, Uwe L. Korn <uwelk@xxxxxxxxxx> wrote:
>> 
>> Hello Ovidiu,
>> 
>> we actually would like to keep the tutorial in line with the API we have in master. Can you post the errors you're getting?
>> 
>> This would help us in understanding what is outdated in the steps.
>> 
>> Uwe
>> 
>> On Wed, May 2, 2018, at 11:53 AM, Ovidiu-Cristian MARCU wrote:
>>> Hi,
>>> 
>>> Trying to follow the steps in 
>>> https://arrow.apache.org/docs/cpp/md_tutorials_plasma.html 
>>> <https://arrow.apache.org/docs/cpp/md_tutorials_plasma.html> with master 
>>> branch does not work.
>>> Could you please provide the arrow version that is compatible with this 
>>> tutorial?
>>> 
>>> Thanks,
>>> Ovidiu
>