git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Fineract CN Server - Insufficient memory - Java Run time error


Hey Victor,

I did check for different metrics to analyse server performance in
cloudwatch before trying out swap space. Nginx, MySQL, Cassandra and
Fineract Application are the only resource consuming applications. Each
microservice takes around 2GB of memory when starting.. so, 12 micro
services taking up around 25GB of Memory space leaving very limited
resource for MySQL, Cassandra, Nginx to run. For now i am not considering
elastic scalability. This is the reason, i commented out few microservices
and starting the server to optimise memory consumption.

Regards,
Karthik

On Mon, Oct 1, 2018 at 1:42 AM Victor Manuel Romero Rodriguez <
vic_romero@xxxxxxxxxxx> wrote:

> Hi Karthik,
>
> You have faced the error that all might face when running the services.
>
> I can view in the image log that you are using a EC2 machine, you can
> check the behaviour of the server and make the proper changes using
> CloudWatch.
>
> https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring_ec2.html
>
> Changes are not only at JVM, but also at OS level and of course the
> Elastic Cloud could help to jump from one machine size to another in a
> "smooth" way (always there is pain).
>
> Even for testing/demo you should check what is the resource consumption
> for suggesting or proposing a better sizing of the machine based on the
> services that will be enabled.
>
> Regards
>
> Victor
>
>
> El 30/09/18 a las 14:06, Karthik Ananthakrishnan escribió:
> > Hello Isaac and others,
> >
> > Many thanks for the suggestions. I tried with xms4G and was not very
> > helpful. When i looked at the root cause, I found that cassandra trips
> > during the server boot process when the memory hits peak(free memory less
> > than 100MB) in 32GB RAM server. So, i added swap space to it which
> helped.
> > But still the cassandra would go down when memory hits peak. So, i
> decided
> > to run the server with fewer microservices by commenting out a few. Once
> i
> > did that server started properly and has been stable for around a week.
> > Even in local machine, i could run the application by running fewer
> > microservices.
> >
> > Thanks,
> > Karthik
> >
> > On Tue, Sep 18, 2018 at 10:31 PM Karthik Ananthakrishnan <
> > karthik.a@xxxxxxxxxxxxxxxx> wrote:
> >
> >> Hey Issac. I did try the 2G and didnt really help. So, i have already
> made
> >> changes as xms2G and Xmx4G. Will get back to you if it worked.
> >>
> >> On Tue, Sep 18, 2018 at 7:20 PM, Isaac Kamga <isaac.kamga@xxxxxxxxx>
> >> wrote:
> >>
> >>> Thanks for helping out with the image Ed.
> >>>
> >>> The image never showed up for me...probably a pesky internet
> connection.
> >>>
> >>> @Karthik Ananthakrishnan <karthyk.a@xxxxxxxxx> , I think my
> >>> aforementioned
> >>> suggestion should work. If it doesn't, then use `* -Xms4G -Xmx4G* `
> >>> instead. Also, provide feedback on whether or not it helped and update
> >>> documentation appropriately.
> >>>
> >>> Cheers,
> >>> Isaac Kamga
> >>>
> >>> On Tue, Sep 18, 2018 at 2:43 PM Ed Cable <edcable@xxxxxxxxx> wrote:
> >>>
> >>>> The image showed up for me. I've added it to imgur here:
> >>>>
> >>>> https://imgur.com/a/Qt145wA
> >>>>
> >>>> Ed
> >>>>
> >>>> On Tue, Sep 18, 2018 at 3:50 AM Courage Angeh <courageangeh@xxxxxxxxx
> >
> >>>> wrote:
> >>>>
> >>>>> Hi Karthik,
> >>>>>
> >>>>> You still haven't attached the image. You could try using
> >>>>> https://pastebin.com/ instead.
> >>>>>
> >>>>> On Mon, Sep 17, 2018 at 11:54 AM Karthik Ananthakrishnan
> >>>>> <karthik.a@xxxxxxxxxxxxxxxx.invalid> wrote:
> >>>>>
> >>>>>> Thanks Issac. Attached the image as well as the err log for your
> >>>>>> reference. Let me try using the xms  and xmx options. Are you using
> >>>> those
> >>>>>> when you run the server.?
> >>>>>>
> >>>>>>
> >>>>
> >>>> --
> >>>> *Ed Cable*
> >>>> President/CEO, Mifos Initiative
> >>>> edcable@xxxxxxxxx | Skype: edcable | Mobile: +1.484.477.8649
> >>>>
> >>>> *Collectively Creating a World of 3 Billion Maries | *
> http://mifos.org
> >>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
> >>>>
> >>
>
>

-- 
Karthik Krishnan