[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Go] High memory usage on CSV read into table

Wasn't sure where the best place to discuss this, but I've noticed that when running the following piece of code

On a CSV files that contains roughly 1 million records (about 100mb of data), the memory usage of the process leaps to about 9.1GB

The records look something like this


I've attached a pprof output of the process.

From the looks of it the heavy use of _strings_ might be where most of the memory is going.

Is this expected? I'm new to the code, happy to help where possible!

Attachment: pprof001.svg
Description: image/svg