Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Leak in Encoder / Encoder.parse #39

Open
Aea opened this issue Jan 15, 2019 · 0 comments
Open

Memory Leak in Encoder / Encoder.parse #39

Aea opened this issue Jan 15, 2019 · 0 comments

Comments

@Aea
Copy link

Aea commented Jan 15, 2019

I suspect this may be related to #32

Every instantiation of borc.Encoder whether explicit or through the use of the convenience function borc.decode leaks memory. The leak appears to be caused by the parse property (e.g. the ASM decoder object) holding onto memory.

Merely creating 50K Encoder objects in a loop consumes nearly 700MB.

node --expose-gc ...

'use strict';

const borc = require('borc')
const crypto = require('crypto')

const SAMPLES = 5e4;
const SAMPLE_RATE = 1e3;
const SEP = '\r';

const memory = (i, sep) => {
  i = i || 0;
  sep = sep || SEP;
  
  const padded = i.toString().padStart(9, ' ');
  const mem = process.memoryUsage().rss / 1024 ** 2;
  
  process.stdout.write(sep + `${i}\t\trss: ${mem.toFixed(1)}`);
};

// create decoder
console.info('\ncreate decode')
for(let i = 0; i <= SAMPLES; ++i) {
  new borc.Decoder();
  (i % SAMPLE_RATE == 0 || i == SAMPLES) && memory(i) && global.gc(); 
}

Output:

create decode
50000		rss: 695.0

node --expose-gc ...

'use strict';

const borc = require('borc')
const crypto = require('crypto')

const SAMPLES = 5e4;
const SAMPLE_RATE = 1e3;
const SEP = '\r';

const memory = (i, sep) => {
  i = i || 0;
  sep = sep || SEP;
  
  const padded = i.toString().padStart(9, ' ');
  const mem = process.memoryUsage().rss / 1024 ** 2;
  
  process.stdout.write(sep + `${i}\t\trss: ${mem.toFixed(1)}`);
};

// baseline
console.info('\nbaseline')
for(let i = 0; i <= SAMPLES; ++i) {
  crypto.randomBytes(2048).toString('base64');

  (i % SAMPLE_RATE == 0 || i == SAMPLES) && memory(i) && global.gc(); 
}

// encode 
console.info('\nencode')
for(let i = 0; i <= SAMPLES; ++i) {
  borc.encode(crypto.randomBytes(2048).toString('base64'));

  (i % SAMPLE_RATE == 0 || i == SAMPLES) && memory(i) && global.gc(); 
}

// external decode
console.info('\nexternal decode')
const decoder = new borc.Decoder();

for(let i = 0; i <= SAMPLES; ++i) {
  decoder.decodeAll(borc.encode(crypto.randomBytes(2048).toString('base64')));

  (i % SAMPLE_RATE == 0 || i == SAMPLES) && memory(i) && global.gc(); 
}

// helper decode
console.info('\nhelper decode')
for(let i = 0; i <= SAMPLES; ++i) {
  borc.decode(borc.encode(crypto.randomBytes(2048).toString('base64')));

  (i % SAMPLE_RATE == 0 || i == SAMPLES) && memory(i) && global.gc();
}

Output

baseline
50000		rss: 36.7
encode
50000		rss: 39.4
external decode
50000		rss: 45.3
helper decode
50000		rss: 718.5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant