Organize unit tests and add integration tests with hsd

[pinheadmz]

Closes #23

This cleans up the framework for unit tests (written C and compiled into executable test_hnsd) and also introduces a new integration test system using nodejs and hsd.

--> ./test_hnsd
Testing hnsd...
test_base32
 test_base32_encode_decode
test_resource
 test_resource_pointer_to_ip
ok
--> cd integration/
--> npm run test

> [email protected] test
> bmocha --reporter spec test/*.js


  Basic sync & resolve
    ✓ should register names and sync (845ms)
    ✓ should resolve DS
    ✓ should resolve NS
    ✓ should resolve TXT
    ✓ should resolve GLUE4 without glue
    ✓ should resolve GLUE4 with glue
    ✓ should resolve GLUE6 without glue
    ✓ should resolve GLUE6 with glue
    ✓ should resolve SYNTH4 with glue
    ✓ should resolve SYNTH6 with glue

  10 passing (953ms)

[Falci]

Suggested change

	waitForSync() {
	return new Promise((resolve, reject) => {
	// Hack
	setTimeout(() => {
	resolve();
	}, 5000);
	// // TODO: Fix hnsd stdout parsing for chain height
	// setTimeout(() => {
	// reject(new Error('Timeout waiting for sync'));
	// }, 5000);
	// setInterval(() => {
	// if (this.hnsdHeight === this.node.chain.height)
	// resolve();
	// }, 100);
	});
	}
	async waitForSync() {
	if (!this.hnsd || !this.hnsd.killed) {
	return new Promise(resolve => setTimeout(resolve, 100))
	.then(() => this.waitForSync);
	}
	}

[Falci]

Suggested change

	this.hnsd.stdout.on('data', (data) => {
	// TODO: `data` is always 8192 bytes and output gets cut off, why?
	const chunk = data.toString('ascii');
	const lines = chunk.split(/\n/);
	for (const line of lines) {
	const words = line.split(/\s+/);
	if (words[0] !== 'chain' || words.length < 2)
	continue;
	this.hnsdHeight = parseInt(words[1].slice(1, -2));
	}
	});
	this.hnsd.stdout.on('data', (data) => {
	const chunk = data.toString('ascii');
	// check for "chain (xxx)"
	const height = chunk.match(/chain \((?<height>\d+)\).*(?![\s\S]*chain)/)?.groups?.height;
	if (height)
	this.hnsdHeight = parseInt(height);
	if (timer)
	clearTimeout(timer);
	if (!this.hnsd.killed)
	timer = setTimeout(() => {
	this.hnsd.kill();
	}, 200);
	});

[pinheadmz]

Thanks, this is a great-looking regex and cleans up the code. However, we (@nodech and myself) are finding that monitoring hnsd's stdout is not reliable. The waitForSync() mechanism will be improved in #102 where we can actually use DNS requests to ask hnsd for its chain height.

[pinheadmz]

Update: added command make e2e in makefile so integration tests can be launched easily from root directory

[rithvikvibhu]

Very cool CI with 3 platforms! Both hnsd and integration tests work great locally.
I did forget to build for regtest (like the readme mentions) and the tests just timed out with no specific errors.

Can hnsd print the network name on start? It would help identify binaries and can also make the test suite confirm that it's build with the correct network. Just an idea, not blocking this PR.

[rithvikvibhu]

I know bns is installed because of hsd, but since the test script explicitly uses bns, should be be included here?

[pinheadmz]

Can hnsd print the network name on start? It would help identify binaries and can also make the test suite confirm that it's build with the correct network. Just an idea, not blocking this PR.

Yeah re-building for each network is annoying, maybe we can work that out in the future and start hnsd with a network argument. I think we need to avoid using hnsd's stdout for anything important, and using HS class won't really work either since the "API" ports will be different for each network.

I think the fastest solution to this may be a version command or something that could work like this:

$ hnsd --version
1.0.0 (regtest)

[pinheadmz]

@rithvikvibhu added in 1715b09

--> make e2e
npm --prefix ./integration/ install

up to date, audited 37 packages in 332ms

found 0 vulnerabilities
npm --prefix ./integration/ run test

> [email protected] test
> bmocha --reporter spec test/*.js


  Basic sync & resolve
    1) "before all" hook

  0 passing (89ms)
  1 failing

  1) Basic sync & resolve
       "before all" hook:

      AssertionError [ERR_ASSERTION]: Network or version mismatch

      Expected values to be strictly equal:
      
      + actual - expected
      
      + '1.0.0 (main)\n'
      - '1.0.0 (regtest)\n'
                ^
      
      assert.strictEqual(
             ^

      at TestUtil.open (/Users/matthewzipkin/Desktop/work/hnsd/integration/test-util.js:36:12)
      at async Context.<anonymous> (/Users/matthewzipkin/Desktop/work/hnsd/integration/test/sync-resolve-test.js:18:5)

make: *** [e2e] Error 1

[rithvikvibhu]

Sweet, version shows network and test fails. LGTM