_metadata.json isnt being created

[dmlaj68]

Hey
I'm having an issue whereby the image and individual json files are being created however at the end of the generation it isnt creating the _metadata.json file

I've read another thread that suggested to use their version of update_info.js but that didn't work
It could be that my collection is large and its a size error as I generated 500,000 but im new to doing this so I thought I'd ask the experts

when I've ran a smaller test collection it has worked fine.

the error is as folllows using the original update_info.js;

Created edition: 171239, with DNA: b64d0c4b1d6cc18ca1c7cabf09a4851046f83221
C:\SBS\src\main.js:429
writeMetaData(JSON.stringify(metadataList, null, 2));
^

RangeError: Invalid string length
at JSON.stringify ()
at startCreating (C:\SBS\src\main.js:429:22)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

The error when using the update_info.js from @bolshoytoster gave this error;

C:\SBS\utils\update_info.js:55
JSON.stringify(data, null, 2)
^

RangeError: Invalid string length
at JSON.stringify ()
at Object. (C:\SBS\utils\update_info.js:55:8)
at Module._compile (node:internal/modules/cjs/loader:1369:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
at Module.load (node:internal/modules/cjs/loader:1206:32)
at Module._load (node:internal/modules/cjs/loader:1022:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12)
at node:internal/main/run_main_module:28:49

any help would be greatly appreciated

[bolshoytoster]

@dmlaj68 the RangeError: Invalid string length message isn't very accurate, it means that you're running out of memory creating the _metadata.json file. Make sure you have more memory when you create your collection. If you don't want to have to regenerate, you could probably create _metadata.json from the existing metadata files.

[dmlaj68]

@dmlaj68 the RangeError: Invalid string length message isn't very accurate, it means that you're running out of memory creating the _metadata.json file. Make sure you have more memory when you create your collection. If you don't want to have to regenerate, you could probably create _metadata.json from the other metadata files.

how would i go about doing that?
thanks for your reply

[dmlaj68]

on this particular run the machine has 16gb
I also ran a much slower generation on a VPS which had 120gb Ram lol and that threw the same error

[dmlaj68]

@dmlaj68 the RangeError: Invalid string length message isn't very accurate, it means that you're running out of memory creating the _metadata.json file. Make sure you have more memory when you create your collection. If you don't want to have to regenerate, you could probably create _metadata.json from the existing metadata files.

Is there any other solution you can think of
When running the operation the memory doesn't go above 72% usage and as above the VPS I tried on has 120gb ram so that definitely shouldn't run out

[bolshoytoster]

@dmlaj68 maybe your collection is just too big for node to stringify at once. You could try using my fork, which creates the json as a stream. (my fork is also much faster).

[dmlaj68]

@dmlaj68 maybe your collection is just too big for node to stringify at once. You could try using my fork, which creates the json as a stream. (my fork is also much faster).

just tried your suggestion but im getting an error when running node index.js to run the generation

PS C:\hashlips_art_engine-main> node index.js
node:internal/modules/cjs/loader:1146
throw err;
^

Error: Cannot find module 'JSONStream'
Require stack:

• C:\hashlips_art_engine-main\src\main.js
• C:\hashlips_art_engine-main\index.js
  at Module._resolveFilename (node:internal/modules/cjs/loader:1143:15)
  at Module._load (node:internal/modules/cjs/loader:984:27)
  at Module.require (node:internal/modules/cjs/loader:1231:19)
  at require (node:internal/modules/helpers:179:18)
  at Object. (C:\hashlips_art_engine-main\src\main.js:6:20)
  at Module._compile (node:internal/modules/cjs/loader:1369:14)
  at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
  at Module.load (node:internal/modules/cjs/loader:1206:32)
  at Module._load (node:internal/modules/cjs/loader:1022:12)
  at Module.require (node:internal/modules/cjs/loader:1231:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
  'C:\hashlips_art_engine-main\src\main.js',
  'C:\hashlips_art_engine-main\index.js'

its setup the same as with the OG files so not sure what the issue is, the files is says are missing are definitely there

[bolshoytoster]

@dmlaj68 have you tried running npm i again?