Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support using a Conf store from multiple processes #146

Open
timoxley opened this issue Mar 11, 2021 · 4 comments
Open

Support using a Conf store from multiple processes #146

timoxley opened this issue Mar 11, 2021 · 4 comments

Comments

@timoxley
Copy link

Currently if you have multiple processes using conf with the same storage location they can, and do, clobber each other's data.

e.g.

  1. p1 reads store
  2. p2 reads store
  3. p1 writes store
  4. p2 writes store (this clobbers p1's changes in step 3, even if the change was a totally different key)

Any suggestions on how to work around this?

I feel like this could warrant a big disclaimer in the readme.

sindresorhus added a commit that referenced this issue Mar 19, 2021
@sindresorhus
Copy link
Owner

I added a note to the readme: ce1dcee

I simply never considered this use-case. It would be nice to support though, but it would require locking the store file during read/write, and the file locking story in Node.js is not great. We would also need to read the config from disk on write and merge it in before writing.

@sindresorhus sindresorhus changed the title Data clobbering when multiple processes write to a single store Support using a Conf store from multiple processes Mar 19, 2021
@timoxley
Copy link
Author

timoxley commented Mar 22, 2021

it would require locking the store file during read/write, and the file locking story in Node.js is not great.

I had a crack at working around this using https://www.npmjs.com/package/lockfile but this was a non-starter since conf implements a sync interface and lockfile, "obviously" doesn't support waiting for a resource to become unlocked using synchronous operations.

For my use-case I (was) using a Map-interface wrapper around conf as the persistent cache for mem, so even if conf became async, it wouldn't solve my particular issue.

I wonder if there's some way to do a synchronous wait that doesn't involve assaulting the cpu/disk with a while loop. Perhaps some hackery with exec.execSync.

@papb
Copy link

papb commented Mar 23, 2021

I wonder if there's some way to do a synchronous wait that doesn't involve assaulting the cpu/disk with a while loop. Perhaps some hackery with exec.execSync.

There is, by Sindre: https://github.com/sindresorhus/make-synchronous

@sindresorhus
Copy link
Owner

Actually, https://github.com/sindresorhus/sleep-synchronously would be better for this purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants