feat: continuously keep in sync

[WhyNotHugo]

I've been trying out neverest and it's been working well for me to keep email in sync.

My main annoyance right now is that email is synchronised, and then neverest exits, so I need to manually run it again every now and then.

This is a feature request to run continuously synchronising emails. Which IDLE or NOTIFY support would be ideal, I consider as an extended goal beyond this feature. Just running a regular sync every 5 minutes with the same process would also suffice too.

Alternatives considered

I can't script it to auto-run every 5 minutes because I'd get a prompt from the secret storage to disclose the password every 5 minutes, which would be an annoyance.

[soywod]

Initially, the sync and the watch were part of the Himalaya project. Both domains moved away into dedicated projects:

• Neverest, for synchronization
• Mirador, for watching mailboxes: this is what you are looking for

I prefer not to do tight integration between both, so people have free hands on the way they want to use and compose them. I like the Unix approach, where one tool has one purpose, so that they compose nicely together (without explicit integration).
I do have a strong opinion on the subject, but I am open to discussion as usual.

[soywod]

Side note on both CLIs: they did not yet reach the v1, but should be soon! The main blocker is the IMAP CONDSTORE extension which will drastically improve the experience (both for sync and for detecting changes on watch).

[WhyNotHugo]

I don't see how using Mirador would help for this situation.

Mirador executes a separate command each time the remote storage changes. If that program were neverest, it would try to read credentials from the secrets manager every time that a change happens, so I'd get an authorisation prompt every time an email arrives. That's not really an improvement over getting a prompt every five minutes or some similar schedule.

Does mirador have some mechanism to allow me to pass through the credentials to neverest?

[soywod]

I apology, I did not read carefully your initial request. You need the same process to run so you don't have enter credentials everytime. Indeed it would not work with a separate CLI. It should not be difficult to implement, yet I found that this feature is slighly out of the scope.

Instead of keeping a Neverest loop in a process (and keeping secrets in memory), it would be more efficient to use a safe in-memory database.

This usecase reminds me one of our discussion about OAuth: it could be great to have a dedicated tool that takes care of authentication/authorization. A bit like does pizauth, but with a more general purpose.

[WhyNotHugo]

A secret storage service (e.g.: KeePassXC, himitisu, or even potentially something like 1Password) is a "safe in-memory database". If neverest exist, then the next time that it runs it opens a new connection to that secret service, so the secret service will show an interactive prompt for the user to confirm the access request again.

[soywod]

A secret storage service (e.g.: KeePassXC, himitisu, or even potentially something like 1Password) is a "safe in-memory database".

When I said safe in-memory I meant non-persistent. Secret storage are all by default on-disk, but not all of them propose a in-memory service for fast access.

What I had in mind is a layer at the top of secret storages. Once you ask a password from your secret storage, a copy is safely held inside this in-memory layer and is erased after reboot (or delay). On Linux, this layer could just be the Linux Kernel Keyutil.

To give a concrete example:

• First you run this in-memory layer, your password will be prompted and kept safely in memory
• Then you run Neverest that will just reuse the same password from the in-memory layer (rather than directly from the secret storage). No more prompt.

[WhyNotHugo]

What I had in mind is a layer at the top of secret storages. Once you ask a password from your secret storage, a copy is safely held inside this in-memory layer and is erased after reboot (or delay).

I've considered this in the past for similar situations. Neverest will connect to this service, and request credentials. When neverest exist, it will drop its connection. When another instance runs it will start a new connection. The in-memory storage sees this as a new client, so won't disclose secrets. It would need to prompt the user for approval before doing so, but now we're in the same situation with which we started.

[soywod]

How come the GPG agent is capable of holding the same session for various clients? I personally use pass and GPG, and my password is only prompted according to the given GPG agent rules (every week, or after 2 days of inactivity). I have the feeling to miss knowledge here.

[WhyNotHugo]

I use it with a hardware key (a Yubikey) and it works as follows:

• On the first usage, I'm prompted for a PIN code.
• The PIN code unlocks the secret on the device.
• When I want to sign or decrypt data, I need to tap on the yubikey physically.
• When the yubikey is unplugged (or the system reboots), it locks its secret again. Next time, it will need a PIN before being usable again.

In this case, I have to tap on the Yubikey on each usage, so this is the equivalent "prompt for confirmation" before performing any operation with the secret that it contains.

---

I use KeePassXC as a password manager. It has a very similar flow:

• On the first usage, it prompts for the password.
• Each time that an application tries to read a credential via the API (including the secret-service API), KPXC will show a pop-up, asking me for confirmation before disclosing some secret.

I also use himitsu, a secret storage service which has an almost identical flow.

---

Note that clients can enumerate entries in the password manger or secret store, but they cannot retrieve secrets without explicit confirmation. Being able to do so would imply that any client which ever has access to a single credential has unrestricted access to all of them.

[soywod]

Thanks a lot for all these precisions, I learnt a lot.

The conclusion is that you need to reuse the same process. If there any way to make the process sleep, and to make it sync whenever it wakes up? This way we don't need to (re)implement the timer part, this can be done by any other top program like cron.

[WhyNotHugo]

Interesting approach. The simplest thing I can think of is to just wait for SIGUSR1 and sync when the signal is received.

[WhyNotHugo]

If you want to block while waiting for a signal, you can use std::sync::Condvar to block a thread or tokio::sync::Notify to await for an event.

[soywod]

Alright, then I will give it a shot as soon as my focus will be one the sync part. I guess the watch part could be interested in such feature as well?