Allow ets/dets as backends in place of redis

[OldhamMade]

It would be great if toniq could use ets/dets as a backend in place of redis, to reduce dependencies for use-cases where persistence of jobs/tasks isn't necessary.

[joakimk]

I want this feature too someday but I don't have any need for it myself and I'm not motivated to work on toniq just for fun in the near future. Contributions are welcome.

I'm curious, if you don't persist, what makes you use toniq? Retries? High level DSL for jobs?

[OldhamMade]

A workload where starting from the beginning isn't that costly, or something run regularly where incremental failures can be ignored.

[whitfin]

Agreed with @OldhamMade - same use case. Jobs are gonna run frequently so if they fail it doesn't matter because they'll just run again in a few hours.

I'll take a look at contributing but don't have much free time atm so no promises.

[joakimk]

That seems reasonable.

As toniq is an in-memory queue that only uses redis as backup, making this change should not be that hard.

I think skipping persistance could be accomplished by changing enqueue to build a job in memory instead of storing it and changing how process_result is handled.

Exactly on what level to do these changes I don't know without diving in and trying to make the change :). I won't do this anytime soon though, so feel free to try.

[jalcine]

This is the reason I'm opting for Toniq in https://koype.net; largely for the DSL + retry logic. I'm guessing (ignorantly) that no headway towards decoupling Redis has been made?