Clean up <bee_workdir>

[pagrubel]

To make this light work that we can pick up here and there check off the tasks and include a PR# by the task as they are completed. Tasks for organizing the bee_workdir:

• rename workflows directory to active_workflows and archives to archived_workflows
• move all *.db files into a directory called database_files
• move the redis.sock from the redis directory into the sockets directory
• change deps directory to dependencies and move the redis and celery directories into it

We started with sockets all in one directory, now there are sockets for redis in another directory. Also, we had the idea of having one dependent container in <bee_workdir>/deps.

We need to clean up <bee_workdir> and make some rules for it.

[pagrubel]

Combining items from #733 here and closing #733
all in bee_workdir
Rename deps to dependencies with subdirectories redis and redis_container and neo4j_container (rename dependent_container -> neo4j_container.

[pagrubel]

Data base files are in the top level reconsider.

Determine the rules and publish in contributing area.

[pagrubel]

This has been cleaned up some, here is listing of the current directories in bee_workdir (by default ~/.beeflow):

These are the current directories:
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 27 16:04 archives
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 16:33 celery
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 17:24 container_archive
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 16:33 deps
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 16:33 gdb_mount
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 16:33 logs
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 17:11 redis
-rw-r--r-- 1 pagrubel pagrubel 8192 Aug 26 17:12 sched.db
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 17:12 scheduler
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 26 17:11 sockets
-rw-r--r-- 1 pagrubel pagrubel 24576 Aug 27 16:19 tm.db
-rw-r--r-- 1 pagrubel pagrubel 24576 Aug 27 16:04 wfm.db
drwxrwxr-x 2 pagrubel pagrubel 4096 Aug 27 16:04 workflows

dependent containers (neo4j & redis) are unpacked into deps now that they are named _container we could put their associated directories in there

Should we just leave well enough alone? Maybe just have a README that describes the top level files and have some future suggested changes such as a directory for all the database files (tm.db, sched.db, wfm.db), move redis/redis.sock with the other sockets, move redis and celery into deps, call archives archived_workflows and workflows, active-workflows

Or I could start working on the changes. even break it into small PR's

The top level would then look like this:

active_workflows (was workflows)
archived_workflows (was archives)
container_archive
database_files (will have all .db files)
dependencies (with installed containers and other dependency directories)
gdb_mount
logs
scheduler
sockets (will now have redis.socket)

[rstyd]

I think that's a great suggestion! The current version definitely needs some refining.

[pagrubel]

Ok I made a list at the top of this issue and plan to start on it. If anyone picks part of it up let me know. I'll start with the first item.