irods_sync script fixes and enhancements (plus one other bug fix)
#294
Conversation
|
Are you looking for feedback on this PR even though it's in draft? |
|
I want to confirm that this is doing what we want before marking as ready for review. Feel free to leave feedback on what you see, but the implementation may change significantly depending on how the requirements settle out. |
|
Ok. Will put eyes on this once it's out of draft. |
alanking
force-pushed
the
irods_sync-fixes
branch
2 times, most recently
from
c32360e to
49139e8
Compare
|
I squashed the two #210 commits together since we're pretty into the stopped jobs list. The stopped jobs list now includes all the information shown in the active jobs list subcommand output. I think this is ready for review... |
|
Ok. Will get eyes on it. |
|
Anything left for this? Are the tests passing? |
|
I'll run the test suite once more for good measure, but these changes mainly affect the sync script rather than the framework, so these should probably be passing. I'll report back with results soon. I would also like to update the commit message for "Reset Redis handles for stopped jobs", but that's about it for this one. |
|
Sounds good. Will wait for the test results before approving. |
|
Tests are passing. |
alanking
force-pushed
the
irods_sync-fixes
branch
from
6f7b0ea to
f4e4dd5
Compare
|
Squashed, tweaked commit message, and added a thing that Black formatter caught. |
This ensures that interrupting watch (or start --progress) does not display the whole stacktrace and exit with an error code.
This commit adds a new Redis key for ingest job start times. The value is reset along with the other information related to a job. This allows monitoring jobs with accurate "elapsed time" reporting.
Adds a new list to the tracked Redis keys: irods_ingest_stopped_jobs. This tracks jobs which are manually stopped with the stop subcommand. Displays a warning message to the irods_sync script's log output when a job being monitored by start --progress or watch is stopped. The list subcommand now shows the list of stopped jobs and last-known information about them when they were stopped.
This commit adds job information which is available when monitoring a job to the list subcommand. These include the job name (already shown previously), total number of tasks, number of tasks remaining, number of failed tasks, number of retried tasks, a formatted timestamp of what time the job started, and the amount of time elapsed since the job started. Also aligns watch subcommand output with information now being shown in list subcommand output.
This commit documents several new irods_sync script features as well as adds documentation for some of the subcommands which previously had little to no documentation. The stop subcommand now adds job names to a new Redis key, which is being documented. The list subcommand got some new information to display in addition to being able to list stopped jobs. The irods_sync script section has been reorganized as well so that the parts fit together a little better.
alanking
force-pushed
the
irods_sync-fixes
branch
from
f4e4dd5 to
31dd296
Compare
|
#'d, mergin |
Addresses #91
Addresses #92
Addresses #93
Addresses #210
Addresses #293
I'm contemplating adding tests for these subcommands, but it seems a little tricky (hence, draft PR) and there are no existing tests for any of the subcommands other than
start. Existing tests are passing.Also, the two commits referencing #210 (only) can be squashed into one if we like the solution of maintaining a list of stopped jobs. Otherwise, we can drop the second commit because the first one addresses the issue (albeit, not as well, IMO).