Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make workers part of a single job #13

Open
esheldon opened this issue Feb 9, 2022 · 1 comment
Open

make workers part of a single job #13

esheldon opened this issue Feb 9, 2022 · 1 comment

Comments

@esheldon
Copy link

esheldon commented Feb 9, 2022

It will make killing the jobs easier when debugging if it is a single job (with sub-jobs) rather than different jobs for each worker

E.g. these job numbers

11.0  esheldon  02/09 11:56:02  0+00:01:11  C   0    976.6   job-b17222a75feb46e0a3f27657e460c796-f45c2a42b53f4349b96c29611c0f20f4
 12.0  esheldon  02/09 11:56:02  0+00:01:13  C   0    1220.7  job-b17222a75feb46e0a3f27657e460c796-27b8b2f6b61c49f5b4744b415a41d9ca
 13.0  esheldon  02/09 11:56:02  0+00:01:11  C   0    976.6   job-b17222a75feb46e0a3f27657e460c796-3818e5d7991d4b0b93f1e618d131bcc2
 14.0  esheldon  02/09 11:56:02  0+00:02:42  C   0    1220.7  job-b17222a75feb46e0a3f27657e460c796-ff23db37ba4b4a2a899541d708d7c314
 15.0  esheldon  02/09 11:56:02  0+00:01:13  C   0    976.6   job-b17222a75feb46e0a3f27657e460c796-eb1dbb171875464f91a93c44baaf4664
 16.0  esheldon  02/09 11:56:02  0+00:01:12  C   0    1220.7  job-b17222a75feb46e0a3f27657e460c796-e9cec699e17843a3971b7c7ff02c1bc2
 17.0  esheldon  02/09 11:56:02  0+00:01:13  C   0    976.6   job-b17222a75feb46e0a3f27657e460c796-879af623102346298457e024c85599b4
 18.0  esheldon  02/09 11:56:02  0+00:01:15  C   0    976.6   job-b17222a75feb46e0a3f27657e460c796-4007966290a24d03ab07fcfc55c509c9
 19.0  esheldon  02/09 11:56:02  0+00:01:11  C   0    976.6   job-b17222a75feb46e0a3f27657e460c796-6590931b40a14a47b9a9c1eb85ed482b
 20.0  esheldon  02/09 11:56:02  0+00:01:15  C   0    976.6   job-b17222a75feb46e0a3f27657e460c796-2a039ab0b028496a87a244c964b305ec

could have been

11.1
11.2
11.3
etc.

then could kill with condor_rm 11

@beckermr
Copy link
Owner

Does condor allow adding queue commands to a submitted job?

The big issue here is that this is not a pilot job model. The way that concurrent.futures works constrains the executor to submit one condor job per pool.submit function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants