Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to deploying job-runner via docker image #68

Open
bloodearnest opened this issue May 16, 2022 · 2 comments
Open

Move to deploying job-runner via docker image #68

bloodearnest opened this issue May 16, 2022 · 2 comments
Assignees
Labels

Comments

@bloodearnest
Copy link
Member

bloodearnest commented May 16, 2022

Currently, job-runner is checked out in a git repo in /srv/jobrunner/code, and run with a systemd unit.

We build a job-runner docker image currently, but don't use it. But partners like graphnet do.

Running as a docker image gives us a better deployment story, and parity with our partners.

There are two issue blocking this:

  • job-runner needs access to the host docker instance. This is working in theory, but we cannot get it to pass tests atm: https://github.com/opensafely-core/backend-server/tree/madwort/deploy-job-runner-with-docker-2

  • We are now on the VM, and have recently limited total docker memory usage. However, if we run job-runner in docker, then it will be inside that limit and subject to memory exhaustion pressure from running jobs. It would be nice if we could ensure job-runner had it's own memory pool within that, but it's not critical. This is probably better solved by per-job limits

@bloodearnest
Copy link
Member Author

The job-runner docker image is now running as root, so this is unblocked opensafely-core/job-runner#447

@bloodearnest bloodearnest self-assigned this Jul 1, 2022
@benbc
Copy link
Contributor

benbc commented Sep 6, 2022

@bloodearnest We labelled this as part of the Graphnet work a while ago, just in order to nudge ourselves into getting it done. But it's really orthogonal, isn't it, and since there's no other work for this team on the Graphnet stuff at the moment it seems a bit odd. Shall we remove the label?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants