Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run docker image as non-root user #1011

Merged
merged 14 commits into from
Nov 14, 2024
Merged

Run docker image as non-root user #1011

merged 14 commits into from
Nov 14, 2024

Conversation

pixil98
Copy link
Contributor

@pixil98 pixil98 commented Oct 18, 2024

Fixes #439

Warning

This PR contains breaking changes for the docker image. Read Docker.md before upgrading.

This contains a pretty big overhaul of the docker image. The main changes are:

  • The docker image now runs as user 1001 and group 1001 by default. It also supports having that overridden by docker at runtime.
  • Settings.json no longer needs to be edited by hand to specify the correct file paths. The image will substitute the correct values as runtime.
  • If the database can't be found, it will attempt to make a blank database and use it.
  • The default behavior is now to run once and exit, rather than loop every 30 minutes.

Together, these should make it easier for a new user to get the image up and running. At a minimum now, they only need to provide the AccountsSettings.json and it will work.

@wtanksleyjr @robflate @muchtall - Are any of you willing to review this PR and test it before it's merged?

Testing

I tested the following scenarios:

  • Image runs successfully as default user 1001:1001
  • Image runs successfully when overridden to user 2002:2002
  • Image creates a blank AccountsSettings.json and Settings.json if none are provided.
  • Image supports the AccountsSettings.json and Settings.json being mounted read only.
  • Image warns the user if the data directory isn't mounted.
  • Image overrides the provided Settings.json for Books and InProgress
  • Image loads the database first from /db, then from, /config, and finally creates a new one in /config.
  • Image errors out of it tries to create a database and doesn't have permission.
  • Image defaults to running once and exiting.

@rmcrackan
Copy link
Owner

This is all greek to me 😄 Let me know anytime you want me to merge.

@rmcrackan
Copy link
Owner

Would you mind also taking a look at the docker documentation to ensure it's up to date?

https://github.com/rmcrackan/Libation/blob/master/Documentation/Docker.md

@pixil98 pixil98 marked this pull request as draft October 19, 2024 03:14
@pixil98 pixil98 changed the title Draft: Run docker image as non-root user Run docker image as non-root user Oct 19, 2024
@pixil98 pixil98 force-pushed the master branch 4 times, most recently from ccf9049 to 9bc53e4 Compare October 19, 2024 06:31
@pixil98 pixil98 force-pushed the master branch 4 times, most recently from 96e5022 to a74722e Compare October 21, 2024 21:49
@pixil98 pixil98 force-pushed the master branch 5 times, most recently from 3bf4c6b to 0179fd8 Compare October 22, 2024 05:26
@pixil98 pixil98 force-pushed the master branch 2 times, most recently from 2434618 to 011efe3 Compare October 22, 2024 05:39
@pixil98 pixil98 force-pushed the master branch 2 times, most recently from 8dee854 to 4f7a2ce Compare October 22, 2024 07:54
@pixil98 pixil98 force-pushed the master branch 7 times, most recently from bd09e1e to 9e69b4a Compare October 22, 2024 14:20
@wtanksleyjr
Copy link
Contributor

I am not going to have time until November ... oof.

Creating a blank database seems to me like it might cause some deferred risks. Specifically, the user might simply have misplaced or misnamed the DB they actually wanted to use, and wind up doing multiple downloads and having multiple databases with conflicting statuses, and we're going to get confused questions about what they need to do now. I feel like, since we already need to share accounts.json with a GUI instance, we might as well just also share the DB with them.

On the other hand, the other settings are a real pain to share with a GUI. The GUI has some clever logic that does code point replacements for invalid Windows characters, but it only does that when it creates a new config file, and only when it starts on Windows. I want those replacements on my Docker, because that Docker is creating files I'm going to use on Windows (and Linux, and Android). But I also want the paths to keep working for Docker, because of course the same file with those settings also contains paths that are specific to the Windows installation that created them.

I guess what I'm saying is, maybe we could process the settings files to make them appropriate for Docker.

@pixil98
Copy link
Contributor Author

pixil98 commented Nov 1, 2024

I was a bit concerned about creating the blank db for that reason. I would really like to make initial setup of the docker image as easy as possible. One easy thing I can do is check for any .db file and assume that's the database before creating a new one. I can also add an env var which, if set, defines the name of the db to use. Another option would be to gate creation of the db behind a new env var.

For the character replacement, I'd rather not hard code those values into the docker script. It would be best if Libation could generate the defaults on demand and we can either use it as the default settings file or I can merge that section with the provided settings.

@pixil98
Copy link
Contributor Author

pixil98 commented Nov 14, 2024

I've reworked the image startup script to do the following regarding database files:

  • Check whether /db is mounted, if it is expect the database to be there; otherwise expect it in /config
  • Figure out the expected db file name:
    • If LIBATION_DB_FILE is set, expect the database file to use that name
    • If it is not set, check how many *.db files are in the directory
      • If there is more than one, error and tell the user to set LIBATION_DB_FILE
      • If there is one, use it
      • If there are zero set the expected db name to the default file name
  • Check if the expected file exists in the directory we've chosen
    • If it does, use it
    • If it does not, and LIBATION_CREATE_DB is true (it's true by default), create an empty db file.

This isn't fool proof, but it adds a decent amount of guard rails around database loading while trying to maximize how easy it is to adopt. If we start seeing an influx of people having problems with the database, I'm happy to revisit these.

I've also had my modified image (pre-db-rework) running at home for the last few weeks without problem/

@rmcrackan
Copy link
Owner

Sounds good. I assume since the PR is still in draft that you intend to do more before I merge.

@pixil98
Copy link
Contributor Author

pixil98 commented Nov 14, 2024

I'd like it if someone was willing to try the various envvar combinations in case I missed something with my testing. I'll move it out of draft and leave it up to you on how tested is enough tested.

@pixil98 pixil98 marked this pull request as ready for review November 14, 2024 19:58
@pixil98
Copy link
Contributor Author

pixil98 commented Nov 14, 2024

@rmcrackan When you create the release, can you put a warning section in the notes? You can copy this one:

Warning

This release contains breaking changes for the docker image. Read Docker.md before upgrading.

@rmcrackan
Copy link
Owner

I don't have docker testers per se. It's whoever volunteers. I'll merge this and create a pre-release with your warning. Nice warning markdown btw -- I've not seen that before.

@rmcrackan rmcrackan merged commit 5fff22a into rmcrackan:master Nov 14, 2024
9 checks passed
@rmcrackan
Copy link
Owner

It looks like the docker release no longer works after this PR:

https://github.com/rmcrackan/Libation/actions/runs/11844958812/job/33009709209


2024-11-14T20:33:52.9740347Z Current runner version: '2.320.0'
2024-11-14T20:33:52.9763245Z ##[group]Operating System
2024-11-14T20:33:52.9763926Z Ubuntu
2024-11-14T20:33:52.9764399Z 22.04.5
2024-11-14T20:33:52.9764696Z LTS
2024-11-14T20:33:52.9765051Z ##[endgroup]
2024-11-14T20:33:52.9765483Z ##[group]Runner Image
2024-11-14T20:33:52.9765916Z Image: ubuntu-22.04
2024-11-14T20:33:52.9766329Z Version: 20241103.1.0
2024-11-14T20:33:52.9767369Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20241103.1/images/ubuntu/Ubuntu2204-Readme.md
2024-11-14T20:33:52.9768769Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20241103.1
2024-11-14T20:33:52.9769644Z ##[endgroup]
2024-11-14T20:33:52.9770202Z ##[group]Runner Image Provisioner
2024-11-14T20:33:52.9770784Z 2.0.384.1
2024-11-14T20:33:52.9771151Z ##[endgroup]
2024-11-14T20:33:52.9786793Z ##[group]GITHUB_TOKEN Permissions
2024-11-14T20:33:52.9788566Z Actions: write
2024-11-14T20:33:52.9789197Z Attestations: write
2024-11-14T20:33:52.9790136Z Checks: write
2024-11-14T20:33:52.9790535Z Contents: write
2024-11-14T20:33:52.9790891Z Deployments: write
2024-11-14T20:33:52.9791429Z Discussions: write
2024-11-14T20:33:52.9791843Z Issues: write
2024-11-14T20:33:52.9792187Z Metadata: read
2024-11-14T20:33:52.9792650Z Packages: write
2024-11-14T20:33:52.9793025Z Pages: write
2024-11-14T20:33:52.9793358Z PullRequests: write
2024-11-14T20:33:52.9793865Z RepositoryProjects: write
2024-11-14T20:33:52.9794315Z SecurityEvents: write
2024-11-14T20:33:52.9794702Z Statuses: write
2024-11-14T20:33:52.9795156Z ##[endgroup]
2024-11-14T20:33:52.9798076Z Secret source: Actions
2024-11-14T20:33:52.9798647Z Prepare workflow directory
2024-11-14T20:33:53.0424942Z Prepare all required actions
2024-11-14T20:33:53.0578818Z Getting action download info
2024-11-14T20:33:53.2559558Z Download action repository 'actions/download-artifact@v4' (SHA:fa0a91b85d4f404e444e00e005971372dc801d16)
2024-11-14T20:33:53.5765321Z Download action repository 'softprops/action-gh-release@v2' (SHA:e7a8f85e1c67a31e6ed99a94b41bd0b71bbee6b8)
2024-11-14T20:33:53.8043998Z Download action repository 'dwenegar/upload-release-assets@v2' (SHA:52129c5149f68c1f8d5a85d7765ccd7316ec144e)
2024-11-14T20:33:54.1876056Z Complete job name: release
2024-11-14T20:33:54.2785289Z ##[group]Run actions/download-artifact@v4
2024-11-14T20:33:54.2786064Z with:
2024-11-14T20:33:54.2786451Z   path: artifacts
2024-11-14T20:33:54.2786883Z   merge-multiple: false
2024-11-14T20:33:54.2787444Z   repository: rmcrackan/Libation
2024-11-14T20:33:54.2787936Z   run-id: 11844958812
2024-11-14T20:33:54.2788355Z ##[endgroup]
2024-11-14T20:33:54.6692710Z Found 9 artifact(s)
2024-11-14T20:33:54.6695561Z No input name or pattern filtered specified, downloading all artifacts
2024-11-14T20:33:54.6697384Z An extra directory with the artifact name will be created for each download
2024-11-14T20:33:54.6698748Z Preparing to download the following artifacts:
2024-11-14T20:33:54.6700149Z - Libation.11.5.3-windows-chardonnay.zip (ID: 2189862295, Size: 120826995)
2024-11-14T20:33:54.6701929Z - Classic-Libation.11.5.3-windows-classic.zip (ID: 2189860513, Size: 104760940)
2024-11-14T20:33:54.6703441Z - rmcrackan~Libation~KELK7Q.dockerbuild (ID: 2189859630, Size: 108144)
2024-11-14T20:33:54.6705156Z - Libation.11.5.3-linux-chardonnay-amd64.deb (ID: 2189857084, Size: 65495048)
2024-11-14T20:33:54.6706788Z - Libation.11.5.3-linux-chardonnay-arm64.deb (ID: 2189856720, Size: 57732017)
2024-11-14T20:33:54.6708208Z - Libation.11.5.3-linux-chardonnay-arm64.rpm (ID: 2189856666, Size: 81219915)
2024-11-14T20:33:54.6709900Z - Libation.11.5.3-linux-chardonnay-amd64.rpm (ID: 2189856436, Size: 87281603)
2024-11-14T20:33:54.6711605Z - Libation.11.5.3-macOS-chardonnay-arm64.tgz (ID: 2189856159, Size: 85986740)
2024-11-14T20:33:54.6713107Z - Libation.11.5.3-macOS-chardonnay-x64.tgz (ID: 2189855296, Size: 91261984)
2024-11-14T20:33:54.7423580Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-d6f0ff85-c3a5-58a8-8588-7310fc867add/artifacts/1b2dc2553474aac3e2b9d3f20d500ae5376865088908f7683ce591ab387d4797.zip
2024-11-14T20:33:54.7426193Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Libation.11.5.3-windows-chardonnay.zip
2024-11-14T20:33:54.7721989Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-1a6a7b61-2e51-5ed1-1619-cbc830c18182/artifacts/3d0a31d6a086415a553f5fcd03c66dc4b5b749f8b5ebb6c87a8191eed5492a51.zip
2024-11-14T20:33:54.7725794Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Libation.11.5.3-linux-chardonnay-arm64.deb
2024-11-14T20:33:54.7768016Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-2210eb47-3597-5257-07c3-594eeafac6e1/artifacts/cbb1271be522ce5926516302f0cf12b65531165a5c9d8bb9c8b104e7b6fd16ee.zip
2024-11-14T20:33:54.7771826Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Libation.11.5.3-macOS-chardonnay-arm64.tgz
2024-11-14T20:33:54.7788329Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-bdaad074-93fb-5136-9b9b-068d5b484bd0/artifacts/0aa796440f3df741369de5351fabc8539ce764d53692d54efefc4f8e84a9bb4b.zip
2024-11-14T20:33:54.7792659Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Classic-Libation.11.5.3-windows-classic.zip
2024-11-14T20:33:54.7805563Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-a169b1d7-a2f0-5367-c5ae-fae4f17224a2/artifacts/c8efc8e88fd156cdb7d6e690a674799f1a334a72656d6e07b0a24e86d0f29d04.zip
2024-11-14T20:33:54.7809313Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Libation.11.5.3-macOS-chardonnay-x64.tgz
2024-11-14T20:33:54.7835941Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-89b0335e-dfee-5005-8822-a285ddb1163e/artifacts/2049d78af72310acb264239ddfdfc98c903c50cf12b7503d58179e52570d92e8.zip
2024-11-14T20:33:54.7839708Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Libation.11.5.3-linux-chardonnay-amd64.rpm
2024-11-14T20:33:54.7853909Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-744e6962-01a9-5a6d-f115-7effaf80fe8e/artifacts/ccbc3fc0da8dc5efb5e2e4d25bf7051d8f587da5125a15ce11a9a9507407abe5.zip
2024-11-14T20:33:54.7857809Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Libation.11.5.3-linux-chardonnay-amd64.deb
2024-11-14T20:33:54.7958220Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-aa4d2816-b89c-5445-258d-ca8d34fd5683/artifacts/88a44ae927ec03ce92cc021f07ecd34753d217e3f9c0e0c7e97e46348aab7751.zip
2024-11-14T20:33:54.7962026Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/Libation.11.5.3-linux-chardonnay-arm64.rpm
2024-11-14T20:33:54.7979264Z Redirecting to blob download url: https://productionresultssa6.blob.core.windows.net/actions-results/006bcbba-39c1-44ea-a60b-eb27b97b5b49/workflow-job-run-67c575a9-2f58-5c86-c74c-ea504f146b8e/artifacts/526119b1d72915088da200b47568d5c8fa56687ba2c0e87bbacde29202b33403.zip
2024-11-14T20:33:54.7982804Z Starting download of artifact to: /home/runner/work/Libation/Libation/artifacts/rmcrackan~Libation~KELK7Q.dockerbuild
2024-11-14T20:33:54.8199129Z (node:1629) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
2024-11-14T20:33:54.8201997Z (Use `node --trace-deprecation ...` to show where the warning was created)
2024-11-14T20:33:57.9234714Z Artifact download completed successfully.
2024-11-14T20:33:58.2823847Z Artifact download completed successfully.
2024-11-14T20:33:58.8915876Z Artifact download completed successfully.
2024-11-14T20:33:59.0299737Z Artifact download completed successfully.
2024-11-14T20:33:59.0638550Z Artifact download completed successfully.
2024-11-14T20:33:59.1332643Z Artifact download completed successfully.
2024-11-14T20:33:59.3007097Z Artifact download completed successfully.
2024-11-14T20:33:59.4660573Z Artifact download completed successfully.
2024-11-14T20:34:19.9007895Z ##[error]Unable to download artifact(s): Unable to download and extract artifact: Artifact download failed after 5 retries.
2024-11-14T20:34:44.9418214Z Cleaning up orphan processes

@pixil98
Copy link
Contributor Author

pixil98 commented Nov 14, 2024

I'll get a fix for it shortly.

@rmcrackan
Copy link
Owner

Hey docker folks, can you take a look at this comment about these recent changes? #1049 (comment)

I'm not familiar with how docker uses latest and it may be messing with people.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Docker container creates all files owned by root
3 participants