Issue#78 #102
Closed
Issue#78 #102
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The demo preparation includes a number of changes to the CLI, improving the experience of creating tasks, scheduling tasks and more. Refactor create_task and add support for pipelines The manifests have been updated to work with GKE, this require a few things related to pull secrets, RBAC and container users. The prepare_cluster.sh script has been included to easily reproduce a single node GKE cluster with a GPU for demo purposes.
Issue: #78 This commit fixes failing CLI tests as preparation to reimplementing the Rust controller in Rust. The failures were caused by prepartion for latest demo. A few tests were disabled, as they don't really carry over to the new components The changes are not polished as they will not be longlived, due to the pending migration to rust.
Issue: #78 As a part of migrating AME to the new Rust based stack, the Task controller and CRD must be reimplemented. Therefore this commit does just that with a few omissions. Some of the existing features have been left out, such as Task pipelines, as these will change drastically due to recent feedback and options with the new stack that weren't possible before. Therefore a new Task specification which takes all this into account will implemented in a separate commit. This commit focuses on creating a solid foundation for building controllers in Rust. A few other tools have been setup in this commit aswell: - justfile, this is a better alternative to Make for our usecase. - common rust code quality tools. - Updated CI to use these new tools. - Kind has been replaced with K3S, as this allows for a more "real" testing experience.
Issue: #78 In order to work with Tasks through the CLI and frontend it is important that basic CRUD operations are exposed through the gRPC service.
Issue: #84 The user should be able to override default images to support other usecases. Therefore, this commit exposes an image field in the Task config which then overrides the image used in the Workflow that handles executing the Task.
Issue: #78 Before AME is put in production we must have logging, tracing and health checks. Therefore this commit using Tower middleware to enable such features. Currently the health check is pretty barebones in that it just checks that the server kan reach the Kubernetes API. This is currently the only real requirement the server has for functioning. As the server grows in compexity the health check should be expanded accordingly.
Issue: #78 This commits implements the Rust based CLI with a limited feature. The features and polish are intentially limited here to focus on the new project structure and tools. Polish and more features will be left for a future commit. The AME service was expanded to handling logging and project file uploads inorder to faciliate the CLI features.
This commit improves various parts of the new Rust based components. A lot of different components are touched in this commit, normally that would be unacceptable. However as we are sacaffolding a new project structure this is hard to avoid. There are too many changes to describe them all in detail but here are the highlights: - Get the Task logging to work across multiple pods. - Add deployment manifests for minio. - Correct deployment manifests for the controller. - A new test project was created using the new AME file specification.
Issue: #78 As AME has been migrated to a new stack, the old implementation and associated tools are no longer needed. Therefore this commit removes all unrequired files. Extra: Update CI configuration to test the new implementation.
jmintb
force-pushed
the
issue#78
branch
2 times, most recently
from
0576d61 to
29fe4d1
Compare
jmintb
force-pushed
the
issue#78
branch
2 times, most recently
from
7e60df3 to
559d94f
Compare
jmintb
commented
Jan 4, 2023
jmintb
force-pushed
the
issue#78
branch
4 times, most recently
from
93eeec6 to
106233a
Compare
The CI configuration had multilple issues around the Rust related tooling. This commit addresses those and corrects a few typos. Extra: Delete Makefile as it is has been replaced with a justfile. Use minio credentials from K8S secret instead of environment variables. Address PR comments.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.