Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise script/task to refine logic for updating model #46

Merged
merged 11 commits into from
Feb 6, 2025

Conversation

rlskoeser
Copy link
Contributor

refinements for #39 and #42

@rlskoeser rlskoeser requested a review from cmroughan February 4, 2025 15:44
@cmroughan
Copy link
Collaborator

Looks like we're passing the usual tests for training from scratch, refining, and refining + overwriting on segmentation and transcription jobs! Cleanup, task messaging, and model uploading (or not uploading, if a poorer model would overwrite a better one) is working correctly.

I committed a change to the overwrite handling to address the case where "overwrite" is selected, kraken finds a _best.mlmodel, but kraken's best is lower accuracy than the starting model. In that case I think it makes sense to continue following the logic of not overwriting the higher accuracy model.

Also added a commit to handle the edge case where a queued job is cancelled in MyDella, since this results in no .out file being created and was causing the script to error out on a FileNotFoundError.

I'm setting up one last round of tests to run tonight and will double check the results in the morning but I think we're looking pretty good here!

@cmroughan
Copy link
Collaborator

cmroughan commented Feb 5, 2025

Results of last night's tests successfully showed the behavior we want to see. The various data is also being sent to Task messages correctly and this continues to be very helpful for testing and troubleshooting. I also ran tests to confirm that when one person shares a model with another person, the second person's user permissions still do not permit overwriting the original person's model (though they can use it for non-overwrite refining as usual).

@rlskoeser rlskoeser merged commit 62d5bb0 into develop Feb 6, 2025
@rlskoeser rlskoeser deleted the feature/script-update-model branch February 6, 2025 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants