Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ml pipelines #333

Closed
wants to merge 5 commits into from
Closed

feat: ml pipelines #333

wants to merge 5 commits into from

Conversation

nkoppisetty
Copy link
Member

@nkoppisetty nkoppisetty commented Dec 11, 2023

Explain what this PR does.

Adding ml_pipelines and a pipeline udf for configuring multiple ML pipelines for a single stream config.

Signed-off-by: Nandita Koppisetty <[email protected]>
Copy link

codecov bot commented Dec 11, 2023

Codecov Report

Attention: 13 lines in your changes are missing coverage. Please review.

Comparison is base (28fa28f) 93.34% compared to head (616f6c4) 93.25%.

Files Patch % Lines
numalogic/udfs/_base.py 73.33% 4 Missing ⚠️
numalogic/udfs/ml_pipeline.py 88.23% 4 Missing ⚠️
numalogic/udfs/__main__.py 60.00% 1 Missing and 1 partial ⚠️
numalogic/udfs/trainer/_prom.py 0.00% 2 Missing ⚠️
numalogic/udfs/preprocess.py 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #333      +/-   ##
==========================================
- Coverage   93.34%   93.25%   -0.10%     
==========================================
  Files          82       83       +1     
  Lines        3712     3778      +66     
  Branches      294      296       +2     
==========================================
+ Hits         3465     3523      +58     
- Misses        196      205       +9     
+ Partials       51       50       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@s0nicboOm s0nicboOm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments..

can we bump up the dev version for this change please :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is a new vertex. If yes, naming seems little confusing for this UDF.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah couldn't think of anything, let me know if you have something on mind

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

named it ml_pipeline for now, feel free to update it later with whatever suits best

numalogic/udfs/pipeline.py Outdated Show resolved Hide resolved
numalogic/udfs/pipeline.py Outdated Show resolved Hide resolved
numalogic/udfs/preprocess.py Show resolved Hide resolved
Signed-off-by: Nandita Koppisetty <[email protected]>
Signed-off-by: Nandita Koppisetty <[email protected]>
Signed-off-by: Nandita Koppisetty <[email protected]>
Signed-off-by: Nandita Koppisetty <[email protected]>
@s0nicboOm
Copy link
Contributor

Merged the changes after refactoring: #336

@s0nicboOm s0nicboOm closed this Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants