Release 0.8.5

[nnansters]

Hello everybody!

We've just released NannyML 0.8.5! We're introducing a new metric called business value, fixed our broken container builds, and improved consistency in our plots. Let's dive in!

Installing / upgrading

You can get this latest version by using pip:

pip install -U nannyml

Or conda:

conda install -c conda-forge nannyml

What’s new?

Estimated and realized business value

We've added the business value metric for estimated and realized binary classification performance. It allows you to assign a value (or cost) to true positive, true negative, false positive, and false negative predictions.
As you calculate or estimate the confusion matrix for your model over time, you can track its business value and impact.
The following snippet shows how to estimate business value in the absence of ground truth labels:

import nannyml as nml
from IPython.display import display

reference_df = nml.load_synthetic_car_loan_dataset()[0]
analysis_df = nml.load_synthetic_car_loan_dataset()[1]

display(reference_df.head(3))

estimator = nml.CBPE(
    y_pred_proba='y_pred_proba',
    y_pred='y_pred',
    y_true='repaid',
    timestamp_column_name='timestamp',
    metrics=['business_value'],
    chunk_size=5000,
    problem_type='classification_binary',
    business_value_matrix=[[5, -10], [-50, 50]],
    normalize_business_value="per_prediction",
)

estimator.fit(reference_df)

results = estimator.estimate(analysis_df)
display(results.filter(period='analysis').to_df())

metric_fig = results.plot()
metric_fig.show()

Read more in the business value tutorials (estimated or realized) or the how it works page.

Create your own custom binary classification metric!

We've added a new tutorial teaching how you can create your own custom metrics using NannyML. Learn how to estimate the balanced accuracy metric here!

What's changed?

• Our Docker pipeline had been failing silently, causing our images to be outdated for a while. We've fixed the issues there so you can enjoy freshly baked images again!
• We've applied some simple styling rules to our plots:
  • Estimated metrics will be plotted as an indigo (that's purple) dotted line
  • Non-estimated (realized, calculated) metrics will be plotted as a blue, solid line
  • Comparison plots try to respect these rules, but might use the other color to clearly separate the lines

What's next?

We should have another update for you quite soon as we wrap up some data-quality related work.
We hope the additions in 0.8.5 bring you some additional business value (see what I did there?). As always, any feedback is encouraged! Reach out in our community Slack, log a bug or a feature request our repository or just leave us a star for good karma!