Explainer api for local classifiers #102
Conversation
There was a problem hiding this comment.
@jannikgro and me had a look into it!
Great work - thanks for sharing! We will now try to use it to gain more insights but here are a few first thoughts from us about it!
hiclass/Explainer.py
Outdated
| for node in self.hierarchical_model.nodes: | ||
| model_at_node = self.hierarchical_model.get_model_at_node(node) | ||
| # Create a SHAP explainer for each node model | ||
| self.explainers[node] = shap.Explainer(model_at_node, background_data[node]) |
There was a problem hiding this comment.
here you hard-coded the Explainer class from shap instead of the subclasses that you found out earlier - doesn't that overwrite the choice the user took earlier?
in addition as a user having to sort the data for each node without knowing how you traverse it might it make hard to create the expected dictionary that is reference by background_data[node] don't you think? Would it be possible for you to sort that? (No is a valid answer ;))
hiclass/Explainer.py
Outdated
|
|
||
| def _explain_lcppn(self, X): | ||
| shap_values_dict = {} | ||
| # TODO: Use predictions to restrict traversal to only visited path while computing shap values |
There was a problem hiding this comment.
One idea would be to use the predict method of the individual classifiers which points you the way to the next node doesn't it?
| shap_values = local_explainer.shap_values(X) | ||
| shap_values_dict[parent_node] = shap_values | ||
|
|
||
| return shap_values_dict |
There was a problem hiding this comment.
This is cool to have as a dictionary but not 100% useful if we do not know which was the predicted and/or the correct path the model has chosen. Would it be possible to return the predicted label and the corresponding shap values?
|
We are not completely sure, because you wanted the explainer to be installable via a pip flag which we might have missed but as we installed it, the |
ashishpatel16
changed the title
iwan-tee
force-pushed
the
explainer_api
branch
from
1de360c to
0ea8956
Compare
Explainer api lcpl
Explainer api lcpn
| model.hierarchy_.nodes[node]["classifier"] | ||
| for node in model.hierarchy_.nodes | ||
| if "classifier" in model.hierarchy_.nodes[node] | ||
| def get_predict_proba(self, X): |
There was a problem hiding this comment.
@jannikgro I have updated the function to return a dict, however it currently just returns predict_proba for all nodes in the hierarchy. I'm woking on another version which only gives the dict for only traversed nodes.
ashishpatel16
changed the title
keeping it unchanged for now
This PR aims to introduce explainer class which uses shap to compute shapley values for each classifier present at a specific position in the hierarchy.