Here we provide the results of the models in MMIA. For supervised models, we have recorded three dimensions of metrics: ID classification, ID+OOD classification, and OOD detection.
| Dataset | Model Name | Dialogue Type | ID Classification | ID+OOD Classification | OOD Detection | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Acc | WF1 | WP | F1 | P | R | ACC | F1-Known | F1-Open | F1 | AUROC | AUPR-IN | AUPR-OUT | FPR95 | DER | |||
| MintRec | MAG_BERT | Multi | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MintRec | MULT | Multi | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MintRec | MAG_BERT | Single | 72.404 | 72.06 | 72.938 | 68.286 | 68.868 | 69.216 | - | - | - | - | 76.628 | 75.016 | 75.545 | 73.78 | 39.122 |
| MintRec | MULT | Single | 72.222 | - | - | 69.394 | 70.112 | 69.368 | - | - | - | - | - | - | - | - | - |
| MintRec | MMIM | Single | 72 | 71.964 | 72.906 | 68.856 | 69.72 | 69.046 | - | - | - | - | 76.142 | 76.392 | 773.768 | 76.622 | 40.538 |
| MintRec | SDIF | Single | 71.642 | 71.336 | 71.736 | 68.188 | 69.076 | 68.3 | - | - | - | - | 78.608 | 77.28 | 78.122 | 70.578 | 37.556 |
| MintRec | TCL_MAP | Single | 73.166 | 72.656 | 72.974 | 68.922 | 68.9 | 69.988 | - | - | - | - | 75.208 | 74.58 | 73.274 | 78.888 | 41.64 |
| MintRec2.0 | MAG_BERT | Multi | 60.66 | 59.776 | 59.89 | 53.78 | 55.698 | 53.938 | 56.056 | 47.108 | 62.334 | 47.598 | - | - | - | - | - |
| MintRec2.0 | MULT | Multi | 59.48 | 59.332 | 60.036 | 53.9 | 54.908 | 54.148 | 56.074 | 46.448 | 62.928 | 46.98 | - | - | - | - | - |
| MintRec2.0 | MAG_BERT | Single | 60.384 | 59.61 | 59.996 | 54.738 | 57.454 | 54.538 | 45.738 | 46.266 | 39.27 | 46.04 | - | - | - | - | - |
| MintRec2.0 | MULT | Single | 60.66 | 59.546 | 60.122 | 54.118 | 58.016 | 53.768 | 46.144 | 45.646 | 38.572 | 45.416 | - | - | - | - | - |
| MintRec2.0 | MMIM | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MintRec2.0 | SDIF | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MintRec2.0 | TCL_MAP | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MELD-DA | MAG_BERT | Multi | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MELD-DA | MULT | Multi | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MELD-DA | MAG_BERT | Single | 64.808 | 63.286 | 62.994 | 54.458 | 57.738 | 53.704 | - | - | - | - | 68.654 | 84.86 | 45.516 | 77.072 | 59.256 |
| MELD-DA | MULT | Single | 63.35 | 62.706 | 62.928 | 54.79 | 59.856 | 54 | - | - | - | - | 66.622 | 84.33 | 37.6 | 85.106 | 65.298 |
| MELD-DA | MMIM | Single | 62.208 | 60.836 | 61.378 | 53.166 | 54.208 | 54.522 | - | - | - | - | 71.704 | 86.91 | 47.63 | 77.918 | 59.858 |
| MELD-DA | SDIF | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| MELD-DA | TCL_MAP | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| IEMOCAP-DA | MAG_BERT | Multi | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| IEMOCAP-DA | MULT | Multi | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| IEMOCAP-DA | MAG_BERT | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| IEMOCAP-DA | MULT | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| IEMOCAP-DA | MMIM | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| IEMOCAP-DA | SDIF | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| IEMOCAP-DA | TCL_MAP | Single | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
For unsupervised models:
| Dataset | Model Name | Dialogue Type | NMI | ARI | ACC | fmi |
|---|---|---|---|---|---|---|
| MintRec | MCN | Single | 19.892 | 2.33 | 17.978 | 9.408 |
| MintRec | CC | Single | 47.142 | 22.206 | 42.024 | 27.006 |
| MintRec | sccl | Single | 45.04 | 14.718 | 37.394 | 24.098 |
| MintRec | USNID | Single | - | - | - | - |
| MintRec | UMC | Single | - | - | - | - |
| MintRec2.0 | MCN | Single | - | - | - | - |
| MintRec2.0 | CC | Single | - | - | - | - |
| MintRec2.0 | sccl | Single | - | - | - | - |
| MintRec2.0 | USNID | Single | - | - | - | - |
| MintRec2.0 | UMC | Single | - | - | - | - |
| MELD-DA | MCN | Single | - | - | - | - |
| MELD-DA | CC | Single | - | - | - | - |
| MELD-DA | sccl | Single | - | - | - | - |
| MELD-DA | USNID | Single | - | - | - | - |
| MELD-DA | UMC | Single | - | - | - | - |
| IEMOCAP-DA | MCN | Single | - | - | - | - |
| IEMOCAP-DA | CC | Single | - | - | - | - |
| IEMOCAP-DA | sccl | Single | - | - | - | - |
| IEMOCAP-DA | USNID | Single | - | - | - | - |
| IEMOCAP-DA | UMC | Single | - | - | - | - |