1:1人脸相似度准确率非常低

[chenqinggang001]

我用一些用户数据测试结果，准确率较低，有很多相似度低于0.5的，android 端模型是Megatron

[chenqinggang001]

这些用户数据在我们自己的云服务上都是0.9以上的相似度，甚至部分数据出现负值或者检测不到人脸，有什么办法可以增强识别的准确率吗？
我的代码:

void compareFaceImages(List<FaceImgData> ls) {
        String folder = InspireFace.copyResourceFileToApplicationDir(this);
        boolean launchStatus = InspireFace.GlobalLaunch(folder + "/Megatron");
        if (!launchStatus) {
            Log.e(TAG, "Failed to launch InspireFace");
            return;
        }
        CustomParameter parameter = InspireFace.CreateCustomParameter()
                .enableRecognition(true)
                .enableFaceQuality(true) 
                .enableFaceAttribute(true)
                .enableInteractionLiveness(true) 
                .enableLiveness(true) 
                .enableMaskDetect(true);
        Session session = InspireFace.CreateSession(parameter, InspireFace.DETECT_MODE_ALWAYS_DETECT, 1, -1, -1);
        for (int i = 0; i < ls.size(); i++) {
            FaceImgData faceImgData = ls.get(i);
            Bitmap originalFace = getBitmapFromURL(faceImgData.getOriginalFaceUrl());
            Bitmap liveFace = getBitmapFromURL(faceImgData.getLiveFaceUrl());
            ImageStream stream1 = InspireFace.CreateImageStreamFromBitmap(originalFace, InspireFace.CAMERA_ROTATION_0);
            MultipleFaceData multipleFaceData1 = InspireFace.ExecuteFaceTrack(session, stream1);
            if (multipleFaceData1.detectedNum > 0) {
                FaceFeature feature1 = InspireFace.ExtractFaceFeature(session, stream1, multipleFaceData1.tokens[0]);
                ImageStream stream2 = InspireFace.CreateImageStreamFromBitmap(liveFace, InspireFace.CAMERA_ROTATION_0);
                MultipleFaceData multipleFaceData2 = InspireFace.ExecuteFaceTrack(session, stream2);
                if (multipleFaceData2.detectedNum > 0) {
                    FaceFeature feature2 = InspireFace.ExtractFaceFeature(session, stream2, multipleFaceData2.tokens[0]);
                    float similarity = InspireFace.FaceComparison(feature1, feature2);
                    if (similarity < 0.5) {
                        Log.e(TAG, "id: " + faceImgData.getId() + " faceSimilarity: " + similarity);
                    } else  {
                        Log.i(TAG, "id: " + faceImgData.getId() + " faceSimilarity: " + similarity);
                    }
                } else {
                    Log.i(TAG, "id: " + faceImgData.getId() + " faceSimilarity1: " + 0.0);
                }
                InspireFace.ReleaseImageStream(stream2);
            } else {
                Log.i(TAG, "id: " + faceImgData.getId() + " faceSimilarity2: " + 0.0);
            }
            InspireFace.ReleaseImageStream(stream1);
        }
        InspireFace.ReleaseSession(session);
        InspireFace.FeatureHubDataDisable();
    }

[tunmx]

这个相似度的比对结果是Cosine距离（它并不是以百分比概率来分布的），可能比较容易让人产生误会。拿仓库里面的Pikachu模型来举例，通过来说人脸特征1:1的结果值大于0.48以上基本可以判断相似度较高，而Megatron模型可以把阈值定得低一点，这些值是在训练过程中产生的，不同的数据分布、训练条件所产生的特征模型会有差异。你可以准备一批人脸比对数据集进行测试统计，找出适合你业务场景的Cosine阈值，也可以使用一些非线性的激活函数如SIGMOD通过设置阈值来把Cosine对比结果变换到百分比区间。

[tunmx]

Hi @chenqinggang001 ,

关于你的问题，我们在最新版本(1.1.12)中添加了对人脸比对结果的一些接口或许可以帮助你解决一些问题；

1. 获取模型中的推荐阈值

/**
 * @brief Get recommended cosine threshold from loaded resource.
 *  Use it to determine face similarity. Note: it's just a reference and may not be optimal for your task.
 * @return HResult indicating the success or failure of the operation.
 */
HYPER_CAPI_EXPORT extern HResult HFGetRecommendedCosineThreshold(HPFloat threshold);

2. 把Cosine相似度转换成百分比

/**
 * @brief Convert cosine similarity to percentage similarity.
 *  This is a nonlinear transformation function. You can adjust curve parameters to map the similarity distribution you need.
 * @note The conversion parameters are primarily read from the Resource file configuration, as different models
 *       have different conversion parameters. The parameters provided in the Resource file are only reference
 *       values. If they do not meet your specific use case requirements, you can implement your own conversion
 *       function.
 * @param similarity The cosine similarity score.
 * @param result Pointer to the floating-point value where the percentage similarity will be stored.
 * @return HResult indicating the success or failure of the operation.
 */
HYPER_CAPI_EXPORT extern HResult HFCosineSimilarityConvertToPercentage(HFloat similarity, HPFloat result);

3. 修改转换器的参数

/**
 * @brief Similarity converter configuration.
 */
typedef struct HFSimilarityConverterConfig {
    HFloat threshold;    ///< If you think that the threshold for judging the same person using cosine is some value such as 0.42,
                         // you need to convert him to a percentage of 0.6(pass), you can modify it.
    HFloat middleScore;  ///< Cosine threshold converted to a percentage reference value,
                         // usually set 0.6 or 0.5, greater than it indicates similar, pass
    HFloat steepness;    ///< Steepness of the curve, usually set 8.0
    HFloat outputMin;    ///< Minimum value of output range, usually set 0.01
    HFloat outputMax;    ///< Maximum value of output range, usually set 1.0
} HFSimilarityConverterConfig, *PHFSimilarityConverterConfig;

/**
 * @brief Update the similarity converter configuration.
 * @note The default configuration is loaded from the resource file during initialization.
 *       This function allows you to override those default settings if needed.
 * @param config The new similarity converter configuration to apply.
 * @return HResult indicating the success or failure of the operation.
 */
HYPER_CAPI_EXPORT extern HResult HFUpdateCosineSimilarityConverter(HFSimilarityConverterConfig config);

转换得分的过程是非线性，以Megatron模型为例，你可以尝试使用我们提供的默认值0.32（如下图），也可以自行调节曲线参数：

从图上看，经过处理，如果把Megatron模型的Cosine及格线定为0.32映射到百分比的0.6，那0.5的cosine相似度相当于0.9的百分比相似度，以此类推，你可以根据自己的业务场景来调节参数

Example:

    // Run comparison
    HFloat similarity;
    ret = HFFaceComparison(feature1, feature2, &similarity);
    if (ret != HSUCCEED) {
        HFLogPrint(HF_LOG_ERROR, "Feature comparison error: %d", ret);
        return ret;
    }

    HFloat recommended_cosine_threshold;
    ret = HFGetRecommendedCosineThreshold(&recommended_cosine_threshold);
    if (ret != HSUCCEED) {
        HFLogPrint(HF_LOG_ERROR, "Get recommended cosine threshold error: %d", ret);
        return ret;
    }

    if (similarity > recommended_cosine_threshold) {
        HFLogPrint(HF_LOG_INFO, "%.3f > %.3f ✓ Same face", similarity, recommended_cosine_threshold);
    } else {
        HFLogPrint(HF_LOG_WARN, "%.3f < %.3f ✗ Different face", similarity, recommended_cosine_threshold);
    }
    HFLogPrint(HF_LOG_INFO, "Similarity score: %.3f", similarity);

    // Convert cosine similarity to percentage similarity.
    // Note: conversion parameters are not optimal and should be adjusted based on your specific use case.
    HFloat percentage;
    ret = HFCosineSimilarityConvertToPercentage(similarity, &percentage);
    if (ret != HSUCCEED) {
        HFLogPrint(HF_LOG_ERROR, "Convert similarity to percentage error: %d", ret);
        return ret;
    }
    HFLogPrint(HF_LOG_INFO, "Percentage similarity: %f", percentage);

Output:

0.683 > 0.320 ✓ Same face
Similarity score: 0.683
Percentage similarity: 0.982451

如果你想使用它，请获取最新的代码和最新的模型文件

[chenqinggang001]

修改后，测试结果，相比于云服务的REST API 计算的准确率在 80% 左右，这个是我自己应用的数据的测试结果

[chenqinggang001]

旋转后的图片（对比图片不旋转，拍摄图片旋转）识别率很低

[tunmx]

旋转的图像有旋转回正脸角度或者使用ImageStream参数设置对应角度进行调整再比对吗

[chenqinggang001]

旋转的图像有旋转回正脸角度或者使用ImageStream参数设置对应角度进行调整再比对吗

旋转后识别正常，这个其实可以通过业务层的代码控制一下旋转角度，问题不大

[chenqinggang001]

再请教一个问题，为啥0.32是一个可信的值，把Cosine相似度定到0.32有什么参考吗 @tunmx

[tunmx]

0.32只是我用一批测试集数据集跑出来大致的结果，对于你的业务并不是最可信的，你可以根据你的场景调整它