Implement fpcalc command line tool

[Holzhaus]

I need this crate to generate fpcalc compatible fingerprints (for AcoustID). Since this project already has a placeholder fpcalc, I decided to work on the actual implementation.

I rewrote the compression code since the existing one was incomplete and I already had some code written indepently.

Important: For some reason the fingerprints are not the same, so this is not ready to merge yet. Unsure if this is a bug in the compression code, the fingerprinter code or in the fpcalc implementation.

[Holzhaus]

Ok, the compression code definitely works, I tested it using the output from the C++ implementation, like this:

#include <cstring>
#include <cstdio>
#include "pack_int5_array.h"

int main() {
    const unsigned char data[] = {
        0xA2, 0x87, 0xE3, 0xED, 0xAA, 0xD7, 0xE8, 0x94, 0x53, 0x4E, 0x9B, 0xD5, 0x83, 0x12, 0x05, 0x43,
        0x67, 0x7E, 0x0A, 0xAF, 0x2D, 0x85, 0xB4, 0x03, 0xEB, 0x13, 0x8E, 0x47, 0x07, 0xA6, 0x76, 0x5D,
        0x43, 0x67, 0x8D, 0x9F, 0xEA, 0xAD, 0x3F, 0x34, 0x86, 0xF4, 0x25, 0xC8, 0xA2, 0xBF, 0xF1, 0x22,
        0xB5, 0xA6, 0xB8, 0x4A, 0xED, 0xA2, 0xF5, 0x25, 0xDB, 0x62, 0x70, 0xC2, 0xB7, 0x9C, 0xB1, 0x3C,
    };
    size_t data_size = 64;

    size_t output_size = chromaprint::GetPackedInt5ArraySize(data_size);
    unsigned char* output = (unsigned char*)malloc(output_size);
    chromaprint::PackInt5Array(data, data + data_size, output);
    for (size_t i=0; i < output_size; i++) {
        printf("%02x ", output[i]);
    }
}

This will output:

e2 8c a6 2e a2 d3 ed 3a 64 19 c7 ab d7 0a 1d 6b ba 73 8c ed e3 b4 af da a7 86 16 24 7e 14 d5 60 d5 44 2d 5b 40 71 79 e4

I added that to the unittests.

That means that either the fpcalc code is the culprit, or there is a bug/misconfiguration in the chromaprint library. I suspect the former, but I have no idea where it is.

[Holzhaus]

Okay, there actually was a bug in the compression code. The missed that the first bit index is 1, not 0. Fixed now.

Added a test based on the actual compression output from:

#include <cstdint>
#include <cstring>
#include <cstdio>
#include <vector>
#include "fingerprint_compressor.h"

int main() {
    std::vector<uint32_t> data = {
        0x0FCAF446, 0xE3519E89, 0xD3494DD6, 0x8F219806,
        0x9200D530, 0x06B1D52F, 0xB48CC681, 0x428991C3,
        0x59AFBD6B, 0x6ECFB2E5, 0xE8EB7BC3, 0x99A44270,
        0x31FFEC13, 0x4A4D81DA, 0x53887C82, 0x2BB7BEC2,
        0xAB895A65, 0x9D7C0AE4, 0xDA356857, 0xE030F7D8,
        0x4D428EEE, 0x0558E019, 0xC3278998, 0xA1D035E4,
        0x582E98E5, 0x44C8B708, 0x2E8BA9E2, 0xCB13BC48,
        0xB169A3D8, 0x861274AF, 0x1213EF1C, 0x1F9F06B8,
    };
    auto output = std::string();
    auto compressor = chromaprint::FingerprintCompressor();
    compressor.Compress(data, 1, output);

    unsigned char* out = output.c_str();
    for (size_t i=0; i < output.size(); i++) {
        printf("%02x ", out[i]);
    }
}

[Holzhaus]

The output is now almost the same, but not exactly the same. No idea what causes this.

[Holzhaus]

Anyway, I suppose this is ready for review.

[darksv]

The output is now almost the same, but not exactly the same. No idea what causes this.

You mean there is a difference in the output from the fingeprint? I think it can be due to different resampling method or something. I'll have to investigate.

Overall, code looks good and I think we can merge it, or at least just the compression part. I'll compare the fpcalc code with the original implementation when I find some time.

[Holzhaus]

You mean there is a difference in the output from the fingeprint? I think it can be due to different resampling method or something. I'll have to investigate.

👍

Overall, code looks good and I think we can merge it, or at least just the compression part.

Great, feel free to hit merge ;-)

[darksv]

I did some digging, and it turns out there was a minor divergence in the FFT calculation that I fixed in #10.

After that, I calculated the fingerprints of some uncompressed .wav files, and the fingerprints were identical across the implementations. However, when I used .mp3 files, there were some differences, likely due to the impact of audio decompression on the algorithm. So far, I've only tested with resampling disabled, which probably has an influence as well. I'm not sure yet how these differences will affect matching quality.

For now, I'll go ahead and merge this PR.