De-Duplication of DAT Files appears to only use size for comparisons.

[wormley]

Paste the command

igir report -vvv --dat tosec-dat/TOSEC-PIX/ --input TOSEC-PIX-ROM/

Describe the bug

When loading a large number of DAT files where some have the same file-size then all but one of the same size DATs appear to be ignored.

[22:02:16.418] TRACE: DATScanner: found 1,132 DAT files
[22:02:29.790] TRACE: DATScanner: parsing 1,025 DAT files

ls tosec-dat/TOSEC-PIX/.dat | wc -l
1132
ls -l tosec-dat/TOSEC-PIX/.dat | awk '{print $5}' | sort | uniq -c | wc -l
1025
(Number of unique file sizes)

Expected behavior

All 1132 files should be parsed.

Debug logs

igir.trace.txt.gz

DAT(s) used

https://www.tosecdev.org/downloads/category/58-2024-05-17

igir version

v3.0.1

Node.js version

N/A

Operating system

Debian 12.5

Additional context

No response