How to deal with false positives in the vulnerabilities check?

[HubTou]

<TL;DR>
Use the vuxml tool with the vulnerability IDs reported to check if you are facing a real match or a false positive.
One day I'll provide a mechanism to signal false-positives...
</TL;DR>

Let's take the example of [email protected] who helped me a lot with the FreeBSD port:

$ ptl -m eduardo
[...]
ptl: WARNING: Found VuXML vulnerability 'bd579366-5290-11d9-ac20-00065be4b5b6' for port greed-game-4.3
ptl: WARNING: Found VuXML vulnerability '92442c4b-6f4a-11db-bd28-0012f06707f0' for port imlib2-1.12.2_1,2
ptl: WARNING: Found VuXML vulnerability '10dee731-c069-11ee-9190-84a93843eb75' for port R-cran-openssl-2.1.1
[... skipping 35 other R-cran-openssl vulnerabilities ...]

Issues per maintainer:
  [email protected]:
    [...]
    Vulnerable port:
      greed-game-4.3 imlib2-1.12.2_1,2 R-cran-openssl-2.1.1

Selected 217 ports out of 34416 in the FreeBSD port tree, and found:
  [...]
  3 ports with a vulnerable version (warning)

Using the reported vulnerabilities IDs with my vuxml tool (that gets installed as a portstreelint dependency), we see:

$ vuxml -i 92442c4b-6f4a-11db-bd28-0012f06707f0
Vulnerability ID: 92442c4b-6f4a-11db-bd28-0012f06707f0
  Topic: Imlib2 -- multiple image file processing vulnerabilities
  Affects:
    imlib2:
      < 20060926_1,1 ;
  References:
    bid: 20903
    cvename: CVE-2006-4806
    cvename: CVE-2006-4807
    cvename: CVE-2006-4808
    cvename: CVE-2006-4809
  Discovery date: 2006-11-03
  Entry date: 2006-11-08

1 vulnerability found

So this one is probably a false positive because we see that the version scheme of the imlib2 tool is inconsistent, and that version 1.12.2_1,2 seems to be lower than version 20060926_1,1.

For the greed-game vulnerability, it's another story:

vuxml -i bd579366-5290-11d9-ac20-00065be4b5b6
Vulnerability ID: bd579366-5290-11d9-ac20-00065be4b5b6
  Topic: greed -- insecure GRX file processing
  Affects:
    greed:
      <= 0.81p ;
  References:
    cvename: CVE-2004-1273
    cvename: CVE-2004-1274
    url: http://tigger.uic.edu/~jlongs2/holes/greed.txt
    url: http://secunia.com/advisories/13534/
    mlist: http://marc.theaimsgroup.com/?l=bugtraq&m=110321888413132
  Discovery date: 2004-12-15
  Entry date: 2005-01-03
  Modified date: 2005-01-13

Version 4.3 seems to be higher than version 0.81p, unfortunately the library I use for versions comparisons doesn't know how to handle letters in versions (one day I'll write my own!). This can be confirmed using the --debug option of portstreelint:

$ ptl -m eduardo --debug
[...]
ptl: DEBUG: portname='greed' version='4.3' assumed for port greed-game-4.3
ptl: DEBUG: Invalid version '0.81p' (translated as '0.81p') for affected package 'greed'
ptl: WARNING: Found VuXML vulnerability 'bd579366-5290-11d9-ac20-00065be4b5b6' for port greed-game-4.3
[...]

We see that the greed-game port has an assumed PORTNAME=greed (maybe a bug in portstreelint!) and that portname is associated with a vulnerability.

Unfortunately, as the version number is "invalid" with my versions comparison library, portstreelint can't be sure this version is corrected and thus report it as vulnerable...

For the third one, it's the same thing than for the greed-game:

$ ptl -m eduardo --debug
[...]
ptl: DEBUG: portname='openssl' version='2.1.1' assumed for port R-cran-openssl-2.1.1
ptl: DEBUG: Invalid version '1.1.1u_1,1' (translated as '1.1.1u.1') for affected package 'openssl'
ptl: DEBUG: Invalid version '1.1.1u,1' (translated as '1.1.1u') for affected package 'openssl'
ptl: DEBUG: Invalid version '1.1.1t,1_2' (translated as '1.1.1t') for affected package 'openssl'
ptl: DEBUG: Invalid version '1.1.1t,1_1' (translated as '1.1.1t') for affected package 'openssl'
ptl: DEBUG: Invalid version '1.1.1t,1' (translated as '1.1.1t') for affected package 'openssl'
[...]

Again the version scheme prevents an effective comparison.

For an upcoming version, I plan to add a user configuration file, with the ability to declare vulnerability IDs to ignore, in order to avoid bugging a port maintainer with false positives. In the meantime, I think it's better to signal them so the maintainer can check if they are relevant...

[HubTou]

Version 1.3.0 now provides a way to discard false-positive vulnerabilities.

Start by generating a default configuration file:

$ ptlint --gencfg $HOME/.ptlint

Modify the file section [exclusions]/[vulnerabilities] to add all vulnerability IDs that you want to exclude from your results.

For example :

[exclusions]
# (multilines) lists of space separated Vulnerabilities IDs:
vulnerabilities = 92442c4b-6f4a-11db-bd28-0012f06707f0
    bd579366-5290-11d9-ac20-00065be4b5b6

And voilà! Bye bye false-positives :-)

I also investigated the greed-game case described above:

We see that the greed-game port has an assumed PORTNAME=greed (maybe a bug in portstreelint!) and that portname is associated with a vulnerability.

This doesn't seem to be a bug, at least in portstreelint.

That port sits in games/greed and has a PORTNAME=greed line in its Makefile but an INDEX entry as greed-game, which is strange...

VuXML references a vulnerability in something called greed but which is an entirely different software (Get and Resume Elite EDition).
It's not clear if that other software was part of the ports collection in the past, but if not it shouldn't be in VuXML at all...