Add ddr experiment

[frcroth]

Checklist

• I have read the contribution guidelines
• reference issue for this pull request: New experiment request: DDR queries probe#2819
• if you changed anything related to how experiments work and you need to reflect these changes in the ooni/spec repository, please link to the related ooni/spec pull request: Add DDR experiment spec spec#300
• if you changed code inside an experiment, make sure you bump its version number

Description

As described here: ooni/probe#2819, I would be interested to run experiments of the DDR protocol (RFC 9462). This experiment uses the system resolver, as I am interested in whether the local system resolvers support this protocol (we have already conducted tests using zmap from one vantage point).

To get the system resolver I am checking /etc/resolve, it should also be possible by querying using the default DNS provider and checking the answer or config, however I did not get this to work.

I have no idea about what the spec should look like. Right now I just put the DDR responses as JSON objects into the testkeys. If that is not supported because it could violate the spec, it would also be possible to just insert the string / do some parsing here to get some more useful information (e.g. if DoT or DoH is supported, what the target is...)

TODOs

• Testing
• PR for spec

[hellais]

Hi, thanks for proposing this experiment and for working on an implementation.

In general I think it would be great to have a specification written for it in order to assist reviewing the implementation better. Some particularly sensitive areas which I would like to ensure are covered in the specification are if there are any privacy implications for users running this test.

For example, is there a risk that when performing the lookup for the designated resolvers, the list will include information that uniquely identifies a user? Has this been observed or is it possible as per the spec?

Regarding the specific implementation I see in this PR, the biggest issue is that it doesn't extend the DNS test specification (see: https://github.com/ooni/spec/blob/master/data-formats/df-002-dnst.md#dns-data-format). Ideally we would extend that to support encoding SVCB records and we might even opt for offloading the job of parsing them to the data pipeline instead of doing it inline in the probe.

Finally, I wonder if instead of creating a new specialized test for DDR, we could instead work on adding support for this to our dnscheck test or if not have a more general purpose DNS lookup test that allows us to run any query we might need.

This would also help us address the issue we had with needing to ship DNS queries as web_connectivity tests where it would have instead been cleaner to do it as part of a DNS specific test (see: citizenlab/test-lists#1883).

In summary the next steps to move this PR forward are:

• Create PR updating the dns base test specification to support SVCB records
• Document somewhere the eventual privacy considerations associated with performing DDR queries
• Evaluate if it's possible to extend DNSCheck to do SVCB queries or if we should instead create a new simplified general purpose DNS experiement (@DecFox can you help with this?)
• Write spec for such test or update dnscheck spec to mention svcb record support
• Implement code for it

[frcroth]

Hi, thanks for proposing this experiment and for working on an implementation.

In general I think it would be great to have a specification written for it in order to assist reviewing the implementation better. Some particularly sensitive areas which I would like to ensure are covered in the specification are if there are any privacy implications for users running this test.

For example, is there a risk that when performing the lookup for the designated resolvers, the list will include information that uniquely identifies a user? Has this been observed or is it possible as per the spec?

The result would include information about the user's recursive resolver, including IP address and the DDR configuration, which could be used to uniquely identify this recursive resolver.
Since DDR is pretty new, there is basically no literature, so there has been nothing on identifying users afaik.

Here is a description of what the experiment does as requested, sorry for that: ooni/spec#300

Regarding the specific implementation I see in this PR, the biggest issue is that it doesn't extend the DNS test specification (see: ooni/spec@master/data-formats/df-002-dnst.md#dns-data-format). Ideally we would extend that to support encoding SVCB records and we might even opt for offloading the job of parsing them to the data pipeline instead of doing it inline in the probe.

Finally, I wonder if instead of creating a new specialized test for DDR, we could instead work on adding support for this to our dnscheck test or if not have a more general purpose DNS lookup test that allows us to run any query we might need.

This would also help us address the issue we had with needing to ship DNS queries as web_connectivity tests where it would have instead been cleaner to do it as part of a DNS specific test (see: citizenlab/test-lists#1883).

From what I have seen, the dnscheck experiment queries a specified resolver, while this experiment would ideally always query the configured recursive resolver, because that is what I really know, how the configured resolver e.g. by the ISP supports DDR and advertises encrypted resolvers or not. Generally most of the DNS stuff I have seen here assumes A/AAAA, also e.g. test key types and such (e.g. ArchivalDNSAnswer), so reuse is difficult. It would be possible to create a more abstract DNS implementation, right now that might be overkill when every other use case seems to be A/AAAA.

[hellais]

I left a comment in the PR for the spec here: https://github.com/ooni/spec/pull/300/files#r1935814538