Make root hints configurable, take 2

[mattias-p]

Purpose

In current code the root hints are hard coded. This PR opens for use cases where private root must be used, see #850. The driving use case for this PR is the possibility to use it for unit tests.

Context

Fixes #850.

This PR is based on and replaces #1132. The main difference is in the approach to APIs.

This PR leverages the existing fake addresses API for programmatic configurability, parses IANA root hints files and leaves application user interface design up to the application.

This PR only extends Engine's programmatic interface. Applications need to be adapted to make use of this new feature. E.g. zonemaster-cli and zonemaster-backend. Those interface changes are made separately.

Changes

• The fake addresses API in Zonemaster::Engine::Recursor is extended in two ways:
  1. A new remove_fake_addresses() method is added, and
  2. instead of ignoring fake addresses for the root zone, they are used as the root name server addresses.
• A parser for the IANA root hints file format is added. It was designed for convenient use with add_fake_addresses().
• The way we store root name server addresses is changed. Instead of keeping them in a custom JSON blob inlined into a Perl module file, they are now stored in a separate named.root file taken directly from IANA.

How to test this PR

1. Set up a name server with a custom root zone.
2. Use a version of zonemaster-cli that overrides the root hints like the examples in the documentation of Zonemaster::Engine::Recursor->root_servers.
   • E.g. you could use Add --hints option zonemaster-cli#284 and provide a hints file for your own root zone.
3. Start tcpdump and watch for any traffic to the IANA root name servers. (There shouldn't be any.)
4. Run zonemaster-cli on a zone under your custom root.

It would be super nice to automate this test but that is out of scope for this PR.

[matsduf]

This looks like a good suggestion. It is better to use normal hint file format. To utilize this CLI and Backend need to be updated.

[marc-vanderwal]

We could check for the error message in the unit test if the presence of forbidden directives is checked before parsing the file.

[mattias-p]

I agree. That's better.

[mattias-p]

I converted this PR back to draft status because I plan to update the test instruction.

[mattias-p]

@mvw-afnic I've made updates for your comments. Do you want to review again?

[marc-vanderwal]

Hi @mattias-p , I saw you replied to my comments but I don’t see any changes since my first review. Did you push to your branch?

[marc-vanderwal]

Looks good to me.

[ghost]

v2022.2 release testing

This works as expected.

However when it comes to deal with wrong files, I encountered several behaviors:

1. die with only the error message, for instance: Error loading hints file: Forbidden directive $TTL

   % cat custom.root
   $TTL 40
   

2. die with context: unable to parse RR string at /usr/local/share/perl5/Net/DNS/ZoneFile.pm line 534.[...]

   % cat custom.root
   nonsensedata
   

3. don't die at all:

   % cat custom.root
   .                        3600000      NS    A.ROOT-SERVERS.NET.
   A.ROOT-SERVERS.NET.      3600000      A
   A.ROOT-SERVERS.NET.      3600000      AAAA  192.11
   A.ROOT-SERVERS.NET.      3600000      AAAA  hello
   

[matsduf]

There are sever limitations in the error handling of Net::DNS. It is known. Another alternative would be to write our own verification script.