• Contenu
• DNS récursif Ouvert DoH DoT & haute disponibilité
  • Configurations communes à pan1 et pan2
    • Nginx
    • Unbound - 20260802
    • NFTables - 20260802
    • Sysctl - 20260802
    • AppArmor
  • Configurations spécifiques
    • Génération du certificat SSL sur pan1
    • Copie du certificat sur gaia
    • sha1 du certif pour la page web https://dns.aquilenet.fr
    • Clée sdns (auth par épinglage de la clé / key pining)
    • Keepalived - 20260802
      • pan1 Master
      • pan2 Slave
      • /etc/keepalived/keepalived-master.sh
    • Télémétrie
      • Exporter Prometheus @(pan1|pan2)
      • Prometheus @seraphin
  • Exploitation
  • Indexes de serveurs DNS ouverts:
  • Ressources

DNS récursif Ouvert DoH DoT & haute disponibilité¶

45.67.81.23 - dns.aquilenet.fr

On propose un DNS récursif ouvert cf https://www.aquilenet.fr/services/dns/ https://dns.aquilenet.fr/
On a mis aussi un DNS ouvert sur l'IP 45.67.81.23 car facile à retenir, avec l'adresse de dns.aquilenet.fr (voir http://dns.aquilenet.fr) . Nous en avons profité pour ajouter DoH et DoT et d'avoir cette adresse en IP Virtuelle partagée entre pan1 et pan2.
Cette IP virtuelle est portée par Keepalived sur pan1 le primaire et pan2 le secondaire. Si le démon Keepalived est arrêté sur pan1, l'ip bascule sur pan2. Si on redémarre le démon sur pan1, ce dernier reprend l'IP: pratique pour les mises à jour, le service continue à fonctionner.

Pan[1|2] portent l'ip 45.67.81.23 gracieusement offerte par les copains de https://yaal.coop/fr un vlan est dédié sur ce réseau de l'hyperviseur.
Ces VM on 2 vCPU et 2Go de RAM

Télémétrie: https://grafana.aquilenet.fr/d/9FQf4fEWz/unbound

Configurations communes à pan1 et pan2¶

Nginx¶

• /etc/nginx/sites-enabled/default

server {
    listen       45.67.81.23:80 ;
    listen       [2a0c:e300::1337]:80 ;

    location ~ /.well-known/acme-challenge {
        allow all;
        root /var/www/letsencrypt;
    }
    location / {
        return 301 https://dns.aquilenet.fr;
        }

    server_name dns.aquilenet.fr; # managed by Certbot

    location ~ /.well-known/acme-challenge {
        allow all;
        root /var/www/letsencrypt;
    }
}
server {
    listen       45.67.81.23:443 ssl http2;
    listen       [2a0c:e300::1337]:443 ssl http2;
    root /var/www/dns.aquilenet.fr;
    index index.htm;

    ssl_certificate     /etc/unbound/ssl/dns.aquilenet.fr/fullchain.pem;
    ssl_certificate_key /etc/unbound/ssl/dns.aquilenet.fr/privkey.pem;
location /dns-query {
    grpc_pass grpc://127.0.0.1:443;
        }
}

Unbound - 20260802¶

/!\ cookie-secret n'est pas celui utilisé

include-toplevel: "/etc/unbound/unbound.conf.d/*.conf"

server:
    verbosity: 1 
    num-threads: 2
    msg-cache-size: 256m
    rrset-cache-size: 512m
    key-cache-size: 64m
    neg-cache-size: 64m
    outgoing-range: 16384 
    num-queries-per-thread: 16384 
    jostle-timeout: 3000 
    serve-expired: yes
    serve-expired-ttl: 86400
    serve-expired-reply-ttl: 30
    infra-cache-numhosts: 30000
    infra-cache-lame-size: 50000
    infra-keep-probing: yes
    infra-host-ttl: 60

    ip-ratelimit: 200
    ip-ratelimit-factor: 10 
    ip-ratelimit-size: 4m
    ip-ratelimit-slabs: 2 
    outgoing-num-tcp: 256 
    incoming-num-tcp: 256

    ratelimit: 1000
    ratelimit-factor: 10 
    ratelimit-size: 4m
    ratelimit-slabs: 2 

    msg-cache-slabs: 2 
    rrset-cache-slabs: 2
    infra-cache-slabs: 2
    key-cache-slabs: 2

    answer-cookie: yes
    cookie-secret: "5fe5dce2beaea78361c95ac5f3f9416d"
    ip-ratelimit-cookie: 100

    access-control: 0.0.0.0/0 allow
    access-control: ::/0 allow

    interface: 45.67.81.23
    interface: 45.67.81.23@853
    interface: 2a0c:e300::1337
    interface: 2a0c:e300::1337@853
#    interface: 45.67.81.21 # Pour Nagios

### Marche pas avec la vip keepalived
#    interface: 0.0.0.0
#    interface: 0.0.0.0@853
#    interface: ::0
#    interface: ::0@853
### interface-automatic: yes 
# l'écoute en 127.0.0.1 ne fonctionne plus avec

    http-notls-downstream: yes
    interface: 127.0.0.1@443
    interface: ::1@443

    port: 53
    tls-port: 853
    https-port: 443

    root-hints: "/usr/share/dns/root.hints"

    aggressive-nsec: yes
    harden-glue: yes
    harden-large-queries: yes
    harden-dnssec-stripped: yes
    harden-below-nxdomain: yes
    harden-referral-path: yes
    harden-unknown-additional: yes
    harden-algo-downgrade: yes
    harden-short-bufsize: yes
    use-caps-for-id: no # makes lots of queries fail
    edns-buffer-size: 1232
    max-udp-size: 1232
    rrset-roundrobin: yes
    cache-min-ttl: 300
    cache-max-ttl: 86400
    serve-expired: yes
    hide-identity: yes
    identity: "dns.aquilenet.fr"
    hide-version: yes
    do-daemonize: no
    deny-any: yes
    minimal-responses: yes
    prefetch: yes
    prefetch-key: yes
    so-reuseport: yes 
    so-rcvbuf: 32m
    so-sndbuf: 32m
    unwanted-reply-threshold: 5000
    log-queries: no
    log-replies: no
    log-servfail: no
    log-local-actions: no
    logfile: /dev/null

    # Anti DNS-Rebinding
    private-address: 10.0.0.0/8
    private-address: 172.16.0.0/12
    private-address: 192.168.0.0/16
    private-address: 169.254.0.0/16
    private-address: fd00::/8
    private-address: fe80::/10

    # imposer la QNAME minimisation (RFC 7816)
    qname-minimisation: yes
    # même si le serveur faisant autorité ne le veut pas
    #   après discussion, il est possible que cette option ne soit
    #   pas recommandée dans le cadre d'un résolveur ouvert
    qname-minimisation-strict: no 

   # DoH
   tls-service-key: "/etc/unbound/ssl/dns.aquilenet.fr/privkey.pem"
   tls-service-pem: "/etc/unbound/ssl/dns.aquilenet.fr/fullchain.pem"

   module-config: "validator iterator"

NFTables - 20260802¶

• /etc/nftables.conf

#!/usr/sbin/nft -f

flush ruleset

table inet raw {

    chain prerouting {
        type filter hook prerouting priority raw;
        policy accept;

        udp dport 53 udp length < 12 drop
        ip frag-off & 0x1fff != 0 drop
        ip6 nexthdr ipv6-frag drop

        udp dport 53 notrack
        udp sport 53 notrack
        tcp dport 53 notrack
        tcp sport 53 notrack

    }

    chain output {
        type filter hook output priority raw;
        policy accept;

        ip saddr 127.0.0.0/8 ip daddr 127.0.0.0/8 notrack

        udp dport 53 notrack
        udp sport 53 notrack
        tcp dport 53 notrack
        tcp sport 53 notrack 

    }
}

table inet filter {

    chain input {

        type filter hook input priority filter;
        policy drop;

        ct state invalid drop
        ct state established,related accept
        ct state untracked accept

        iifname "lo" accept

        ip saddr 0.0.0.0/8 drop
        ip saddr 10.0.0.0/8 drop
        ip saddr 100.64.0.0/10 drop
        ip saddr 127.0.0.0/8 drop
        ip saddr 169.254.0.0/16 drop
        ip saddr 172.16.0.0/12 drop
        ip saddr 192.168.0.0/16 drop
        ip saddr 198.18.0.0/15 drop
        ip saddr 224.0.0.0/4 drop
        ip saddr 240.0.0.0/4 drop

        ip6 saddr ::1/128 iifname != "lo" drop

        ip protocol icmp accept

        icmpv6 type {
            destination-unreachable,
            packet-too-big,
            time-exceeded,
            parameter-problem,
            echo-request,
            echo-reply,
            nd-router-solicit,
            nd-router-advert,
            nd-neighbor-solicit,
            nd-neighbor-advert
        } accept

        tcp dport 55555 ct state new \
            limit rate 30/minute burst 20 packets accept

        tcp dport 55555 drop

        tcp dport 80 accept
        tcp dport 443 accept
        tcp dport 853 accept

        ip saddr { 185.233.100.230, 91.224.149.127 } tcp dport 5666 accept
        ip saddr 185.233.100.60 tcp dport 9167 accept

        iifname "enX0" \
            ip protocol vrrp \
            ip saddr { 45.67.81.21, 45.67.81.22 } \
            accept

        ip daddr { 45.67.81.21, 45.67.81.22 } udp dport 53 accept
        ip daddr { 45.67.81.21, 45.67.81.22 } tcp dport 53 accept
        ip daddr 45.67.81.23 udp dport 53 accept
        ip daddr 45.67.81.23 tcp dport 53 accept
        ip6 daddr {
            2a0c:e300::1335,
            2a0c:e300::1336,
            2a0c:e300::1337
        } udp dport 53 accept

        ip6 daddr {
            2a0c:e300::1335,
            2a0c:e300::1336,
            2a0c:e300::1337
        } tcp dport 53 accept

        limit rate 5/second log prefix "NFT DROP: " level warn

        drop
    }

    chain forward {
        type filter hook forward priority filter;
        policy drop;
    }

    chain output {
        type filter hook output priority filter;
        policy accept;
    }
}

Sysctl - 20260802¶

• /etc/sysctl.conf

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 2097152
net.core.wmem_default = 2097152

# Autoriser l'écoute sur des IP non encore assignées localement (très utile pour Keepalived / VIP)
net.ipv4.ip_nonlocal_bind = 1
net.ipv6.ip_nonlocal_bind = 1

# Augmenter la file d'attente des paquets reçus par la carte réseau
net.core.netdev_max_backlog = 10000

# Increase connection tracking
net.netfilter.nf_conntrack_max = 1000000

# Increase local port range
net.ipv4.ip_local_port_range = 1024 65535

# Enable TCP fast open
net.ipv4.tcp_fastopen = 3

net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.default.accept_redirects=0
net.ipv6.conf.all.accept_redirects=0
net.ipv4.conf.all.accept_source_route=0

net.netfilter.nf_conntrack_udp_timeout=10
net.netfilter.nf_conntrack_udp_timeout_stream=30
net.netfilter.nf_conntrack_tcp_timeout_time_wait=30

AppArmor¶

• /etc/apparmor.d/usr.sbin.unbound
  décomenter:
  include

• /etc/apparmor.d/local/usr.sbin.unbound

  /etc/letsencrypt/archive/** r,
  /etc/letsencrypt/live/** r,
  

  appliquer avec: apparmor_parser -r /etc/apparmor.d/usr.sbin.unbound

Configurations spécifiques¶

Génération du certificat SSL sur pan1¶

certbot --apache --agree-tos --email sysop@aquilenet.fr -d dns.aquilenet.fr
chgrp bind /etc/letsencrypt/live/dns.aquilenet.fr/privkey.pem
chmod 0640 /etc/letsencrypt/live/dns.aquilenet.fr/privkey.pem
openssl dhparam -out /etc/bind/dhparam.pem 4096

Copie du certificat sur gaia¶

• /etc/letsencrypt/renewal/dns.aquilenet.fr.conf

+ post_hook = /root/certificats.sh

• /root/certificats.sh

#!/bin/sh
cp -RL /etc/letsencrypt/live/dns.aquilenet.fr/ /etc/unbound/ssl/
systemctl restart unbound
systemctl restart nginx
if [ $? -eq 0 ]; then
  # make sure that the configuration is identical on pan2:
  scp -qrP 55555 /etc/nginx/sites-available/default pan2.aquilenet.fr:"/etc/nginx/sites-available/"

  # copy the new certificate to pan2:
  ssh -p 55555 pan2.aquilenet.fr "mkdir -p /etc/letsencrypt/live/dns.aquilenet.fr/"
  scp -qrP 55555 /etc/letsencrypt/live/dns.aquilenet.fr/fullchain.pem /etc/letsencrypt/live/dns.aquilenet.fr/privkey.pem pan2.aquilenet.fr:"/etc/letsencrypt/live/dns.aquilenet.fr/"
  scp -qrP 55555 /etc/letsencrypt/options-ssl-nginx.conf /etc/letsencrypt/ssl-dhparams.pem pan2.aquilenet.fr:"/etc/letsencrypt/"

  scp -qrP 55555 /etc/unbound/ssl/dns.aquilenet.fr pan2.aquilenet.fr:/etc/unbound/ssl/
  ssh -p 55555 pan2.aquilenet.fr "systemctl restart unbound && systemctl restart nginx"
fi

sha1 du certif pour la page web https://dns.aquilenet.fr¶

• /root/sha1.sh

#!/bin/bash

set -e

file="/var/www/dns.aquilenet.fr/index.htm"
host="dns.aquilenet.fr"

get_sha1()
{
    port="$1"

    echo | openssl s_client \
        -servername "$host" \
        -connect "$host:$port" 2>/dev/null \
        | openssl x509 -noout -fingerprint -sha1 \
        | cut -d= -f2 \
        | tr -d '\r'
}

sha1_443=$(get_sha1 443)
sha1_853=$(get_sha1 853)

if [ -z "$sha1_443" ] || [ -z "$sha1_853" ]; then
    echo "Erreur récupération certificat"
    exit 1
fi

sed -i \
    -e "s|^SHA1 Fingerprint 443=.*|SHA1 Fingerprint 443=$sha1_443<br />|" \
    -e "s|^SHA1 Fingerprint 853=.*|SHA1 Fingerprint 853=$sha1_853<br />|" \
    "$file"

echo "Mise à jour OK"
echo "443 : $sha1_443"
echo "853 : $sha1_853"

• /etc/cron.d/sha1

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

0 */12 * * * root /root/sha1.sh

Clée sdns (auth par épinglage de la clé / key pining)¶

• A partir de la clé

pan1|19:21:08|:~# openssl rsa -in /etc/letsencrypt/live/dns.aquilenet.fr/privkey.pem -outform der -pubout | openssl dgst -sha256 -binary | openssl enc -base64
writing RSA key
lBk09CRIPJ+J4O9rmbvJkEiGYoH5r9rxOIQdxkYxyII=

• ou à partir du certificat

pan1|19:22:21|:~# openssl x509 -in /etc/letsencrypt/live/dns.aquilenet.fr/cert.pem -pubkey -noout | openssl rsa -pubin -outform der | openssl dgst -sha256 -binary | openssl enc -base64
writing RSA key
lBk09CRIPJ+J4O9rmbvJkEiGYoH5r9rxOIQdxkYxyII=

Keepalived - 20260802¶

on utilise un hook post bascule de la VIP pour relancer Unbond/Nginx pour qu'ils écoutent bien sur celle-ci.

pan1 Master¶

• /etc/keepalived/keepalived.conf

global_defs {
  notification_email {
    sysop@aquilenet.fr
  }
  notification_email_from vrrp-dns@aquilenet.fr
  smtp_server localhost 
  smtp_connect_timeout 30
}

vrrp_instance DNS {
  state MASTER 
  interface enX0
  virtual_router_id 101
  priority 100 
  advert_int 5 
  smtp_alert
  unicast_src_ip 45.67.81.21 
  unicast_peer {
    45.67.81.22
  }
  virtual_ipaddress {
    45.67.81.23/32 dev enX0 scope global # VIP
  }
  virtual_ipaddress_excluded {
    2a0c:e300::1337/128 dev enX1
    }

notify_master "/etc/keepalived/keepalived-master.sh master"
}

pan2 Slave¶

• /etc/keepalived/keepalived.conf

global_defs {
  notification_email {
    sysop@aquilenet.fr
  }
  notification_email_from vrrp-dns@aquilenet.fr
  smtp_server localhost
  smtp_connect_timeout 30
}

vrrp_instance DNS {
  state SLAVE 
  interface enX0
  virtual_router_id 101 
  priority 50 
  advert_int 5 
  smtp_alert
  unicast_src_ip 45.67.81.22
  unicast_peer {
    45.67.81.21
  }
  virtual_ipaddress {
    45.67.81.23/32 dev enX0 scope global # VIP
  }
  virtual_ipaddress_excluded {
    2a0c:e300::1337/128 dev enX1
    }

notify_master "/etc/keepalived/keepalived-master.sh master"
}

/etc/keepalived/keepalived-master.sh¶

• /etc/keepalived/keepalived-master.sh

#!/bin/sh

case "$1" in
    master)
        systemctl restart unbound
        systemctl restart nginx
        ;;
esac

exit 0

Télémétrie¶

Exporter Prometheus @(pan1|pan2)¶

wget https://github.com/letsencrypt/unbound_exporter/releases/download/v0.6.0/unbound_exporter-v0.6.0.x86_64.deb
apt install ./unbound_exporter-v0.6.0.x86_64.deb

• /etc/systemd/system/unbound_exporter.service

[Unit]
Description=Unbound Prometheus Exporter
After=unbound.service

[Service]
User=unbound
Group=unbound

ExecStart=/usr/bin/unbound_exporter \
    -web.listen-address=:9167 \
    -unbound.host=unix:///run/unbound.ctl \
    -unbound.ca="" \
    -unbound.cert=""

Restart=always
RestartSec=5

NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes

[Install]
WantedBy=multi-user.target

Prometheus @seraphin¶

• /etc/prometheus/prometheus.yml

  - job_name: unbound
    static_configs:
      - targets:
        - pan1.aquilenet.fr:9167
        - pan2.aquilenet.fr:9167

Exploitation¶

• validation du service

  dig +short dns.aquilenet.fr @45.67.81.23 
  45.67.81.23
  dig +https +short dns.aquilenet.fr @45.67.81.23      
  45.67.81.23
  dig +tls +short dns.aquilenet.fr @45.67.81.23      
  45.67.81.23
  

• bascule keepalived

sur pan1: systemctl stop keepalived
sur pan2: ip a |grep /32doit donner "inet 45.67.81.23/32 scope global eth1"
https://dns.aquilenet.fr/hostname.txt permet de savoir quel est le serveur qui porte l'IP virtuelle (il est aussi en bas à droite en noir sur noir sur la page d'accueil ;)

• Avoir des volumétries en cours

watch -n1 'unbound-control stats_noreset | egrep "requestlist.max|requestlist.current|requestlist.exceeded|queries_timed_out|ratelimited"'

• voir les 100 dernières requêtes

unbound-control dump_requestlist | head -100

Indexes de serveurs DNS ouverts:¶

https://github.com/curl/curl/wiki/DNS-over-HTTPS#publicly-available-servers <= fait
https://diyisp.org/dokuwiki/doku.php?id=technical:dnsresolver <= fait
https://european-alternatives.eu/category/public-dns

https://dnsprivacy.org/public_resolvers/

Ressources¶

https://www.bortzmeyer.org/doh-bortzmeyer-fr-policy.html
https://www.bortzmeyer.org/7858.html
https://kb.isc.org/docs/aa-01386
https://dnscrypt.info/stamps-specifications/