Mentiodns » Historique » Révision 58
Révision 57 (sacha, 11/11/2018 10:54) → Révision 58/59 (louis.leveque, 14/02/2019 20:30)
# Mentiodns reCréation de la liste de censure gérée par le ministère de l'intérieur et envoyée automatiquement aux principaux FAI Français. Validation des DNS à partir d'une liste exhaustive par tld. On vérifie si les résolutions DNS envoient sur les dns menteurs du ministère. ## Noeud actifs | Nom | Bloc | | --------- | ------------------------------- | | | domain\_names.com\_sortedan | | 1000i100 | domain\_names.com\_sortedab | | 1000i100 | domain\_names.com\_sortedaj | | 1000i100 | domain\_names.com\_sortedak | | 1000i100 | domain\_names.com\_sortedal | | sacha | domain\_names.net\_sortedad | | sacha | domain\_names.com\_sortedae | | sacha | domain\_names.com\_sortedaf | | sacha | domain\_names.com\_sortedag | | sacha | domain\_names.com\_sortedam | | tazi | domain\_names.com\_sortedad-1 | | tazi | domain\_names.com\_sortedad-2 | | tazi | domain\_names.com\_sortedad-3 | | tazi | domain\_names.com\_sortedaq-1 | | tazi | domain\_names.com\_sortedaq-2 | | tazi | domain\_names.com\_sortedaq-3 | | louisl l | domain\_names.com\_sortedar | | louisl l | domain\_names.net\_sortedaa | | louisl l | domain\_names.net\_sortedab | | louisl l | domain\_names.com\_sortedah | | louisl l | domain\_names.com\_sortedai | | jerem | domain\_names.com\_sortedao-1 | | jerem | domain\_names.com\_sortedao-2 | | jerem | domain\_names.com\_sortedao-3 | | jerem | domain\_names.com\_sortedap-1 | | jerem | domain\_names.com\_sortedap-2 | | jerem | domain\_names.com\_sortedap-3 | ## Traités domain\_names.com\_sortedac domain\_names.org\_sortedaa 2000000 domain\_names.org\_sortedab 2000000 domain\_names.org\_sortedac 2000000 domain\_names.org\_sortedad 2000000 domain\_names.org\_sortedae domain\_names.org\_sortedaf domain\_names.net\_sortedac domain\_names.net\_sortedae domain\_names.net\_sortedae 1937733 ## Mentio ### mentio-check6 Packages: apt-get install aptitude curl dnsutils python socat tmux unbound whois ``` #-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-# # MENTIODNS : Check for lying DNS (France) # #--------------------------------------------# # Version 1.6.1 date bug # Version 1.6 - conf file # # Version 1.5 - test Dig resolving # # Version 1.4 - Socat SSL sending results # # Version 1.3 - tld optioN # # Version 1.2 - Round robin on DNS_ISP_LIST # # For each request # # Version 1.1 - Allow resume on basename # # Version 1.0 - Parallel process with DIG # #--------------------------------------------# # (c) Sacha at Aquilenet.fr part of FFDN.org # #--------------------------------------------# # This shity script intend to bruteforce the ISP lying DNS Servers to identify which one # is going on Ministry of Interior Blocking page and compare the IP result from your favorite DNS server # Use this script with the following parameters # $1 MODE: client server local # $2 File source: list of domain names whithout tld # $3 tld: com, org, ... # $4 count number (if none from zero or from count file based on file name) # If you relanch the script it will check if it has a counter for the given file to resume # Blacklisted sites in $BLACKLIST_LOG file # Diff ip from a domain name are in $DIFF_LOG # 1st launch creating config file # Copy generated certificates: # FILENAME=mentio_ssl-server # openssl genrsa -out $FILENAME.key 1024 # openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt # cat $FILENAME.key $FILENAME.crt >$FILENAME.pem # FILENAME=mentio_ssl-client # ... ########################################################## HOMEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) MENTIOCONF="$HOMEDIR/mentio.conf" # Number of parallel requests thruw dig parallel=10 ########################################################## ########################################################## ### CHECK CONFIG if [ ! -f $MENTIOCONF ]; then echo "==================================================================" echo "MENTIODNS" echo "------------------------------------------------------------------" echo "1st time configuring" echo -n "IP UNBOUND ? " read DNS_MY echo 'HOMEDIR="'$HOMEDIR'"' > $MENTIOCONF echo 'DNS_MY="'$DNS_MY'"' >> $MENTIOCONF IP_PUB=`curl ifconfig.io` ASN=`whois -h whois.cymru.com $IP_PUB |cut -d' ' -f1|sed -n "2p"` DNS_LIST="$HOMEDIR/mentio-DNS_ISP_LIST" DNS_ISP_LIST=`grep $ASN $DNS_LIST|cut -d' ' -f3-` echo 'DNS_ISP_LIST="'$DNS_ISP_LIST'"' >> $MENTIOCONF echo "------------------------------------------------------------------" echo " CONFIGURATION FILE:" echo " please check and relaunch" echo "------------------------------------------------------------------" cat $MENTIOCONF echo "------------------------------------------------------------------" exit 1 fi ########################################################## ########################################################## ### PARAMETERS to execute the script # Mode Log export with socat "client" "server" "local" MODE=$1 if [ $MODE == "server" ]; then socat -v -u openssl-listen:65522,fork,reuseaddr,cert=mentio_ssl-server.pem,cafile=mentio_ssl-client.crt OPEN:$HOMEDIR/MENTIO-DNS_DIFF,creat,append exit 1 fi ########################################################## ########################################################## ### Check if commandline parameters are less than 3 if [ $# -lt 3 ]; then echo "==================================================================" echo "MENTIODNS" echo "------------------------------------------------------------------" echo "Missing Parameter, please enter:" echo echo "mentio-check client|server|local filename tld (count number)" echo exit 1 fi ########################################################## ########################################################## ### Get parameters # From config file source $MENTIOCONF # From command line # $2 DNS source file name DNS_SOURCE=$2 # $3 TLD name (com, org...) tld=$3 # line counter from the dns source file, nothing for auto-resuming COUNT=$4 ########################################################## ### SOCAT SERVER="SOMEIP:65522" SENDSOCAT="socat stdio openssl-connect:$SERVER,verify=0,cert=$HOMEDIR/mentio_ssl-client.pem,cafile=$HOMEDIR/mentio_ssl-server.crt" ########################################################## ### COLORS RED='\e[31m' GREEN='\e[32m' YELLOW='\e[33m' GRAY='\e[90m' NC='\033[0m' # No Color ########################################################## ### Various variables DNS_SOURCE_BASENAME=`basename $DNS_SOURCE` DIFF_LOG="$HOMEDIR/DNS_DIFF" BLACKLIST_LOG="$HOMEDIR/DNS_BLACKLISTED" lines=`wc -l $DNS_SOURCE|awk -F " " '{print $1}'` countfile="$HOMEDIR/DNS_Count-$DNS_SOURCE_BASENAME" ########################################################## ### Dig parameters DIG_FAST="+nodnssec +short +timeout=1 +tries=2" DIG_SLOW="+nodnssec +short +timeout=5 +tries=3 " ########################################################## ########################################################## ### Generate list for dig: round robin from dns list ### Like (@DNS-server domain) x parallel _check(){ i=0 url="" while [ $i -lt $parallel ] do n=`expr $count + $i` ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1` url="$url @$ISP_DNS `awk -v n="${n}" 'NR==n {print;exit}' $DNS_SOURCE`.$tld" i=`expr $i + 1` done } ########################################################## ########################################################## ### Counter: create one if not existing, use existing instead if [ -z $COUNT ]; then if [ -f $countfile ]; then count=`cat $countfile` else count=0 echo $count > $countfile fi else count=$COUNT echo $count > $countfile fi ########################################################## ########################################################## ### MAIN LOOP while [ "$count" != "$lines" ]; do echo $count > $countfile _check dateus=`date +%Y%m%d-%H%M%S` site="$url" echo "-------------------------------------------------------------------------------" echo "#$count $dateus SITE:$site" if nomentio=`dig @$DNS_MY $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio" ]; then echo -e "$GRAY Unknown zone $site $NC" fi if mentio=`dig $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio" ]; then echo -e "$GRAY Unknown zone $site $NC" fi if [ -n "$nomentio" ] && [ -n "$mentio" ]; then if [ "$nomentio" != "$mentio" ]; then for i in $site; do if nomentio1=`dig $DIG_FAST @$DNS_MY $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio1" ]; then echo -e "$GRAY Unknown zone $i $NC" fi ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1` if mentio1=`dig $DIG_FAST @$ISP_DNS $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio1" ]; then echo -e "$GRAY Unknown zone $i $NC" fi if [ "$nomentio1" != "$mentio1" ]; then if [[ $mentio1 == 90.85.* ]]; then if [ $MODE == "client" ]; then echo "!!! $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" | $SENDSOCAT fi echo -e "$RED !!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" echo "!!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $BLACKLIST_LOG else if [ $MODE == "client" ]; then echo ">>> $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" | $SENDSOCAT fi echo -e "$YELLOW >>> SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" echo ">>> $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $DIFF_LOG fi fi done else echo -e "$GREEN#$count SITE:$site $NC" fi fi count=`expr $count + $parallel` done ########################################################## ``` ### mentio-DNS\_ISP\_LIST Bouygues 5410 194.158.122.10 194.158.122.15 Free 12322 212.27.40.240 212.27.40.241 212.27.40.244 212.27.40.245 Orange 3215 80.10.246.1 80.10.246.2 80.10.246.3 80.10.246.5 80.10.246.7 80.10.246.129 80.10.246.130 80.10.246.132 80.10.246.134 80.10.246.136 81.253.149.1 81.253.149.2 81.253.149.6 81.253.149.9 81.253.149.10 SFR 15557 109.0.66.10 109.0.66.20 ### mentio-monitor hosts="mentio-HOSTS" hosts_tmp="/tmp/mentio-HOSTS.tmp" rm -rf $hosts GREEN='\e[32m' RED='\e[31m' NC='\033[0m' # No Color while true; do timeout 300 tail -n 0 -f MENTIO-DNS_DIFF | cut -d ' ' -f3|sort|uniq> $hosts_tmp mentiohosts=`cat $hosts_tmp` updates=`echo $mentiohosts|xargs -n 1` for i in $updates; do exists=`grep "$i" $hosts` if [ -z "$exists" ]; then echo "ADD $i" echo "$i" >> $hosts sort -o $hosts $hosts name=`echo $i|cut -d "." -f1` fi done dateus=`date +%Y%m%d-%H%M%S` diffs=`diff --side-by-side --suppress-common-lines $hosts $hosts_tmp` echo "======================================================================" if [ -z "$diffs" ]; then echo -e "$GREEN $dateus - ALL HOSTS UP: $NC" echo -e "$GREEN `cat $hosts |xargs |sort` $NC" else echo -e "$RED $dateus - MISSING HOST: $NC" echo -e "$RED $diffs $NC" fi done ## Test (valide au 14/06/18) dig +short shahamat1.com 90.85.16.52 ## Vigies de la neutralité https://ooni.torproject.org https://respectmynet.eu ## Cadre légal https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164\&fastPos;=2\&fastReqId;=606073666\&categorieLien;=cid\&oldAction;=rechTexte#LEGIARTI000029756525 le décret https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=LEGITEXT000030315036\&dateTexte;=20150305\&categorieLien;=cid#LEGITEXT000030315036 https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=JORFTEXT000030195477\&dateTexte;=20180619 ## Unbound server: verbosity: 1 interface: 127.0.0.1 do-ip4: yes do-ip6: no do-udp: yes do-tcp: no access-control: 127.0.0.0/8 allow access-control: 0.0.0.0/0 refuse logfile: /var/log/unbound hide-identity: yes hide-version: yes harden-glue: yes use-caps-for-id: yes do-not-query-localhost: yes ## Lancement tmux new-session -s foo -d "bash mentio-check6 client domain\_names.com\_sortedac com" \\; split-window -h "bash mentio-check6 client domain\_names.com\_sortedae com" \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedam com"\\; selectp -t 0 \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedan com" ## Ansible divers Copy file: ansible mentio -m copy -a "src=mentio-check6 dest=~/MENTIODNS/" Copy file single host: ansible mentio ---limit dam -m copy -a "src=mentio-check6 dest=~/MENTIODNS/" ansible-playbook /etc/ansible/playbooks/mentio.yml ``` --- - hosts: mentio sudo: no tasks: - name: copyfiles copy: src: "{{ item.src }}" dest: "{{ item.dest }}" with_items: - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-check6',dest: '~/MENTIODNS/' } - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-DNS_ISP_LIST',dest: '~/MENTIODNS/' } - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.crt',dest: '~/MENTIODNS/' } - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.key',dest: '~/MENTIODNS/' } - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.pem',dest: '~/MENTIODNS/' } - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-server.crt',dest: '~/MENTIODNS/' } ```