Projet

Général

Profil

Mentiodns » Historique » Révision 46

Révision 45 (sacha, 07/10/2018 16:39) → Révision 46/59 (sacha, 19/10/2018 17:16)

# Mentiodns 

 Validation des DNS à partir d'une liste sur un unbound et un DNS grand FAI et comparaison des résultats ;) 

 ## Noeud actifs 

 | Nom         | Bloc                              | 
 | --------- | ------------------------------- | 
 | dam         | domain\_names.com\_sortedaf       | 
 | dam         | domain\_names.com\_sortedag       | 
 | dam         | domain\_names.com\_sortedah       | 
 | dam         | domain\_names.com\_sortedai       | 
 | mezzanine | domain\_names.com\_sortedad       | 
 | mezzanine | domain\_names.com\_sortedao       | 
 | mezzanine | domain\_names.com\_sortedap       | 
 | mezzanine | domain\_names.com\_sortedaq       | 
 | 1000i100    | domain\_names.com\_sortedab       | 
 | 1000i100    | domain\_names.com\_sortedaj       | 
 | 1000i100    | domain\_names.com\_sortedak       | 
 | 1000i100    | domain\_names.com\_sortedal       | 
 | sacha       | ~~domain\_names.org\_sortedaa~~ | 
 | sacha       | ~~domain\_names.org\_sortedab~~ | 
 | sacha       | ~~domain\_names.org\_sortedac~~ | 
 | sacha       | ~~domain\_names.org\_sortedad~~ | 
 | sacha       | domain\_names.net\_sortedac       | 
 | sacha       | domain\_names.net\_sortedad       | 
 | sacha       | ~~domain\_names.net\_sortedae~~ | 
 | sacha       | domain\_names.com\_sortedac       | 
 | sacha       | domain\_names.com\_sortedae       | 
 | sacha       | domain\_names.com\_sortedam       | 
 | tazi        | domain\_names.com\_sortedan       | 
 | louisl      | domain\_names.com\_sortedar       | 
 | louisl      | domain\_names.org\_sortedae       | 
 | louisl      | ~~domain\_names.org\_sortedaf~~ domain\_names.org\_sortedaf       | 
 | louisl      | domain\_names.net\_sortedaa       | 
 | louisl      | domain\_names.net\_sortedab       | 

 ## Traités 

 domain\_names.org\_sortedaa 2000000 
 domain\_names.org\_sortedab 2000000 
 domain\_names.org\_sortedac 2000000 
 domain\_names.org\_sortedad 2000000 
 domain\_names.net\_sortedae 1937733 

 ## Mentio 

 ### mentio-check6 

 Packages: curl dig python socat tmux unbound whois 

 ```  
 #-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-# 
 #    MENTIODNS : Check for lying DNS (France)    # 
 #--------------------------------------------# 
 #    Version 1.6.1 date bug 
 #    Version 1.6 - conf file                     #  
 #    Version 1.5 - test Dig resolving            # 
 #    Version 1.4 - Socat SSL sending results     #  
 #    Version 1.3 - tld optioN                    # 
 #    Version 1.2 - Round robin on DNS_ISP_LIST # 
 #                  For each request              # 
 #    Version 1.1 - Allow resume on basename      # 
 #    Version 1.0 - Parallel process with DIG     # 
 #--------------------------------------------# 
 # (c) Sacha at Aquilenet.fr part of FFDN.org # 
 #--------------------------------------------# 

 # This shity script intend to bruteforce the ISP lying DNS Servers to identify which one 
 # is going on Ministry of Interior Blocking page and compare the IP result from your favorite DNS server 
 # Use this script with the following parameters  
 # $1 MODE: client server local 
 # $2 File source: list of domain names whithout tld 
 # $3 tld: com, org, ... 
 # $4 count number (if none from zero or from count file based on file name) 


 # If you relanch the script it will check if it has a counter for the given file to resume 
 # Blacklisted sites in $BLACKLIST_LOG file 
 # Diff ip from a domain name are in $DIFF_LOG  

 # 1st launch creating config file 

 # Copy generated certificates: 
 # FILENAME=mentio_ssl-server  
 # openssl genrsa -out $FILENAME.key 1024 
 # openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt 
 # cat $FILENAME.key $FILENAME.crt >$FILENAME.pem 
 # FILENAME=mentio_ssl-client 
 # ... 

 ########################################################## 
 HOMEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) 
 MENTIOCONF="$HOMEDIR/mentio.conf" 
 # Number of parallel requests thruw dig 
 parallel=10 
 ########################################################## 


 ########################################################## 
 ### CHECK CONFIG 
 if [ ! -f $MENTIOCONF ]; then 
 echo "==================================================================" 
 echo "MENTIODNS" 
 echo "------------------------------------------------------------------"  
 echo "1st time configuring" 
 echo -n "IP UNBOUND ? " 
 read DNS_MY 

 echo 'HOMEDIR="'$HOMEDIR'"' > $MENTIOCONF 
 echo 'DNS_MY="'$DNS_MY'"' >> $MENTIOCONF 
 IP_PUB=`curl ifconfig.io` 
 ASN=`whois -h whois.cymru.com $IP_PUB |cut -d' ' -f1|sed -n "2p"` 
 DNS_LIST="$HOMEDIR/mentio-DNS_ISP_LIST" 
 DNS_ISP_LIST=`grep $ASN $DNS_LIST|cut -d' ' -f3-` 
 echo 'DNS_ISP_LIST="'$DNS_ISP_LIST'"' >> $MENTIOCONF 
 echo "------------------------------------------------------------------" 
 echo " CONFIGURATION FILE:" 
 echo " please check and relaunch" 
 echo "------------------------------------------------------------------" 
 cat $MENTIOCONF 
 echo "------------------------------------------------------------------" 
 exit 1 
 fi 
 ########################################################## 


 ########################################################## 
 ### PARAMETERS to execute the script 
 # Mode Log export with socat "client" "server" "local" 

 MODE=$1 

 if [ $MODE == "server" ]; then 
 socat -v -u openssl-listen:65522,fork,reuseaddr,cert=mentio_ssl-server.pem,cafile=mentio_ssl-client.crt OPEN:$HOMEDIR/MENTIO-DNS_DIFF,creat,append 
 exit 1 
 fi 
 ########################################################## 


 ########################################################## 
 ### Check if commandline parameters are less than 3 

 if [ $# -lt 3 ]; then 
 echo "==================================================================" 
 echo "MENTIODNS" 
 echo "------------------------------------------------------------------" 
 echo "Missing Parameter, please enter:" 
 echo 
 echo "mentio-check client|server|local filename tld (count number)" 
 echo 
 exit 1 
 fi 
 ########################################################## 


 ########################################################## 
 ### Get parameters 

 # From config file 
 source $MENTIOCONF 

 # From command line 
 # $2 DNS source file name 
 DNS_SOURCE=$2 
 # $3 TLD name (com, org...) 
 tld=$3 
 # line counter from the dns source file, nothing for auto-resuming 
 COUNT=$4 

 ########################################################## 
 ### SOCAT 
 SERVER="SOMEIP:65522" 
 SENDSOCAT="socat stdio openssl-connect:$SERVER,verify=0,cert=$HOMEDIR/mentio_ssl-client.pem,cafile=$HOMEDIR/mentio_ssl-server.crt" 
 ########################################################## 
 ### COLORS  
 RED='\e[31m' 
 GREEN='\e[32m' 
 YELLOW='\e[33m' 
 GRAY='\e[90m' 
 NC='\033[0m' # No Color 
 ########################################################## 
 ### Various variables 
 DNS_SOURCE_BASENAME=`basename $DNS_SOURCE` 
 DIFF_LOG="$HOMEDIR/DNS_DIFF" 
 BLACKLIST_LOG="$HOMEDIR/DNS_BLACKLISTED" 
 lines=`wc -l $DNS_SOURCE|awk -F " " '{print $1}'` 
 countfile="$HOMEDIR/DNS_Count-$DNS_SOURCE_BASENAME" 
 ########################################################## 
 ### Dig parameters 
 DIG_FAST="+nodnssec +short +timeout=1 +tries=2" 
 DIG_SLOW="+nodnssec +short +timeout=5 +tries=3 " 
 ########################################################## 


 ########################################################## 
 ### Generate list for dig: round robin from dns list 
 ### Like (@DNS-server domain) x parallel  
 _check(){ 
 i=0 
 url="" 
 while [ $i -lt $parallel ] 
 do 
 n=`expr $count + $i` 
 ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1` 
 url="$url @$ISP_DNS `awk -v n="${n}" 'NR==n {print;exit}' $DNS_SOURCE`.$tld" 
 i=`expr $i + 1` 
 done 
 } 
 ########################################################## 


 ########################################################## 
 ### Counter: create one if not existing, use existing instead 
 if [ -z $COUNT ]; then 
         if [ -f $countfile ]; then 
         count=`cat $countfile` 
         else 
         count=0 
         echo $count > $countfile 
         fi 
 else count=$COUNT 
 echo $count > $countfile 
 fi 
 ########################################################## 


 ########################################################## 
 ### MAIN LOOP 

 while [ "$count" != "$lines" ]; do 
 echo $count > $countfile 
 _check 
 dateus=`date +%Y%m%d-%H%M%S` 
 site="$url" 
 echo "-------------------------------------------------------------------------------" 
 echo "#$count $dateus SITE:$site" 
 if nomentio=`dig @$DNS_MY $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio" ]; then 
         echo -e "$GRAY Unknown zone $site $NC"  
 fi 
 if mentio=`dig $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio" ]; then 
         echo -e "$GRAY Unknown zone $site $NC" 
 fi 

 if [ -n "$nomentio" ] && [ -n "$mentio" ]; then 
     if [ "$nomentio" != "$mentio" ]; then 
         for i in $site; do 
                 if nomentio1=`dig $DIG_FAST @$DNS_MY $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio1" ]; then 
                         echo -e "$GRAY Unknown zone $i $NC" 
                 fi 
                 ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1` 
                 if mentio1=`dig $DIG_FAST @$ISP_DNS $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio1" ]; then 
                         echo -e "$GRAY Unknown zone $i $NC" 
                 fi 
                         if [ "$nomentio1" != "$mentio1" ]; then 
                                 if [[ $mentio1 == 90.85.* ]]; then 
                                         if [ $MODE == "client" ]; then 
                                                 echo "!!! $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" | $SENDSOCAT 
                                         fi 
                                         echo -e "$RED !!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" 
                                         echo "!!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $BLACKLIST_LOG 
                                 else 
                                                 if [ $MODE == "client" ]; then 
                                                         echo ">>> $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" | $SENDSOCAT 
                                                 fi                    
                                         echo -e "$YELLOW >>> SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" 
                                         echo ">>> $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $DIFF_LOG 
                                 fi 
                         fi 
         done 
     else 
     echo -e "$GREEN#$count    SITE:$site    $NC" 
     fi 
 fi 

 count=`expr $count + $parallel` 

 done 
 ########################################################## 



 ``` 

 ### mentio-DNS\_ISP\_LIST 

     Bouygues 5410 194.158.122.10 194.158.122.15 
     Free 12322 212.27.40.240 212.27.40.241 212.27.40.244 212.27.40.245 
     Orange 3215 80.10.246.1 80.10.246.2 80.10.246.3 80.10.246.5 80.10.246.7 80.10.246.129 80.10.246.130 80.10.246.132 80.10.246.134 80.10.246.136 81.253.149.1 81.253.149.2 81.253.149.6 81.253.149.9 81.253.149.10 
     SFR 15557 109.0.66.10 109.0.66.20 

 ### mentio-monitor 

     hosts="mentio-HOSTS" 
     hosts_tmp="/tmp/mentio-HOSTS.tmp" 
     rm -rf $hosts 
    
    
     GREEN='\e[32m' 
     RED='\e[31m' 
     NC='\033[0m' # No Color   
    
     while true; do 
             timeout 300 tail -n 0 -f MENTIO-DNS_DIFF | cut -d ' ' -f3|sort|uniq> $hosts_tmp 
             mentiohosts=`cat $hosts_tmp` 
             updates=`echo $mentiohosts|xargs -n 1` 
             for i in $updates; do 
                     exists=`grep "$i" $hosts` 
                     if [ -z "$exists" ]; then 
                             echo "ADD $i" 
                             echo "$i"    >> $hosts  
                             sort -o $hosts $hosts 
                             name=`echo $i|cut -d "." -f1` 
                     fi 
             done 
    
     dateus=`date +%Y%m%d-%H%M%S` 
     diffs=`diff --side-by-side --suppress-common-lines $hosts $hosts_tmp` 
     echo "======================================================================" 
     if [ -z "$diffs" ]; then 
      echo -e "$GREEN $dateus - ALL HOSTS UP: $NC" 
      echo -e "$GREEN `cat $hosts |xargs |sort` $NC"   
     else 
      echo -e "$RED $dateus - MISSING HOST: $NC" 
      echo -e "$RED $diffs $NC" 
     fi 
     done 

 ## Test (valide au 14/06/18) 

 dig +short shahamat1.com   
 90.85.16.52 

 ## Vigies de la neutralité 

 https://ooni.torproject.org   
 https://respectmynet.eu 

 ## Cadre légal 

 https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164\&fastPos;=2\&fastReqId;=606073666\&categorieLien;=cid\&oldAction;=rechTexte#LEGIARTI000029756525   
 le décret https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=LEGITEXT000030315036\&dateTexte;=20150305\&categorieLien;=cid#LEGITEXT000030315036   
 https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=JORFTEXT000030195477\&dateTexte;=20180619 

 ## Unbound 

     server: 
      verbosity: 1 
      interface: 127.0.0.1 
      do-ip4: yes 
      do-ip6: no 
      do-udp: yes 
      do-tcp: no 
      access-control: 127.0.0.0/8 allow  
      access-control: 0.0.0.0/0 refuse 
      logfile: /var/log/unbound 
      hide-identity: yes 
      hide-version: yes 
      harden-glue: yes 
      use-caps-for-id: yes 
      do-not-query-localhost: yes 

 ## Lancement 

 tmux new-session -s foo -d "bash mentio-check6 client domain\_names.com\_sortedac com" \\; split-window -h "bash mentio-check6 client domain\_names.com\_sortedae com" \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedam com"\\; selectp -t 0 \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedan com" 

 ## Ansible divers 

 Copy file: 

 ansible mentio -m copy -a "src=mentio-check6 dest=~/MENTIODNS/" 

 Copy file single host: 

 ansible mentio ---limit dam -m copy -a "src=mentio-check6 dest=~/MENTIODNS/" 

 ansible-playbook /etc/ansible/playbooks/mentio.yml 

 ```  
 --- 

 - hosts: mentio  
   sudo: no 
   tasks: 
     - name: copyfiles  
       copy: 
         src: "{{ item.src }}" 
         dest: "{{ item.dest }}" 
       with_items: 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-check6',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-DNS_ISP_LIST',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.crt',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.key',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.pem',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-server.crt',dest: '~/MENTIODNS/' } 

 ```