Révision 32 - Historique - Mentiodns - Aquilenet - L'atelier d'Aquilenet

Mentiodns » Historique » Révision 32

Révision 31 (sacha, 04/07/2018 22:01) → Révision 32/59 (sacha, 06/07/2018 00:30)

h1. Mentiodns 

 Validation des DNS à partir d'une liste sur un unbound et un DNS grand FAI et comparaison des résultats ;) 

 h2. Noeud actifs 

 |_. Nom |_. Bloc | 
 | dam | domain_names.com_sortedaf| 
 | dam | domain_names.com_sortedag| 
 | dam | domain_names.com_sortedah| 
 | dam | domain_names.com_sortedai| 
 | mezzanine | domain_names.com_sortedad | 
 | mezzanine | domain_names.com_sortedao | 
 | mezzanine | domain_names.com_sortedap | 
 | mezzanine | domain_names.com_sortedaq | 
 | 1000i100 | domain_names.com_sortedab | 
 | 1000i100 | domain_names.com_sortedaj | 
 | 1000i100 | domain_names.com_sortedak | 
 | 1000i100 | domain_names.com_sortedal | 
 | sacha | domain_names.org_sortedaa | 
 | sacha | domain_names.org_sortedab | 
 | sacha | domain_names.org_sortedac | 
 | sacha | domain_names.org_sortedad | 
 | tazi | domain_names.com_sortedac | 
 | tazi | domain_names.com_sortedae | 
 | tazi | domain_names.com_sortedam | 
 | tazi | domain_names.com_sortedan | 
 | louisl |  

 h2. Mentio 

 h3. mentio-check6 

 Packages: curl dig python socat tmux unbound whois  

 <pre> 
 #-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-# 
 #    MENTIODNS : Check for lying DNS (France)    # 
 #--------------------------------------------# 
 #    Version 1.6.1 date bug 
 #    Version 1.6 - conf file                     #  
 #    Version 1.5 - test Dig resolving            # 
 #    Version 1.4 - Socat SSL sending results     #  
 #    Version 1.3 - tld optioN                    # 
 #    Version 1.2 - Round robin on DNS_ISP_LIST # 
 #                  For each request              # 
 #    Version 1.1 - Allow resume on basename      # 
 #    Version 1.0 - Parallel process with DIG     # 
 #--------------------------------------------# 
 # (c) Sacha at Aquilenet.fr part of FFDN.org # 
 #--------------------------------------------# 

 # This shity script intend to bruteforce the ISP lying DNS Servers to identify which one 
 # is going on Ministry of Interior Blocking page and compare the IP result from your favorite DNS server 
 # Use this script with the following parameters  
 # $1 MODE: client server local 
 # $2 File source: list of domain names whithout tld 
 # $3 tld: com, org, ... 
 # $4 count number (if none from zero or from count file based on file name) 


 # If you relanch the script it will check if it has a counter for the given file to resume 
 # Blacklisted sites in $BLACKLIST_LOG file 
 # Diff ip from a domain name are in $DIFF_LOG  

 # 1st launch creating config file 

 # Copy generated certificates: 
 # FILENAME=mentio_ssl-server  
 # openssl genrsa -out $FILENAME.key 1024 
 # openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt 
 # cat $FILENAME.key $FILENAME.crt >$FILENAME.pem 
 # FILENAME=mentio_ssl-client 
 # ... 

 ########################################################## 
 HOMEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) 
 MENTIOCONF="$HOMEDIR/mentio.conf" 
 # Number of parallel requests thruw dig 
 parallel=10 
 ########################################################## 


 ########################################################## 
 ### CHECK CONFIG 
 if [ ! -f $MENTIOCONF ]; then 
 echo "==================================================================" 
 echo "MENTIODNS" 
 echo "------------------------------------------------------------------"  
 echo "1st time configuring" 
 echo -n "IP UNBOUND ? " 
 read DNS_MY 

 echo 'HOMEDIR="'$HOMEDIR'"' > $MENTIOCONF 
 echo 'DNS_MY="'$DNS_MY'"' >> $MENTIOCONF 
 IP_PUB=`curl ifconfig.io` 
 ASN=`whois -h whois.cymru.com $IP_PUB |cut -d' ' -f1|sed -n "2p"` 
 DNS_LIST="$HOMEDIR/mentio-DNS_ISP_LIST" 
 DNS_ISP_LIST=`grep $ASN $DNS_LIST|cut -d' ' -f3-` 
 echo 'DNS_ISP_LIST="'$DNS_ISP_LIST'"' >> $MENTIOCONF 
 echo "------------------------------------------------------------------" 
 echo " CONFIGURATION FILE:" 
 echo " please check and relaunch" 
 echo "------------------------------------------------------------------" 
 cat $MENTIOCONF 
 echo "------------------------------------------------------------------" 
 exit 1 
 fi 
 ########################################################## 


 ########################################################## 
 ### PARAMETERS to execute the script 
 # Mode Log export with socat "client" "server" "local" 

 MODE=$1 

 if [ $MODE == "server" ]; then 
 socat -v -u openssl-listen:65522,fork,reuseaddr,cert=mentio_ssl-server.pem,cafile=mentio_ssl-client.crt OPEN:$HOMEDIR/MENTIO-DNS_DIFF,creat,append 
 exit 1 
 fi 
 ########################################################## 


 ########################################################## 
 ### Check if commandline parameters are less than 3 

 if [ $# -lt 3 ]; then 
 echo "==================================================================" 
 echo "MENTIODNS" 
 echo "------------------------------------------------------------------" 
 echo "Missing Parameter, please enter:" 
 echo 
 echo "mentio-check client|server|local filename tld (count number)" 
 echo 
 exit 1 
 fi 
 ########################################################## 


 ########################################################## 
 ### Get parameters 

 # From config file 
 source $MENTIOCONF 

 # From command line 
 # $2 DNS source file name 
 DNS_SOURCE=$2 
 # $3 TLD name (com, org...) 
 tld=$3 
 # line counter from the dns source file, nothing for auto-resuming 
 COUNT=$4 

 ########################################################## 
 ### SOCAT 
 SERVER="SOMEIP:65522" 
 SENDSOCAT="socat stdio openssl-connect:$SERVER,verify=0,cert=$HOMEDIR/mentio_ssl-client.pem,cafile=$HOMEDIR/mentio_ssl-server.crt" 
 ########################################################## 
 ### COLORS  
 RED='\e[31m' 
 GREEN='\e[32m' 
 YELLOW='\e[33m' 
 GRAY='\e[90m' 
 NC='\033[0m' # No Color 
 ########################################################## 
 ### Various variables 
 DNS_SOURCE_BASENAME=`basename $DNS_SOURCE` 
 DIFF_LOG="$HOMEDIR/DNS_DIFF" 
 BLACKLIST_LOG="$HOMEDIR/DNS_BLACKLISTED" 
 lines=`wc -l $DNS_SOURCE|awk -F " " '{print $1}'` 
 countfile="$HOMEDIR/DNS_Count-$DNS_SOURCE_BASENAME" 
 ########################################################## 
 ### Dig parameters 
 DIG_FAST="+nodnssec +short +timeout=1 +tries=2" 
 DIG_SLOW="+nodnssec +short +timeout=5 +tries=3 " 
 ########################################################## 


 ########################################################## 
 ### Generate list for dig: round robin from dns list 
 ### Like (@DNS-server domain) x parallel  
 _check(){ 
 i=0 
 url="" 
 while [ $i -lt $parallel ] 
 do 
 n=`expr $count + $i` 
 ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1` 
 url="$url @$ISP_DNS `awk -v n="${n}" 'NR==n {print;exit}' $DNS_SOURCE`.$tld" 
 i=`expr $i + 1` 
 done 
 } 
 ########################################################## 


 ########################################################## 
 ### Counter: create one if not existing, use existing instead 
 if [ -z $COUNT ]; then 
         if [ -f $countfile ]; then 
         count=`cat $countfile` 
         else 
         count=0 
         echo $count > $countfile 
         fi 
 else count=$COUNT 
 echo $count > $countfile 
 fi 
 ########################################################## 


 ########################################################## 
 ### MAIN LOOP 

 while [ "$count" != "$lines" ]; do 
 echo $count > $countfile 
 _check 
 dateus=`date +%Y%m%d-%H%M%S` 
 site="$url" 
 echo "-------------------------------------------------------------------------------" 
 echo "#$count $dateus SITE:$site" 
 if nomentio=`dig @$DNS_MY $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio" ]; then 
         echo -e "$GRAY Unknown zone $site $NC"  
 fi 
 if mentio=`dig $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio" ]; then 
         echo -e "$GRAY Unknown zone $site $NC" 
 fi 

 if [ -n "$nomentio" ] && [ -n "$mentio" ]; then 
     if [ "$nomentio" != "$mentio" ]; then 
         for i in $site; do 
                 if nomentio1=`dig $DIG_FAST @$DNS_MY $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio1" ]; then 
                         echo -e "$GRAY Unknown zone $i $NC" 
                 fi 
                 ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1` 
                 if mentio1=`dig $DIG_FAST @$ISP_DNS $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio1" ]; then 
                         echo -e "$GRAY Unknown zone $i $NC" 
                 fi 
                         if [ "$nomentio1" != "$mentio1" ]; then 
                                 if [[ $mentio1 == 90.85.* ]]; then 
                                         if [ $MODE == "client" ]; then 
                                                 echo "!!! $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" | $SENDSOCAT 
                                         fi 
                                         echo -e "$RED !!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" 
                                         echo "!!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $BLACKLIST_LOG 
                                 else 
                                                 if [ $MODE == "client" ]; then 
                                                         echo ">>> $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" | $SENDSOCAT 
                                                 fi                    
                                         echo -e "$YELLOW >>> SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" 
                                         echo ">>> $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $DIFF_LOG 
                                 fi 
                         fi 
         done 
     else 
     echo -e "$GREEN#$count    SITE:$site    $NC" 
     fi 
 fi 

 count=`expr $count + $parallel` 

 done 
 ########################################################## 




 </pre> 

 h3. mentio-DNS_ISP_LIST 

 <pre> 
 Bouygues 5410 194.158.122.10 194.158.122.15 
 Free 12322 212.27.40.240 212.27.40.241 212.27.40.244 212.27.40.245 
 Orange 3215 80.10.246.1 80.10.246.2 80.10.246.3 80.10.246.5 80.10.246.7 80.10.246.129 80.10.246.130 80.10.246.132 80.10.246.134 80.10.246.136 81.253.149.1 81.253.149.2 81.253.149.6 81.253.149.9 81.253.149.10 
 SFR 15557 109.0.66.10 109.0.66.20 
 </pre> 

 h3. mentio-monitor 

 <pre> 
 hosts="mentio-HOSTS" 
 hosts_tmp="/tmp/mentio-HOSTS.tmp" 
 rm -rf $hosts 


 GREEN='\e[32m' 
 RED='\e[31m' 
 NC='\033[0m' # No Color   

 while true; do 
         timeout 300 tail -n 0 -f MENTIO-DNS_DIFF | cut -d ' ' -f3|sort|uniq> $hosts_tmp 
         mentiohosts=`cat $hosts_tmp` 
         updates=`echo $mentiohosts|xargs -n 1` 
         for i in $updates; do 
                 exists=`grep "$i" $hosts` 
                 if [ -z "$exists" ]; then 
                         echo "ADD $i" 
                         echo "$i"    >> $hosts  
                         sort -o $hosts $hosts 
                         name=`echo $i|cut -d "." -f1` 
                 fi 
         done 

 dateus=`date +%Y%m%d-%H%M%S` 
 diffs=`diff --side-by-side --suppress-common-lines $hosts $hosts_tmp` 
 echo "======================================================================" 
 if [ -z "$diffs" ]; then 
  echo -e "$GREEN $dateus - ALL HOSTS UP: $NC" 
  echo -e "$GREEN `cat $hosts |xargs |sort` $NC"   
 else 
  echo -e "$RED $dateus - MISSING HOST: $NC" 
  echo -e "$RED $diffs $NC" 
 fi 
 done 
 </pre> 

 h2. Test (valide au 14/06/18) 

  dig +short shahamat1.com 
  90.85.16.52 

 h2. Vigies de la neutralité 

 https://ooni.torproject.org 
 https://respectmynet.eu 

 h2. Cadre légal 

 https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164&fastPos;=2&fastReqId;=606073666&categorieLien;=cid&oldAction;=rechTexte#LEGIARTI000029756525         
 le décret https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=LEGITEXT000030315036&dateTexte;=20150305&categorieLien;=cid#LEGITEXT000030315036   
 https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=JORFTEXT000030195477&dateTexte;=20180619 

 h2. Unbound 

 <pre> 
 server: 
  verbosity: 1 
  interface: 127.0.0.1 
  do-ip4: yes 
  do-ip6: no 
  do-udp: yes 
  do-tcp: no 
  access-control: 127.0.0.0/8 allow  
  access-control: 0.0.0.0/0 refuse 
  logfile: /var/log/unbound 
  hide-identity: yes 
  hide-version: yes 
  harden-glue: yes 
  use-caps-for-id: yes 
  do-not-query-localhost: yes 
 </pre> 

 

 h2. Lancement 

  tmux new-session -s foo -d "bash mentio-check6 client domain_names.com_sortedac com" \; split-window -h "bash mentio-check6 client domain_names.com_sortedae com" \; split-window -v "bash mentio-check6 client domain_names.com_sortedam com"\; selectp -t 0 \; split-window -v "bash mentio-check6 client domain_names.com_sortedan com" 

 

 h2. Ansible divers 

 Copy file: 

  ansible mentio -m copy -a "src=mentio-check6 dest=~/MENTIODNS/" 


 Copy file single host: 

  ansible mentio --limit dam -m copy -a "src=mentio-check6 dest=~/MENTIODNS/" 

 ansible-playbook /etc/ansible/playbooks/mentio.yml  


 <pre> 
 --- 

 - hosts: mentio  
   sudo: no 
   tasks: 
     - name: copyfiles  
       copy: 
         src: "{{ item.src }}" 
         dest: "{{ item.dest }}" 
       with_items: 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-check6',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-DNS_ISP_LIST',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.crt',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.key',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.pem',dest: '~/MENTIODNS/' } 
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-server.crt',dest: '~/MENTIODNS/' } 


 </pre>

Projet

Général

Profil

Aquilenet

Mentiodns » Historique » Révision 32