Version 19 - Historique - Mentiodns - Aquilenet - L'atelier d'Aquilenet

Mentiodns » Historique » Version 19

sacha, 25/06/2018 22:15

-sacha
+h1. Mentiodns
 Validation des DNS à partir d'une liste sur un unbound et un DNS grand FAI et comparaison des résultats ;)
-sacha
+h2. Noeud actifs
 |_. Nom |_. Bloc |
-sacha
+| Dam64 | domain_names.com_sortedaf|
 | Dam64 | domain_names.com_sortedag|
-sacha
+| Mezzanine | domain_names.com_sortedad |
 | Millicent | domain_names.com_sortedab |
-sacha
+| Sacha | domain_names.org_sortedaa |
 | Sacha | domain_names.org_sortedab |
 | Sacha | domain_names.org_sortedab |
-sacha
+| Sacha | domain_names.org_sortedac |
 | Sacha | domain_names.org_sortedac |
-sacha
+| Taz | domain_names.com_sortedac |
 | Taz | domain_names.com_sortedae |
 sacha
 sacha
-sacha
+h2. Mentio
-sacha
+Packages: curl dig python socat tmux unbound whois
-sacha
+<pre>
 #-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-#
 #  MENTIODNS : Check for lying DNS (France)  #
 #--------------------------------------------#
-sacha
+#  Version 1.6 - conf file                   #
-sacha
+#  Version 1.5 - test Dig resolving          #
 #  Version 1.4 - Socat SSL sending results   #
 #  Version 1.3 - tld optioN                  #
-sacha
+#  Version 1.2 - Round robin on DNS_ISP_LIST #
-sacha
+#		 For each request	     #
-sacha
+#  Version 1.1 - Allow resume on basename    #
 #  Version 1.0 - Parallel process with DIG   #
 #--------------------------------------------#
-sacha
+# (c) Sacha at Aquilenet.fr part of FFDN.org #
 #--------------------------------------------#
 # This shity script intend to bruteforce the ISP lying DNS Servers to identify which one
 # is going on Ministry of Interior Blocking page and compare the IP result from your favorite DNS server
-sacha
+# Use this script with the following parameters
 # $1 MODE: client server local
 # $2 File source: list of domain names whithout tld
 # $3 tld: com, org, ...
 # $4 count number (if none from zero or from count file based on file name)
-sacha
+# If you relanch the script it will check if it has a counter for the given file to resume
 # Blacklisted sites in $BLACKLIST_LOG file
-sacha
+# Diff ip from a domain name are in $DIFF_LOG
 sacha
-sacha
+# 1st launch creating config file
 sacha
-sacha
+# Copy generated certificates:
 # FILENAME=mentio_ssl-server
-sacha
+# openssl genrsa -out $FILENAME.key 1024
-sacha
+# openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt
-sacha
+# cat $FILENAME.key $FILENAME.crt >$FILENAME.pem
-sacha
+# FILENAME=mentio_ssl-client
 # ...
-sacha
+##########################################################
-sacha
+HOMEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
-sacha
+MENTIOCONF="$HOMEDIR/mentio.conf"
 # Number of parallel requests thruw dig
 parallel=10
-sacha
+# Socat server
 SERVER="taz.im:65522"
-sacha
+##########################################################
 sacha
 sacha
 ##########################################################
-sacha
+### CHECK CONFIG
 if [ ! -f $MENTIOCONF ]; then
 echo "=================================================================="
 echo "MENTIODNS"
 echo "------------------------------------------------------------------"
 echo "1st time configuring"
 echo -n "IP UNBOUND ? "
 read DNS_MY
 sacha
-sacha
+echo 'HOMEDIR="'$HOMEDIR'"' > $MENTIOCONF
 echo 'DNS_MY="'$DNS_MY'"' >> $MENTIOCONF
-sacha
+IP_PUB=`curl ifconfig.io`
-sacha
+ASN=`whois -h whois.cymru.com $IP_PUB |cut -d' ' -f1|sed -n "2p"`
-sacha
+DNS_LIST="$HOMEDIR/mentio-DNS_ISP_LIST"
 DNS_ISP_LIST=`grep $ASN $DNS_LIST|cut -d' ' -f3-`
 echo 'DNS_ISP_LIST="'$DNS_ISP_LIST'"' >> $MENTIOCONF
 echo "------------------------------------------------------------------"
 echo " CONFIGURATION FILE:"
 echo " please check and relaunch"
 echo "------------------------------------------------------------------"
 cat $MENTIOCONF
 echo "------------------------------------------------------------------"
 exit 1
 fi
-sacha
+##########################################################
 sacha
 sacha
 ##########################################################
 ### PARAMETERS to execute the script
 # Mode Log export with socat "client" "server" "local"
 MODE=$1
 if [ $MODE == "server" ]; then
 socat -v -u openssl-listen:65522,fork,reuseaddr,cert=mentio_ssl-server.pem,cafile=mentio_ssl-client.crt OPEN:$HOMEDIR/MENTIO-DNS_DIFF,creat,append
 exit 1
 fi
 ##########################################################
 ##########################################################
 ### Check if commandline parameters are less than 3
-sacha
+if [ $# -lt 3 ]; then
 echo "=================================================================="
 echo "MENTIODNS"
 echo "------------------------------------------------------------------"
-sacha
+echo "Missing Parameter, please enter:"
-sacha
+echo
-sacha
+echo "mentio-check client|server|local filename tld (count number)"
-sacha
+echo
 exit 1
 fi
-sacha
+##########################################################
 sacha
 sacha
 ##########################################################
 ### Get parameters
 # From config file
-sacha
+source $MENTIOCONF
 sacha
-sacha
+# From command line
-sacha
+# $2 DNS source file name
-sacha
+DNS_SOURCE=$2
-sacha
+# $3 TLD name (com, org...)
-sacha
+tld=$3
-sacha
+# line counter from the dns source file, nothing for auto-resuming
 COUNT=$4
 sacha
-sacha
+##########################################################
 ### SOCAT
 SENDSOCAT="socat stdio openssl-connect:$SERVER,verify=0,cert=$HOMEDIR/mentio_ssl-client.pem,cafile=$HOMEDIR/mentio_ssl-server.crt"
-sacha
+##########################################################
-sacha
+### COLORS
 RED='\e[31m'
 GREEN='\e[32m'
 YELLOW='\e[33m'
-sacha
+GRAY='\e[90m'
 NC='\033[0m' # No Color
-sacha
+##########################################################
 ### Various variables
-sacha
+DNS_SOURCE_BASENAME=`basename $DNS_SOURCE`
-sacha
+DIFF_LOG="$HOMEDIR/DNS_DIFF"
 BLACKLIST_LOG="$HOMEDIR/DNS_BLACKLISTED"
 lines=`wc -l $DNS_SOURCE|awk -F " " '{print $1}'`
-sacha
+countfile="$HOMEDIR/DNS_Count-$DNS_SOURCE_BASENAME"
-sacha
+dateus=`date +%Y%m%d-%H%M%S`
-sacha
+##########################################################
 ### Dig parameters
-sacha
+DIG_FAST="+nodnssec +short +timeout=1 +tries=2"
 DIG_SLOW="+nodnssec +short +timeout=5 +tries=3 "
-sacha
+##########################################################
 sacha
 sacha
 ##########################################################
 ### Generate list for dig: round robin from dns list
 ### Like (@DNS-server domain) x parallel
-sacha
+_check(){
 i=0
-sacha
+url=""
 while [ $i -lt $parallel ]
-sacha
+do
 n=`expr $count + $i`
 ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1`
 url="$url @$ISP_DNS `awk -v n="${n}" 'NR==n {print;exit}' $DNS_SOURCE`.$tld"
 i=`expr $i + 1`
 done
+}
-sacha
+##########################################################
 sacha
-sacha
+##########################################################
 ### Counter: create one if not existing, use existing instead
-sacha
+if [ -z $COUNT ]; then
-sacha
+	if [ -f $countfile ]; then
-sacha
+	count=`cat $countfile`
 	else
-sacha
+	count=0
 	echo $count > $countfile
 	fi
 else count=$COUNT
 echo $count > $countfile
-sacha
+fi
-sacha
+##########################################################
 sacha
 sacha
 ##########################################################
 ### MAIN LOOP
-sacha
+while [ "$count" != "$lines" ]; do
 echo $count > $countfile
 _check
-sacha
+site="$url"
 echo "-------------------------------------------------------------------------------"
 echo "#$count $dateus SITE:$site"
 if nomentio=`dig @$DNS_MY $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio" ]; then
 	echo -e "$GRAY Unknown zone $site $NC"
 fi
 if mentio=`dig $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio" ]; then
 	echo -e "$GRAY Unknown zone $site $NC"
-sacha
+fi
-sacha
+if [ -n "$nomentio" ] && [ -n "$mentio" ]; then
     if [ "$nomentio" != "$mentio" ]; then
 	for i in $site; do
-sacha
+		if nomentio1=`dig $DIG_FAST @$DNS_MY $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio1" ]; then
-sacha
+			echo -e "$GRAY Unknown zone $i $NC"
 		fi
 		ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1`
 		if mentio1=`dig $DIG_FAST @$ISP_DNS $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio1" ]; then
 			echo -e "$GRAY Unknown zone $i $NC"
 		fi
 			if [ "$nomentio1" != "$mentio1" ]; then
         			if [[ $mentio1 == 90.85.* ]]; then
 		                        if [ $MODE == "client" ]; then
                 		        	echo "!!! $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" | $SENDSOCAT
                         		fi
 					echo -e "$RED !!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
 					echo "!!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $BLACKLIST_LOG
 				else
                         			if [ $MODE == "client" ]; then
                         				echo ">>> $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" | $SENDSOCAT
                         			fi
 					echo -e "$YELLOW >>> SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
 					echo ">>> $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $DIFF_LOG
 				fi
 			fi
-sacha
+	done
-sacha
+    else
-sacha
+    echo -e "$GREEN#$count  SITE:$site  $NC"
-sacha
+    fi
 fi
 count=`expr $count + $parallel`
-sacha
+done
-sacha
+##########################################################
 sacha
 sacha
 sacha
-sacha
+</pre>
 sacha
 h2. Test (valide au 14/06/18)
 sacha
-sacha
+ dig +short shahamat1.com
-sacha
+.85.16.52
 sacha
 h2. Liste de serveurs DNS FAI Français
-sacha
+h3. Free - ASN12322
 sacha
-sacha
+.27.40.240
 .27.40.241
 .27.40.244
-sacha
+.27.40.245
 sacha
-sacha
+h3. Bouygues - ASN5410
 .158.122.10
 .158.122.15
 h3. SFR/Numericable - ASN5410
 .2.0.1
 .2.0.2
 h3. SFR - ASN15557
 .0.66.10
 .0.66.20
 h3. Orange - ASN3215
 sacha
-sacha
+.10.246.1
-sacha
+.10.246.2
 .10.246.3
-sacha
+.10.246.5
 .10.246.7
 .10.246.129
 .10.246.130
 .10.246.132
-sacha
+.10.246.134
 .10.246.136
 .253.149.1
 .253.149.2
-sacha
+.253.149.6
-sacha
+.253.149.9
-sacha
+.253.149.10
 h3. OBS (ouverts)
 .2.0.20
 .2.0.50
 h2. Vigies de la neutralité
 https://ooni.torproject.org
 https://respectmynet.eu
 h2. Cadre légal
 https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164&fastPos;=2&fastReqId;=606073666&categorieLien;=cid&oldAction;=rechTexte#LEGIARTI000029756525
 le décret https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=LEGITEXT000030315036&dateTexte;=20150305&categorieLien;=cid#LEGITEXT000030315036
 https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=JORFTEXT000030195477&dateTexte;=20180619
 sacha
 h2. Unbound
 <pre>
 server:
  verbosity: 1
  interface: 127.0.0.1
  do-ip4: yes
  do-ip6: no
  do-udp: yes
  do-tcp: no
  access-control: 127.0.0.0/8 allow
  access-control: 0.0.0.0/0 refuse
  logfile: /var/log/unbound
  hide-identity: yes
  hide-version: yes
  harden-glue: yes
  use-caps-for-id: yes
  do-not-query-localhost: yes
 </pre>
 sacha
 h2. Ansible divers
 Copy file:
  ansible mentio -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"
 Copy file single host:
  ansible mentio --limit dam -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"
 ansible-playbook /etc/ansible/playbooks/mentio.yml
 <pre>
 ---
 - hosts: mentio
   sudo: no
   tasks:
     - name: copyfiles
       copy:
         src: "{{ item.src }}"
         dest: "{{ item.dest }}"
       with_items:
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-check6',dest: '~/MENTIODNS/' }
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-DNS_ISP_LIST',dest: '~/MENTIODNS/' }
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.crt',dest: '~/MENTIODNS/' }
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.key',dest: '~/MENTIODNS/' }
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.pem',dest: '~/MENTIODNS/' }
         - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-server.crt',dest: '~/MENTIODNS/' }
-sacha
+</pre>

Projet

Général

Profil

Aquilenet

Mentiodns » Historique » Version 19