Projet

Général

Profil

Actions

Mentiodns » Historique » Révision 39

« Précédent | Révision 39/59 (diff) | Suivant »
sacha, 14/07/2018 18:20


h1. Mentiodns

Validation des DNS à partir d'une liste sur un unbound et un DNS grand FAI et comparaison des résultats ;)

h2. Noeud actifs

|. Nom |. Bloc |
| dam | domain_names.com_sortedaf|
| dam | domain_names.com_sortedag|
| dam | domain_names.com_sortedah|
| dam | domain_names.com_sortedai|
| mezzanine | domain_names.com_sortedad |
| mezzanine | domain_names.com_sortedao |
| mezzanine | domain_names.com_sortedap |
| mezzanine | domain_names.com_sortedaq |
| 1000i100 | domain_names.com_sortedab |
| 1000i100 | domain_names.com_sortedaj |
| 1000i100 | domain_names.com_sortedak |
| 1000i100 | domain_names.com_sortedal |
| sacha | -domain_names.org_sortedaa- |
| sacha | -domain_names.org_sortedab- |
| sacha | -domain_names.org_sortedac- |
| sacha | -domain_names.org_sortedad- |
| sacha | domain_names.net_sortedac |
| sacha | domain_names.net_sortedad |
| sacha | domain_names.net_sortedae |
| tazi | domain_names.com_sortedac |
| tazi | domain_names.com_sortedae |
| tazi | domain_names.com_sortedam |
| tazi | domain_names.com_sortedan |
| louisl | domain_names.com_sortedar |
| louisl | domain_names.org_sortedae |
| louisl | domain_names.org_sortedaf |
| louisl | domain_names.net_sortedaa |
| louisl | domain_names.net_sortedab |

h2. Traités

domain_names.org_sortedaa 2000000
domain_names.org_sortedab 2000000

h2. Mentio

h3. mentio-check6

Packages: curl dig python socat tmux unbound whois

#-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-#

MENTIODNS : Check for lying DNS (France)

#--------------------------------------------#

Version 1.6.1 date bug

Version 1.6 - conf file #

Version 1.5 - test Dig resolving

Version 1.4 - Socat SSL sending results #

Version 1.3 - tld optioN

Version 1.2 - Round robin on DNS_ISP_LIST

For each request

Version 1.1 - Allow resume on basename

Version 1.0 - Parallel process with DIG

#--------------------------------------------#

(c) Sacha at Aquilenet.fr part of FFDN.org

#--------------------------------------------#

This shity script intend to bruteforce the ISP lying DNS Servers to identify which one

is going on Ministry of Interior Blocking page and compare the IP result from your favorite DNS server

Use this script with the following parameters

$1 MODE: client server local

$2 File source: list of domain names whithout tld

$3 tld: com, org, ...

$4 count number (if none from zero or from count file based on file name)

If you relanch the script it will check if it has a counter for the given file to resume

Blacklisted sites in $BLACKLIST_LOG file

Diff ip from a domain name are in $DIFF_LOG

1st launch creating config file

Copy generated certificates:

FILENAME=mentio_ssl-server

openssl genrsa -out $FILENAME.key 1024

openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt

cat $FILENAME.key $FILENAME.crt >$FILENAME.pem

FILENAME=mentio_ssl-client

...

##########################################################
HOMEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
MENTIOCONF="$HOMEDIR/mentio.conf"

Number of parallel requests thruw dig

parallel=10
##########################################################

##########################################################

CHECK CONFIG

if [ ! -f $MENTIOCONF ]; then
echo "=================================================================="
echo "MENTIODNS"
echo "------------------------------------------------------------------"
echo "1st time configuring"
echo -n "IP UNBOUND ? "
read DNS_MY

echo 'HOMEDIR="'$HOMEDIR'"' > $MENTIOCONF
echo 'DNS_MY="'$DNS_MY'"' >> $MENTIOCONF
IP_PUB=curl ifconfig.io
ASN=whois -h whois.cymru.com $IP_PUB |cut -d' ' -f1|sed -n "2p"
DNS_LIST="$HOMEDIR/mentio-DNS_ISP_LIST"
DNS_ISP_LIST=grep $ASN $DNS_LIST|cut -d' ' -f3-
echo 'DNS_ISP_LIST="'$DNS_ISP_LIST'"' >> $MENTIOCONF
echo "------------------------------------------------------------------"
echo " CONFIGURATION FILE:"
echo " please check and relaunch"
echo "------------------------------------------------------------------"
cat $MENTIOCONF
echo "------------------------------------------------------------------"
exit 1
fi
##########################################################

##########################################################

PARAMETERS to execute the script

Mode Log export with socat "client" "server" "local"

MODE=$1

if [ $MODE == "server" ]; then
socat -v -u openssl-listen:65522,fork,reuseaddr,cert=mentio_ssl-server.pem,cafile=mentio_ssl-client.crt OPEN:$HOMEDIR/MENTIO-DNS_DIFF,creat,append
exit 1
fi
##########################################################

##########################################################

Check if commandline parameters are less than 3

if [ $# -lt 3 ]; then
echo "=================================================================="
echo "MENTIODNS"
echo "------------------------------------------------------------------"
echo "Missing Parameter, please enter:"
echo
echo "mentio-check client|server|local filename tld (count number)"
echo
exit 1
fi
##########################################################

##########################################################

Get parameters

From config file

source $MENTIOCONF

From command line

$2 DNS source file name

DNS_SOURCE=$2

$3 TLD name (com, org...)

tld=$3

line counter from the dns source file, nothing for auto-resuming

COUNT=$4

##########################################################

SOCAT

SERVER="SOMEIP:65522"
SENDSOCAT="socat stdio openssl-connect:$SERVER,verify=0,cert=$HOMEDIR/mentio_ssl-client.pem,cafile=$HOMEDIR/mentio_ssl-server.crt"
##########################################################

COLORS

RED='\e[31m'
GREEN='\e[32m'
YELLOW='\e[33m'
GRAY='\e[90m'
NC='\033[0m' # No Color
##########################################################

Various variables

DNS_SOURCE_BASENAME=basename $DNS_SOURCE
DIFF_LOG="$HOMEDIR/DNS_DIFF"
BLACKLIST_LOG="$HOMEDIR/DNS_BLACKLISTED"
lines=wc -l $DNS_SOURCE|awk -F " " '{print $1}'
countfile="$HOMEDIR/DNS_Count-$DNS_SOURCE_BASENAME"
##########################################################

Dig parameters

DIG_FAST="+nodnssec +short +timeout=1 +tries=2"
DIG_SLOW="+nodnssec +short +timeout=5 +tries=3 "
##########################################################

##########################################################

Generate list for dig: round robin from dns list

Like (@DNS-server domain) x parallel

_check(){
i=0
url=""
while [ $i -lt $parallel ]
do
n=expr $count + $i
ISP_DNS=echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1
url="$url @$ISP_DNS awk -v n="${n}" 'NR==n {print;exit}' $DNS_SOURCE.$tld"
i=expr $i + 1
done
}
##########################################################

##########################################################

Counter: create one if not existing, use existing instead

if [ -z $COUNT ]; then
if [ -f $countfile ]; then
count=cat $countfile
else
count=0
echo $count > $countfile
fi
else count=$COUNT
echo $count > $countfile
fi
##########################################################

##########################################################

MAIN LOOP

while [ "$count" != "$lines" ]; do
echo $count > $countfile
_check
dateus=date +%Y%m%d-%H%M%S
site="$url"
echo "-------------------------------------------------------------------------------"
echo "#$count $dateus SITE:$site"
if nomentio=dig @$DNS_MY $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' ' && [ -z "$nomentio" ]; then
echo -e "$GRAY Unknown zone $site $NC"
fi
if mentio=dig $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' ' && [ -z "$mentio" ]; then
echo -e "$GRAY Unknown zone $site $NC"
fi

if [ -n "$nomentio" ] && [ -n "$mentio" ]; then
if [ "$nomentio" != "$mentio" ]; then
for i in $site; do
if nomentio1=dig $DIG_FAST @$DNS_MY $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' ' && [ -z "$nomentio1" ]; then
echo -e "$GRAY Unknown zone $i $NC"
fi
ISP_DNS=echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1
if mentio1=dig $DIG_FAST @$ISP_DNS $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' ' && [ -z "$mentio1" ]; then
echo -e "$GRAY Unknown zone $i $NC"
fi
if [ "$nomentio1" != "$mentio1" ]; then
if $mentio1 == 90.85.* ; then
if [ $MODE == "client" ]; then
echo "!!! $dateus hostname SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" | $SENDSOCAT
fi
echo -e "$RED !!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
echo "!!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $BLACKLIST_LOG
else
if [ $MODE == "client" ]; then
echo ">>> $dateus hostname SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" | $SENDSOCAT
fi

echo -e "$YELLOW >>> SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
echo ">>> $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $DIFF_LOG
fi
fi
done
else
echo -e "$GREEN#$count SITE:$site $NC"
fi
fi

count=expr $count + $parallel

done
##########################################################

h3. mentio-DNS_ISP_LIST

Bouygues 5410 194.158.122.10 194.158.122.15
Free 12322 212.27.40.240 212.27.40.241 212.27.40.244 212.27.40.245
Orange 3215 80.10.246.1 80.10.246.2 80.10.246.3 80.10.246.5 80.10.246.7 80.10.246.129 80.10.246.130 80.10.246.132 80.10.246.134 80.10.246.136 81.253.149.1 81.253.149.2 81.253.149.6 81.253.149.9 81.253.149.10
SFR 15557 109.0.66.10 109.0.66.20

h3. mentio-monitor

hosts="mentio-HOSTS"
hosts_tmp="/tmp/mentio-HOSTS.tmp"
rm -rf $hosts

GREEN='\e[32m'
RED='\e[31m'
NC='\033[0m' # No Color

while true; do
timeout 300 tail -n 0 -f MENTIO-DNS_DIFF | cut -d ' ' -f3|sort|uniq> $hosts_tmp
mentiohosts=cat $hosts_tmp
updates=echo $mentiohosts|xargs -n 1
for i in $updates; do
exists=grep "$i" $hosts
if [ -z "$exists" ]; then
echo "ADD $i"
echo "$i" >> $hosts
sort -o $hosts $hosts
name=echo $i|cut -d "." -f1
fi
done

dateus=date +%Y%m%d-%H%M%S
diffs=diff --side-by-side --suppress-common-lines $hosts $hosts_tmp
echo "======================================================================"
if [ -z "$diffs" ]; then
echo -e "$GREEN $dateus - ALL HOSTS UP: $NC"
echo -e "$GREEN cat $hosts |xargs |sort $NC"

else
echo -e "$RED $dateus - MISSING HOST: $NC"
echo -e "$RED $diffs $NC"
fi
done

h2. Test (valide au 14/06/18)

dig +short shahamat1.com
90.85.16.52

h2. Vigies de la neutralité

https://ooni.torproject.org
https://respectmynet.eu

h2. Cadre légal

https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164&fastPos;=2&fastReqId;=606073666&categorieLien;=cid&oldAction;=rechTexte#LEGIARTI000029756525

le décret https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=LEGITEXT000030315036&dateTexte;=20150305&categorieLien;=cid#LEGITEXT000030315036

https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=JORFTEXT000030195477&dateTexte;=20180619

h2. Unbound

server:
verbosity: 1
interface: 127.0.0.1
do-ip4: yes
do-ip6: no
do-udp: yes
do-tcp: no
access-control: 127.0.0.0/8 allow
access-control: 0.0.0.0/0 refuse
logfile: /var/log/unbound
hide-identity: yes
hide-version: yes
harden-glue: yes
use-caps-for-id: yes
do-not-query-localhost: yes

h2. Lancement

tmux new-session -s foo -d "bash mentio-check6 client domain_names.com_sortedac com" \; split-window -h "bash mentio-check6 client domain_names.com_sortedae com" \; split-window -v "bash mentio-check6 client domain_names.com_sortedam com"\; selectp -t 0 \; split-window -v "bash mentio-check6 client domain_names.com_sortedan com"

h2. Ansible divers

Copy file:

ansible mentio -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"

Copy file single host:

ansible mentio --limit dam -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"

ansible-playbook /etc/ansible/playbooks/mentio.yml

  • hosts: mentio sudo: no tasks:
    • name: copyfiles copy: src: "{{ item.src }}" dest: "{{ item.dest }}" with_items:
      • { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-check6',dest: '~/MENTIODNS/' }
      • { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-DNS_ISP_LIST',dest: '~/MENTIODNS/' }
      • { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.crt',dest: '~/MENTIODNS/' }
      • { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.key',dest: '~/MENTIODNS/' }
      • { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.pem',dest: '~/MENTIODNS/' }
      • { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-server.crt',dest: '~/MENTIODNS/' }

Mis à jour par sacha il y a plus de 5 ans · 39 révisions