Projet

Général

Profil

Mentiodns » Historique » Version 57

sacha, 11/11/2018 10:54

1 41 sacha
# Mentiodns
2 1 sacha
3 54 sacha
reCréation de la liste de censure gérée par le ministère de l'intérieur et envoyée automatiquement aux principaux FAI Français.
4
Validation des DNS à partir d'une liste exhaustive par tld. On vérifie si les résolutions DNS envoient sur les dns menteurs du ministère.
5 1 sacha
6 54 sacha
7 41 sacha
## Noeud actifs
8 1 sacha
9 41 sacha
| Nom       | Bloc                            |
10
| --------- | ------------------------------- |
11 57 sacha
|           | domain\_names.com\_sortedan     |
12 41 sacha
| 1000i100  | domain\_names.com\_sortedab     |
13
| 1000i100  | domain\_names.com\_sortedaj     |
14
| 1000i100  | domain\_names.com\_sortedak     |
15
| 1000i100  | domain\_names.com\_sortedal     |
16
| sacha     | domain\_names.net\_sortedad     |
17 1 sacha
| sacha     | domain\_names.com\_sortedae     |
18
| sacha     | domain\_names.com\_sortedaf     |
19
| sacha     | domain\_names.com\_sortedag     |
20
| sacha     | domain\_names.com\_sortedam     |
21 57 sacha
| tazi      | domain\_names.com\_sortedad-1   |
22
| tazi      | domain\_names.com\_sortedad-2   |
23
| tazi      | domain\_names.com\_sortedad-3   |
24
| tazi      | domain\_names.com\_sortedaq-1   |
25
| tazi      | domain\_names.com\_sortedaq-2   |
26
| tazi      | domain\_names.com\_sortedaq-3   |
27 1 sacha
| l         | domain\_names.com\_sortedar     |
28
| l         | domain\_names.net\_sortedaa     |
29
| l         | domain\_names.net\_sortedab     |
30
| l         | domain\_names.com\_sortedah     |
31
| l         | domain\_names.com\_sortedai     |
32 57 sacha
| jerem     | domain\_names.com\_sortedao-1   |
33
| jerem     | domain\_names.com\_sortedao-2   |
34
| jerem     | domain\_names.com\_sortedao-3   |
35
| jerem     | domain\_names.com\_sortedap-1   |
36
| jerem     | domain\_names.com\_sortedap-2   |
37
| jerem     | domain\_names.com\_sortedap-3   |
38 56 sacha
39 1 sacha
40
## Traités
41
42 55 sacha
domain\_names.com\_sortedac
43
44 53 sacha
domain\_names.org\_sortedaa 2000000
45
domain\_names.org\_sortedab 2000000
46 18 sacha
domain\_names.org\_sortedac 2000000
47 41 sacha
domain\_names.org\_sortedad 2000000
48 55 sacha
domain\_names.org\_sortedae
49
domain\_names.org\_sortedaf
50
51
domain\_names.net\_sortedac
52
domain\_names.net\_sortedae
53 42 sacha
domain\_names.net\_sortedae 1937733
54 35 sacha
55 41 sacha
## Mentio
56 4 sacha
57 41 sacha
### mentio-check6
58 27 sacha
59 55 sacha
Packages: apt-get install aptitude curl dnsutils python socat tmux unbound whois
60 19 sacha
61 41 sacha
``` 
62 4 sacha
#-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-#
63
#  MENTIODNS : Check for lying DNS (France)  #
64
#--------------------------------------------#
65 28 sacha
#  Version 1.6.1 date bug
66 16 sacha
#  Version 1.6 - conf file                   # 
67 13 sacha
#  Version 1.5 - test Dig resolving          #
68
#  Version 1.4 - Socat SSL sending results   # 
69
#  Version 1.3 - tld optioN                  #
70 1 sacha
#  Version 1.2 - Round robin on DNS_ISP_LIST #
71 28 sacha
#                For each request            #
72 4 sacha
#  Version 1.1 - Allow resume on basename    #
73
#  Version 1.0 - Parallel process with DIG   #
74
#--------------------------------------------#
75 1 sacha
# (c) Sacha at Aquilenet.fr part of FFDN.org #
76
#--------------------------------------------#
77
78
# This shity script intend to bruteforce the ISP lying DNS Servers to identify which one
79
# is going on Ministry of Interior Blocking page and compare the IP result from your favorite DNS server
80 16 sacha
# Use this script with the following parameters 
81
# $1 MODE: client server local
82
# $2 File source: list of domain names whithout tld
83
# $3 tld: com, org, ...
84
# $4 count number (if none from zero or from count file based on file name)
85
86
87 1 sacha
# If you relanch the script it will check if it has a counter for the given file to resume
88
# Blacklisted sites in $BLACKLIST_LOG file
89 13 sacha
# Diff ip from a domain name are in $DIFF_LOG 
90 1 sacha
91 13 sacha
# 1st launch creating config file
92 1 sacha
93 14 sacha
# Copy generated certificates:
94
# FILENAME=mentio_ssl-server 
95 1 sacha
# openssl genrsa -out $FILENAME.key 1024
96 13 sacha
# openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt
97 1 sacha
# cat $FILENAME.key $FILENAME.crt >$FILENAME.pem
98 13 sacha
# FILENAME=mentio_ssl-client
99
# ...
100
101 16 sacha
##########################################################
102 1 sacha
HOMEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
103 13 sacha
MENTIOCONF="$HOMEDIR/mentio.conf"
104 17 sacha
# Number of parallel requests thruw dig
105
parallel=10
106 16 sacha
##########################################################
107 1 sacha
108 16 sacha
109
##########################################################
110 13 sacha
### CHECK CONFIG
111
if [ ! -f $MENTIOCONF ]; then
112
echo "=================================================================="
113
echo "MENTIODNS"
114
echo "------------------------------------------------------------------" 
115
echo "1st time configuring"
116
echo -n "IP UNBOUND ? "
117
read DNS_MY
118 1 sacha
119 13 sacha
echo 'HOMEDIR="'$HOMEDIR'"' > $MENTIOCONF
120
echo 'DNS_MY="'$DNS_MY'"' >> $MENTIOCONF
121 1 sacha
IP_PUB=`curl ifconfig.io`
122 13 sacha
ASN=`whois -h whois.cymru.com $IP_PUB |cut -d' ' -f1|sed -n "2p"`
123 1 sacha
DNS_LIST="$HOMEDIR/mentio-DNS_ISP_LIST"
124
DNS_ISP_LIST=`grep $ASN $DNS_LIST|cut -d' ' -f3-`
125
echo 'DNS_ISP_LIST="'$DNS_ISP_LIST'"' >> $MENTIOCONF
126
echo "------------------------------------------------------------------"
127
echo " CONFIGURATION FILE:"
128
echo " please check and relaunch"
129
echo "------------------------------------------------------------------"
130
cat $MENTIOCONF
131
echo "------------------------------------------------------------------"
132
exit 1
133
fi
134 16 sacha
##########################################################
135 1 sacha
136 16 sacha
137
##########################################################
138
### PARAMETERS to execute the script
139
# Mode Log export with socat "client" "server" "local"
140
141
MODE=$1
142
143
if [ $MODE == "server" ]; then
144
socat -v -u openssl-listen:65522,fork,reuseaddr,cert=mentio_ssl-server.pem,cafile=mentio_ssl-client.crt OPEN:$HOMEDIR/MENTIO-DNS_DIFF,creat,append
145
exit 1
146
fi
147
##########################################################
148
149
150
##########################################################
151
### Check if commandline parameters are less than 3
152
153 1 sacha
if [ $# -lt 3 ]; then
154
echo "=================================================================="
155
echo "MENTIODNS"
156
echo "------------------------------------------------------------------"
157 14 sacha
echo "Missing Parameter, please enter:"
158 13 sacha
echo
159 1 sacha
echo "mentio-check client|server|local filename tld (count number)"
160 13 sacha
echo
161
exit 1
162
fi
163 16 sacha
##########################################################
164 13 sacha
165 16 sacha
166
##########################################################
167
### Get parameters
168
169
# From config file
170 1 sacha
source $MENTIOCONF
171 13 sacha
172 16 sacha
# From command line
173 1 sacha
# $2 DNS source file name
174 13 sacha
DNS_SOURCE=$2
175 1 sacha
# $3 TLD name (com, org...)
176 13 sacha
tld=$3
177 1 sacha
# line counter from the dns source file, nothing for auto-resuming
178
COUNT=$4
179
180 16 sacha
##########################################################
181 13 sacha
### SOCAT
182 29 sacha
SERVER="SOMEIP:65522"
183 13 sacha
SENDSOCAT="socat stdio openssl-connect:$SERVER,verify=0,cert=$HOMEDIR/mentio_ssl-client.pem,cafile=$HOMEDIR/mentio_ssl-server.crt"
184 16 sacha
##########################################################
185 13 sacha
### COLORS 
186
RED='\e[31m'
187
GREEN='\e[32m'
188
YELLOW='\e[33m'
189 1 sacha
GRAY='\e[90m'
190
NC='\033[0m' # No Color
191 16 sacha
##########################################################
192
### Various variables
193 13 sacha
DNS_SOURCE_BASENAME=`basename $DNS_SOURCE`
194 1 sacha
DIFF_LOG="$HOMEDIR/DNS_DIFF"
195
BLACKLIST_LOG="$HOMEDIR/DNS_BLACKLISTED"
196 13 sacha
lines=`wc -l $DNS_SOURCE|awk -F " " '{print $1}'`
197 4 sacha
countfile="$HOMEDIR/DNS_Count-$DNS_SOURCE_BASENAME"
198 16 sacha
##########################################################
199
### Dig parameters
200 1 sacha
DIG_FAST="+nodnssec +short +timeout=1 +tries=2"
201
DIG_SLOW="+nodnssec +short +timeout=5 +tries=3 "
202 16 sacha
##########################################################
203 13 sacha
204 16 sacha
205
##########################################################
206
### Generate list for dig: round robin from dns list
207
### Like (@DNS-server domain) x parallel 
208 1 sacha
_check(){
209
i=0
210 13 sacha
url=""
211
while [ $i -lt $parallel ]
212 1 sacha
do
213
n=`expr $count + $i`
214
ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1`
215
url="$url @$ISP_DNS `awk -v n="${n}" 'NR==n {print;exit}' $DNS_SOURCE`.$tld"
216
i=`expr $i + 1`
217
done
218
}
219 16 sacha
##########################################################
220 1 sacha
221
222 16 sacha
##########################################################
223
### Counter: create one if not existing, use existing instead
224 1 sacha
if [ -z $COUNT ]; then
225 28 sacha
        if [ -f $countfile ]; then
226
        count=`cat $countfile`
227
        else
228
        count=0
229
        echo $count > $countfile
230
        fi
231 13 sacha
else count=$COUNT
232
echo $count > $countfile
233 1 sacha
fi
234 4 sacha
##########################################################
235 16 sacha
236 4 sacha
237 16 sacha
##########################################################
238
### MAIN LOOP
239
240
while [ "$count" != "$lines" ]; do
241 4 sacha
echo $count > $countfile
242
_check
243 28 sacha
dateus=`date +%Y%m%d-%H%M%S`
244 13 sacha
site="$url"
245
echo "-------------------------------------------------------------------------------"
246
echo "#$count $dateus SITE:$site"
247
if nomentio=`dig @$DNS_MY $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio" ]; then
248 28 sacha
        echo -e "$GRAY Unknown zone $site $NC" 
249 13 sacha
fi
250
if mentio=`dig $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio" ]; then
251 28 sacha
        echo -e "$GRAY Unknown zone $site $NC"
252 1 sacha
fi
253
254
if [ -n "$nomentio" ] && [ -n "$mentio" ]; then
255
    if [ "$nomentio" != "$mentio" ]; then
256
        for i in $site; do
257
                if nomentio1=`dig $DIG_FAST @$DNS_MY $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio1" ]; then
258
                        echo -e "$GRAY Unknown zone $i $NC"
259
                fi
260
                ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1`
261
                if mentio1=`dig $DIG_FAST @$ISP_DNS $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio1" ]; then
262
                        echo -e "$GRAY Unknown zone $i $NC"
263
                fi
264
                        if [ "$nomentio1" != "$mentio1" ]; then
265
                                if [[ $mentio1 == 90.85.* ]]; then
266
                                        if [ $MODE == "client" ]; then
267
                                                echo "!!! $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" | $SENDSOCAT
268
                                        fi
269
                                        echo -e "$RED !!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
270
                                        echo "!!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $BLACKLIST_LOG
271
                                else
272
                                                if [ $MODE == "client" ]; then
273
                                                        echo ">>> $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" | $SENDSOCAT
274
                                                fi                   
275
                                        echo -e "$YELLOW >>> SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
276
                                        echo ">>> $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $DIFF_LOG
277
                                fi
278
                        fi
279
        done
280
    else
281
    echo -e "$GREEN#$count  SITE:$site  $NC"
282
    fi
283
fi
284
285
count=`expr $count + $parallel`
286
287
done
288
##########################################################
289 27 sacha
290 1 sacha
291
292 41 sacha
```
293 1 sacha
294 41 sacha
### mentio-DNS\_ISP\_LIST
295 27 sacha
296 41 sacha
    Bouygues 5410 194.158.122.10 194.158.122.15
297
    Free 12322 212.27.40.240 212.27.40.241 212.27.40.244 212.27.40.245
298
    Orange 3215 80.10.246.1 80.10.246.2 80.10.246.3 80.10.246.5 80.10.246.7 80.10.246.129 80.10.246.130 80.10.246.132 80.10.246.134 80.10.246.136 81.253.149.1 81.253.149.2 81.253.149.6 81.253.149.9 81.253.149.10
299
    SFR 15557 109.0.66.10 109.0.66.20
300 27 sacha
301 41 sacha
### mentio-monitor
302 27 sacha
303 41 sacha
    hosts="mentio-HOSTS"
304
    hosts_tmp="/tmp/mentio-HOSTS.tmp"
305
    rm -rf $hosts
306
    
307
    
308
    GREEN='\e[32m'
309
    RED='\e[31m'
310
    NC='\033[0m' # No Color  
311
    
312
    while true; do
313
            timeout 300 tail -n 0 -f MENTIO-DNS_DIFF | cut -d ' ' -f3|sort|uniq> $hosts_tmp
314
            mentiohosts=`cat $hosts_tmp`
315
            updates=`echo $mentiohosts|xargs -n 1`
316
            for i in $updates; do
317
                    exists=`grep "$i" $hosts`
318
                    if [ -z "$exists" ]; then
319
                            echo "ADD $i"
320
                            echo "$i"  >> $hosts 
321
                            sort -o $hosts $hosts
322
                            name=`echo $i|cut -d "." -f1`
323
                    fi
324
            done
325
    
326
    dateus=`date +%Y%m%d-%H%M%S`
327
    diffs=`diff --side-by-side --suppress-common-lines $hosts $hosts_tmp`
328
    echo "======================================================================"
329
    if [ -z "$diffs" ]; then
330
     echo -e "$GREEN $dateus - ALL HOSTS UP: $NC"
331
     echo -e "$GREEN `cat $hosts |xargs |sort` $NC"  
332
    else
333
     echo -e "$RED $dateus - MISSING HOST: $NC"
334
     echo -e "$RED $diffs $NC"
335
    fi
336
    done
337 18 sacha
338 41 sacha
## Test (valide au 14/06/18)
339 18 sacha
340 41 sacha
dig +short shahamat1.com  
341
90.85.16.52
342 18 sacha
343 41 sacha
## Vigies de la neutralité
344 18 sacha
345 41 sacha
https://ooni.torproject.org  
346 18 sacha
https://respectmynet.eu
347
348 41 sacha
## Cadre légal
349 15 sacha
350 41 sacha
https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164\&fastPos;=2\&fastReqId;=606073666\&categorieLien;=cid\&oldAction;=rechTexte#LEGIARTI000029756525  
351
le décret https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=LEGITEXT000030315036\&dateTexte;=20150305\&categorieLien;=cid#LEGITEXT000030315036  
352
https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=JORFTEXT000030195477\&dateTexte;=20180619
353 15 sacha
354 41 sacha
## Unbound
355 15 sacha
356 41 sacha
    server:
357
     verbosity: 1
358
     interface: 127.0.0.1
359
     do-ip4: yes
360
     do-ip6: no
361
     do-udp: yes
362
     do-tcp: no
363
     access-control: 127.0.0.0/8 allow 
364
     access-control: 0.0.0.0/0 refuse
365
     logfile: /var/log/unbound
366
     hide-identity: yes
367
     hide-version: yes
368
     harden-glue: yes
369
     use-caps-for-id: yes
370
     do-not-query-localhost: yes
371 15 sacha
372 41 sacha
## Lancement
373 15 sacha
374 41 sacha
tmux new-session -s foo -d "bash mentio-check6 client domain\_names.com\_sortedac com" \\; split-window -h "bash mentio-check6 client domain\_names.com\_sortedae com" \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedam com"\\; selectp -t 0 \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedan com"
375 15 sacha
376 41 sacha
## Ansible divers
377 15 sacha
378
Copy file:
379
380 41 sacha
ansible mentio -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"
381 15 sacha
382
Copy file single host:
383
384 41 sacha
ansible mentio ---limit dam -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"
385 15 sacha
386 41 sacha
ansible-playbook /etc/ansible/playbooks/mentio.yml
387 1 sacha
388 41 sacha
``` 
389 1 sacha
---
390
391
- hosts: mentio 
392
  sudo: no
393
  tasks:
394
    - name: copyfiles 
395
      copy:
396
        src: "{{ item.src }}"
397
        dest: "{{ item.dest }}"
398
      with_items:
399
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-check6',dest: '~/MENTIODNS/' }
400
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-DNS_ISP_LIST',dest: '~/MENTIODNS/' }
401
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.crt',dest: '~/MENTIODNS/' }
402
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.key',dest: '~/MENTIODNS/' }
403
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.pem',dest: '~/MENTIODNS/' }
404
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-server.crt',dest: '~/MENTIODNS/' }
405
406 41 sacha
```