Projet

Général

Profil

Mentiodns » Historique » Version 56

sacha, 11/11/2018 10:44

1 41 sacha
# Mentiodns
2 1 sacha
3 54 sacha
reCréation de la liste de censure gérée par le ministère de l'intérieur et envoyée automatiquement aux principaux FAI Français.
4
Validation des DNS à partir d'une liste exhaustive par tld. On vérifie si les résolutions DNS envoient sur les dns menteurs du ministère.
5 1 sacha
6 54 sacha
7 41 sacha
## Noeud actifs
8 1 sacha
9 41 sacha
| Nom       | Bloc                            |
10
| --------- | ------------------------------- |
11
| 1000i100  | domain\_names.com\_sortedab     |
12
| 1000i100  | domain\_names.com\_sortedaj     |
13
| 1000i100  | domain\_names.com\_sortedak     |
14
| 1000i100  | domain\_names.com\_sortedal     |
15
| sacha     | domain\_names.net\_sortedad     |
16 1 sacha
| sacha     | domain\_names.com\_sortedae     |
17
| sacha     | domain\_names.com\_sortedaf     |
18
| sacha     | domain\_names.com\_sortedag     |
19
| sacha     | domain\_names.com\_sortedam     |
20 51 sacha
| tazi      | domain\_names.com\_sortedan     |
21 56 sacha
| tazi      | domain\_names.com\_sortedad     |
22
| tazi      | domain\_names.com\_sortedaq     |
23 47 sacha
| l         | domain\_names.com\_sortedar     |
24 1 sacha
| l         | domain\_names.net\_sortedaa     |
25
| l         | domain\_names.net\_sortedab     |
26
| l         | domain\_names.com\_sortedah     |
27 52 sacha
| l         | domain\_names.com\_sortedai     |
28 56 sacha
| jerem     | domain\_names.com\_sortedao     |
29
| jerem     | domain\_names.com\_sortedap     |
30
31 1 sacha
32
## Traités
33
34 55 sacha
domain\_names.com\_sortedac
35
36 53 sacha
domain\_names.org\_sortedaa 2000000
37
domain\_names.org\_sortedab 2000000
38 18 sacha
domain\_names.org\_sortedac 2000000
39 41 sacha
domain\_names.org\_sortedad 2000000
40 55 sacha
domain\_names.org\_sortedae
41
domain\_names.org\_sortedaf
42
43
domain\_names.net\_sortedac
44
domain\_names.net\_sortedae
45 42 sacha
domain\_names.net\_sortedae 1937733
46 35 sacha
47 41 sacha
## Mentio
48 4 sacha
49 41 sacha
### mentio-check6
50 27 sacha
51 55 sacha
Packages: apt-get install aptitude curl dnsutils python socat tmux unbound whois
52 19 sacha
53 41 sacha
``` 
54 4 sacha
#-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-#
55
#  MENTIODNS : Check for lying DNS (France)  #
56
#--------------------------------------------#
57 28 sacha
#  Version 1.6.1 date bug
58 16 sacha
#  Version 1.6 - conf file                   # 
59 13 sacha
#  Version 1.5 - test Dig resolving          #
60
#  Version 1.4 - Socat SSL sending results   # 
61
#  Version 1.3 - tld optioN                  #
62 1 sacha
#  Version 1.2 - Round robin on DNS_ISP_LIST #
63 28 sacha
#                For each request            #
64 4 sacha
#  Version 1.1 - Allow resume on basename    #
65
#  Version 1.0 - Parallel process with DIG   #
66
#--------------------------------------------#
67 1 sacha
# (c) Sacha at Aquilenet.fr part of FFDN.org #
68
#--------------------------------------------#
69
70
# This shity script intend to bruteforce the ISP lying DNS Servers to identify which one
71
# is going on Ministry of Interior Blocking page and compare the IP result from your favorite DNS server
72 16 sacha
# Use this script with the following parameters 
73
# $1 MODE: client server local
74
# $2 File source: list of domain names whithout tld
75
# $3 tld: com, org, ...
76
# $4 count number (if none from zero or from count file based on file name)
77
78
79 1 sacha
# If you relanch the script it will check if it has a counter for the given file to resume
80
# Blacklisted sites in $BLACKLIST_LOG file
81 13 sacha
# Diff ip from a domain name are in $DIFF_LOG 
82 1 sacha
83 13 sacha
# 1st launch creating config file
84 1 sacha
85 14 sacha
# Copy generated certificates:
86
# FILENAME=mentio_ssl-server 
87 1 sacha
# openssl genrsa -out $FILENAME.key 1024
88 13 sacha
# openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt
89 1 sacha
# cat $FILENAME.key $FILENAME.crt >$FILENAME.pem
90 13 sacha
# FILENAME=mentio_ssl-client
91
# ...
92
93 16 sacha
##########################################################
94 1 sacha
HOMEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
95 13 sacha
MENTIOCONF="$HOMEDIR/mentio.conf"
96 17 sacha
# Number of parallel requests thruw dig
97
parallel=10
98 16 sacha
##########################################################
99 1 sacha
100 16 sacha
101
##########################################################
102 13 sacha
### CHECK CONFIG
103
if [ ! -f $MENTIOCONF ]; then
104
echo "=================================================================="
105
echo "MENTIODNS"
106
echo "------------------------------------------------------------------" 
107
echo "1st time configuring"
108
echo -n "IP UNBOUND ? "
109
read DNS_MY
110 1 sacha
111 13 sacha
echo 'HOMEDIR="'$HOMEDIR'"' > $MENTIOCONF
112
echo 'DNS_MY="'$DNS_MY'"' >> $MENTIOCONF
113 1 sacha
IP_PUB=`curl ifconfig.io`
114 13 sacha
ASN=`whois -h whois.cymru.com $IP_PUB |cut -d' ' -f1|sed -n "2p"`
115 1 sacha
DNS_LIST="$HOMEDIR/mentio-DNS_ISP_LIST"
116
DNS_ISP_LIST=`grep $ASN $DNS_LIST|cut -d' ' -f3-`
117
echo 'DNS_ISP_LIST="'$DNS_ISP_LIST'"' >> $MENTIOCONF
118
echo "------------------------------------------------------------------"
119
echo " CONFIGURATION FILE:"
120
echo " please check and relaunch"
121
echo "------------------------------------------------------------------"
122
cat $MENTIOCONF
123
echo "------------------------------------------------------------------"
124
exit 1
125
fi
126 16 sacha
##########################################################
127 1 sacha
128 16 sacha
129
##########################################################
130
### PARAMETERS to execute the script
131
# Mode Log export with socat "client" "server" "local"
132
133
MODE=$1
134
135
if [ $MODE == "server" ]; then
136
socat -v -u openssl-listen:65522,fork,reuseaddr,cert=mentio_ssl-server.pem,cafile=mentio_ssl-client.crt OPEN:$HOMEDIR/MENTIO-DNS_DIFF,creat,append
137
exit 1
138
fi
139
##########################################################
140
141
142
##########################################################
143
### Check if commandline parameters are less than 3
144
145 1 sacha
if [ $# -lt 3 ]; then
146
echo "=================================================================="
147
echo "MENTIODNS"
148
echo "------------------------------------------------------------------"
149 14 sacha
echo "Missing Parameter, please enter:"
150 13 sacha
echo
151 1 sacha
echo "mentio-check client|server|local filename tld (count number)"
152 13 sacha
echo
153
exit 1
154
fi
155 16 sacha
##########################################################
156 13 sacha
157 16 sacha
158
##########################################################
159
### Get parameters
160
161
# From config file
162 1 sacha
source $MENTIOCONF
163 13 sacha
164 16 sacha
# From command line
165 1 sacha
# $2 DNS source file name
166 13 sacha
DNS_SOURCE=$2
167 1 sacha
# $3 TLD name (com, org...)
168 13 sacha
tld=$3
169 1 sacha
# line counter from the dns source file, nothing for auto-resuming
170
COUNT=$4
171
172 16 sacha
##########################################################
173 13 sacha
### SOCAT
174 29 sacha
SERVER="SOMEIP:65522"
175 13 sacha
SENDSOCAT="socat stdio openssl-connect:$SERVER,verify=0,cert=$HOMEDIR/mentio_ssl-client.pem,cafile=$HOMEDIR/mentio_ssl-server.crt"
176 16 sacha
##########################################################
177 13 sacha
### COLORS 
178
RED='\e[31m'
179
GREEN='\e[32m'
180
YELLOW='\e[33m'
181 1 sacha
GRAY='\e[90m'
182
NC='\033[0m' # No Color
183 16 sacha
##########################################################
184
### Various variables
185 13 sacha
DNS_SOURCE_BASENAME=`basename $DNS_SOURCE`
186 1 sacha
DIFF_LOG="$HOMEDIR/DNS_DIFF"
187
BLACKLIST_LOG="$HOMEDIR/DNS_BLACKLISTED"
188 13 sacha
lines=`wc -l $DNS_SOURCE|awk -F " " '{print $1}'`
189 4 sacha
countfile="$HOMEDIR/DNS_Count-$DNS_SOURCE_BASENAME"
190 16 sacha
##########################################################
191
### Dig parameters
192 1 sacha
DIG_FAST="+nodnssec +short +timeout=1 +tries=2"
193
DIG_SLOW="+nodnssec +short +timeout=5 +tries=3 "
194 16 sacha
##########################################################
195 13 sacha
196 16 sacha
197
##########################################################
198
### Generate list for dig: round robin from dns list
199
### Like (@DNS-server domain) x parallel 
200 1 sacha
_check(){
201
i=0
202 13 sacha
url=""
203
while [ $i -lt $parallel ]
204 1 sacha
do
205
n=`expr $count + $i`
206
ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1`
207
url="$url @$ISP_DNS `awk -v n="${n}" 'NR==n {print;exit}' $DNS_SOURCE`.$tld"
208
i=`expr $i + 1`
209
done
210
}
211 16 sacha
##########################################################
212 1 sacha
213
214 16 sacha
##########################################################
215
### Counter: create one if not existing, use existing instead
216 1 sacha
if [ -z $COUNT ]; then
217 28 sacha
        if [ -f $countfile ]; then
218
        count=`cat $countfile`
219
        else
220
        count=0
221
        echo $count > $countfile
222
        fi
223 13 sacha
else count=$COUNT
224
echo $count > $countfile
225 1 sacha
fi
226 4 sacha
##########################################################
227 16 sacha
228 4 sacha
229 16 sacha
##########################################################
230
### MAIN LOOP
231
232
while [ "$count" != "$lines" ]; do
233 4 sacha
echo $count > $countfile
234
_check
235 28 sacha
dateus=`date +%Y%m%d-%H%M%S`
236 13 sacha
site="$url"
237
echo "-------------------------------------------------------------------------------"
238
echo "#$count $dateus SITE:$site"
239
if nomentio=`dig @$DNS_MY $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio" ]; then
240 28 sacha
        echo -e "$GRAY Unknown zone $site $NC" 
241 13 sacha
fi
242
if mentio=`dig $DIG_SLOW $site|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio" ]; then
243 28 sacha
        echo -e "$GRAY Unknown zone $site $NC"
244 1 sacha
fi
245
246
if [ -n "$nomentio" ] && [ -n "$mentio" ]; then
247
    if [ "$nomentio" != "$mentio" ]; then
248
        for i in $site; do
249
                if nomentio1=`dig $DIG_FAST @$DNS_MY $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$nomentio1" ]; then
250
                        echo -e "$GRAY Unknown zone $i $NC"
251
                fi
252
                ISP_DNS=`echo $DNS_ISP_LIST | xargs -n 1| sort -R | head -n 1`
253
                if mentio1=`dig $DIG_FAST @$ISP_DNS $i|sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4| tr '\r\n' ' '` && [ -z "$mentio1" ]; then
254
                        echo -e "$GRAY Unknown zone $i $NC"
255
                fi
256
                        if [ "$nomentio1" != "$mentio1" ]; then
257
                                if [[ $mentio1 == 90.85.* ]]; then
258
                                        if [ $MODE == "client" ]; then
259
                                                echo "!!! $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC" | $SENDSOCAT
260
                                        fi
261
                                        echo -e "$RED !!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
262
                                        echo "!!! $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $BLACKLIST_LOG
263
                                else
264
                                                if [ $MODE == "client" ]; then
265
                                                        echo ">>> $dateus `hostname` SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" | $SENDSOCAT
266
                                                fi                   
267
                                        echo -e "$YELLOW >>> SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1 $NC"
268
                                        echo ">>> $dateus SITE:$i ISPDNS:$ISP_DNS REAL:$nomentio1 MENTIO:$mentio1" >> $DIFF_LOG
269
                                fi
270
                        fi
271
        done
272
    else
273
    echo -e "$GREEN#$count  SITE:$site  $NC"
274
    fi
275
fi
276
277
count=`expr $count + $parallel`
278
279
done
280
##########################################################
281 27 sacha
282 1 sacha
283
284 41 sacha
```
285 1 sacha
286 41 sacha
### mentio-DNS\_ISP\_LIST
287 27 sacha
288 41 sacha
    Bouygues 5410 194.158.122.10 194.158.122.15
289
    Free 12322 212.27.40.240 212.27.40.241 212.27.40.244 212.27.40.245
290
    Orange 3215 80.10.246.1 80.10.246.2 80.10.246.3 80.10.246.5 80.10.246.7 80.10.246.129 80.10.246.130 80.10.246.132 80.10.246.134 80.10.246.136 81.253.149.1 81.253.149.2 81.253.149.6 81.253.149.9 81.253.149.10
291
    SFR 15557 109.0.66.10 109.0.66.20
292 27 sacha
293 41 sacha
### mentio-monitor
294 27 sacha
295 41 sacha
    hosts="mentio-HOSTS"
296
    hosts_tmp="/tmp/mentio-HOSTS.tmp"
297
    rm -rf $hosts
298
    
299
    
300
    GREEN='\e[32m'
301
    RED='\e[31m'
302
    NC='\033[0m' # No Color  
303
    
304
    while true; do
305
            timeout 300 tail -n 0 -f MENTIO-DNS_DIFF | cut -d ' ' -f3|sort|uniq> $hosts_tmp
306
            mentiohosts=`cat $hosts_tmp`
307
            updates=`echo $mentiohosts|xargs -n 1`
308
            for i in $updates; do
309
                    exists=`grep "$i" $hosts`
310
                    if [ -z "$exists" ]; then
311
                            echo "ADD $i"
312
                            echo "$i"  >> $hosts 
313
                            sort -o $hosts $hosts
314
                            name=`echo $i|cut -d "." -f1`
315
                    fi
316
            done
317
    
318
    dateus=`date +%Y%m%d-%H%M%S`
319
    diffs=`diff --side-by-side --suppress-common-lines $hosts $hosts_tmp`
320
    echo "======================================================================"
321
    if [ -z "$diffs" ]; then
322
     echo -e "$GREEN $dateus - ALL HOSTS UP: $NC"
323
     echo -e "$GREEN `cat $hosts |xargs |sort` $NC"  
324
    else
325
     echo -e "$RED $dateus - MISSING HOST: $NC"
326
     echo -e "$RED $diffs $NC"
327
    fi
328
    done
329 18 sacha
330 41 sacha
## Test (valide au 14/06/18)
331 18 sacha
332 41 sacha
dig +short shahamat1.com  
333
90.85.16.52
334 18 sacha
335 41 sacha
## Vigies de la neutralité
336 18 sacha
337 41 sacha
https://ooni.torproject.org  
338 18 sacha
https://respectmynet.eu
339
340 41 sacha
## Cadre légal
341 15 sacha
342 41 sacha
https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164\&fastPos;=2\&fastReqId;=606073666\&categorieLien;=cid\&oldAction;=rechTexte#LEGIARTI000029756525  
343
le décret https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=LEGITEXT000030315036\&dateTexte;=20150305\&categorieLien;=cid#LEGITEXT000030315036  
344
https://www.legifrance.gouv.fr/affichTexte.do;jsessionid=FE6BFDED672BF1E2EFC5CA70705CF26E.tplgfr21s_3?cidTexte=JORFTEXT000030195477\&dateTexte;=20180619
345 15 sacha
346 41 sacha
## Unbound
347 15 sacha
348 41 sacha
    server:
349
     verbosity: 1
350
     interface: 127.0.0.1
351
     do-ip4: yes
352
     do-ip6: no
353
     do-udp: yes
354
     do-tcp: no
355
     access-control: 127.0.0.0/8 allow 
356
     access-control: 0.0.0.0/0 refuse
357
     logfile: /var/log/unbound
358
     hide-identity: yes
359
     hide-version: yes
360
     harden-glue: yes
361
     use-caps-for-id: yes
362
     do-not-query-localhost: yes
363 15 sacha
364 41 sacha
## Lancement
365 15 sacha
366 41 sacha
tmux new-session -s foo -d "bash mentio-check6 client domain\_names.com\_sortedac com" \\; split-window -h "bash mentio-check6 client domain\_names.com\_sortedae com" \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedam com"\\; selectp -t 0 \\; split-window -v "bash mentio-check6 client domain\_names.com\_sortedan com"
367 15 sacha
368 41 sacha
## Ansible divers
369 15 sacha
370
Copy file:
371
372 41 sacha
ansible mentio -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"
373 15 sacha
374
Copy file single host:
375
376 41 sacha
ansible mentio ---limit dam -m copy -a "src=mentio-check6 dest=~/MENTIODNS/"
377 15 sacha
378 41 sacha
ansible-playbook /etc/ansible/playbooks/mentio.yml
379 1 sacha
380 41 sacha
``` 
381 1 sacha
---
382
383
- hosts: mentio 
384
  sudo: no
385
  tasks:
386
    - name: copyfiles 
387
      copy:
388
        src: "{{ item.src }}"
389
        dest: "{{ item.dest }}"
390
      with_items:
391
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-check6',dest: '~/MENTIODNS/' }
392
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio-DNS_ISP_LIST',dest: '~/MENTIODNS/' }
393
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.crt',dest: '~/MENTIODNS/' }
394
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.key',dest: '~/MENTIODNS/' }
395
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-client.pem',dest: '~/MENTIODNS/' }
396
        - { src: '/home/sacha/0nmyway/00_Aquilenet/FFDN/mentiodns.fr/mentio_ssl-server.crt',dest: '~/MENTIODNS/' }
397
398 41 sacha
```