Projet

Général

Profil

Librehosting » Historique » Version 13

sacha, 02/08/2018 00:43

1 13 sacha
# Configuration of a Libre Hosting aka Anouncing our public IPs to Internet from an indifferent places
2 1 sacha
3 13 sacha
technical resume: IPSec between OPNSense and OpenBSD to announce our ASN IP from an indifferent ISP place. 
4 1 sacha
5 13 sacha
## General presentation
6 1 sacha
7 13 sacha
### Who we are
8
9 12 sacha
Aquilenet is a non profit organization since 2010 and a "do it yourself ISP", member of a Federation of similar ISP in France called [FFDN](https://www.ffdn.org). We are netneutrality builders, helping for more freedom and building networks using and participating Libre Software.
10 1 sacha
We provide xDSL, VPN and we hope soon Fiber accesses, and a lot of services for our members (mail, nextcloud, hosting, VPS...) and for others [searx](https://searx.aquilenet.fr), [Etherpad](https://pad.aquilenet.fr), [Pastebin](https://pastebin.aquilenet.fr), [Peertube](https://tube.aquilenet.fr), ...
11
12 13 sacha
### The need: create a Libre format hosting in our cool local called "la mezzanine".
13 1 sacha
14 12 sacha
To allow our the members of our non-profit organization to connect the hardware they want (like Nuc, Raspberry pi, tower, etc...) in our Libre hosting space called "la mezzanine" with a Public IPv6 and IPv4 and from our [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)), we need to announce our IPs from another place than our Datacenter.
15 1 sacha
16 13 sacha
### How we do
17 1 sacha
18 12 sacha
We have to tunnel all the network from the Libre Hosting to Internet and vice versa. We have tried first with OpenVPN but the userland application use to much resources for needed bandwidth (200Mbps).
19
This IP range will be routed by IPSec to our ASN point of BGP announce in our datacenter, then they will route them to the Libre Hosting.
20 1 sacha
21 12 sacha
From our Libre Hosting we have a OPNSense Firewall and in our Datacenter two clustered [OpenBSD](https://www.openbsd.org/).
22 1 sacha
23
A scheme to explain this:
24
25
![](https://atelier.aquilenet.fr/attachments/download/550/Aquilenet-IPSec-Logical_Scheme.png)
26
27 12 sacha
Scheme explanation:
28 1 sacha
29 12 sacha
* (1) ASN & BGP
30 1 sacha
31 13 sacha
With our ASN with can do BGP to announce our IP adresses in our Datacenter.
32 12 sacha
33
* (2) Firewalls in the Datacenter
34
35
The job is done with a cluster of firewalls using the magic power of [OpenBSD](https://www.openbsd.org/) and [OpenBGPD](http://www.openbgpd.org/).
36 13 sacha
For firewall Redundancy we use [CARP and pfsync](https://www.openbsd.org/faq/pf/carp.html)
37 12 sacha
38
* (3) La Mezzanine
39
40
Is our cool place where we can meet our members and friends.
41
This place use green power with an alternate electricity supplier, called [Enercoop](https://en.wikipedia.org/wiki/Enercoop) is a French electric utility cooperative company, which only uses renewable energy.
42 1 sacha
Here, the firewall is [OPNSense](https://opnsense.org/) doing the PPPOE connection on a general public fiber provider (Yes we had to hack a little to remove this ISP baclkbox).
43
44
* (4) Public IPv6 and v4 from the Mezzanine to Internet
45 12 sacha
46
This firewall is routing threw an IPSec tunnel (for each OpenBSD firewall) our public dedicated IP for this place. These IP allow our friend's to put there their servers servers / Arduino's & co to Internet with our network <3
47
48 13 sacha
---
49
50
## Technical information
51 12 sacha
52
### [OpenBSD](https://www.openbsd.org/) configuration
53
54
We configure the [OpenBSD](https://www.openbsd.org/)'s IPSec configuration file for 2 Phase 2 tunnels one for IPv4 another on for IPv6
55 3 sacha
56 1 sacha
/etc/ipsec.conf
57
58
~~~
59 3 sacha
ike passive esp from any to $LIBRE_HOSTING_PUB_IP_V4/26 \
60
    peer $OPSense_Public_IP\
61 1 sacha
    main group modp2048\
62 3 sacha
    psk "mysupersecurepass"
63 1 sacha
64 3 sacha
flow esp from any to $LIBRE_HOSTING_PUB_IP_V4/26 peer $OPSense_Public_IP
65
flow esp from any to $LIBRE_HOSTING_PUB_IP_V6/48 peer $OPSense_Public_IP
66 1 sacha
~~~
67
68 7 sacha
~~~
69
chmod 500 /etc/ipsec.conf
70
~~~
71 1 sacha
72
Launch the tunnel:
73
74 7 sacha
~~~
75
isakmpd -K  
76
ipsecctl -f /etc/ipsec.conf
77
~~~
78 1 sacha
79 9 sacha
To make it permanent add this to /etc/rc.conf.local:
80 4 sacha
81 3 sacha
~~~
82 1 sacha
isakmpd_flags="-K"
83
ipsec_rules=/etc/ipsec.conf
84
ipsec=YES
85
~~~
86
87 7 sacha
check the configuration:
88 1 sacha
89 7 sacha
~~~
90
ipsecctl -sa 
91
~~~
92
93 8 sacha
Enable some logs on Isakmpd:
94
95 7 sacha
~~~
96 5 sacha
sh -c "echo S \> /var/run/isakmpd.fifo"  
97 1 sacha
less /var/run/isakmpd.result
98
~~~
99
100 8 sacha
and:
101 1 sacha
102 8 sacha
~~~
103 1 sacha
isakmpd -d -DA=70 -K
104 8 sacha
~~~
105 1 sacha
106 8 sacha
Check IPSec Flows:
107 1 sacha
108 8 sacha
~~~
109 1 sacha
ipsecctl -F
110 8 sacha
~~~
111
112 9 sacha
## OPNSense configuration
113 1 sacha
114
![](IPSec-Phase2_Conf1.png) ![](IPSec-Phase1_Conf1.png)
115
116
### \\-\> Configuration1: Performances
117
118
  - AES\_CBC\_256/HMAC\_SHA2\_512\_256/MODP\_8192
119
120
  - Cerbere11
121
122
last pid: 45038; load averages: 0.27, 0.34, 0.31 up 0+00:56:47 19:07:15  
123
49 processes: 1 running, 48 sleeping  
124
CPU 0: 3.9% user, 0.0% nice, 0.8% system, 3.1% interrupt, 92.1% idle  
125
CPU 1: 1.2% user, 0.0% nice, 0.0% system, 6.3% interrupt, 92.5% idle  
126
CPU 2: 0.4% user, 0.0% nice, 0.0% system, 21.3% interrupt, 78.3% idle  
127
CPU 3: 5.9% user, 0.0% nice, 0.0% system, 2.4% interrupt, 91.7% idle  
128
Mem: 109M Active, 145M Inact, 387M Wired, 152M Buf, 7214M Free
129
130
  - Cerbere1
131
132
load averages: 1.18, 0.95, 0.74 cerbere1.aquilenet.fr 19:08:23  
133
43 processes: 1 starting, 40 idle, 1 dead, 1 on processor up 3 days, 18:40 up 3 days, 19:21  
134
CPU0 states: 0.2% user, 0.0% nice, 2.8% system, 7.4% interrupt, 89.6% idle  
135
CPU1 states: 0.0% user, 0.0% nice, 22.4% system, 0.0% interrupt, 77.6% idle  
136
CPU2 states: 0.0% user, 0.0% nice, 20.2% system, 0.0% interrupt, 79.8% idle  
137
CPU3 states: 0.0% user, 0.0% nice, 10.8% system, 0.0% interrupt, 89.2% idle  
138
Memory: Real: 584M/2111M act/tot Free: 5793M Cache: 754M Swap: 0K/8405M
139
140
  - bande passante
141
142
![](IPSec_BW1.png)
143
144
### MTU
145
146
Linux: ping -M do -s 1172 185.233.102.130 =\> OK  
147
Linux: iperf3 -c 185.233.102.130 -M 1160 =\> OK  
148
Linux: ping -M do -s 1173 185.233.102.130 =\> NOK  
149
Linux: iperf3 -c 185.233.102.130 -M 1161 =\> NOK
150
151
  - netcat qui fait un echo  
152
    socat TCP4-LISTEN:4444,fork EXEC:cat