Projet

Général

Profil

Librehosting » Historique » Version 38

sacha, 03/08/2018 14:14

1 38 sacha
# Configuration of a Libre Hosting aka Announcing our public IPs to the Internets from an any place place
2 1 sacha
3 38 sacha
In technical terms: IPSec between OPNSense and OpenBSD to announce our ASN IP from an any place  ISP place.
4 1 sacha
5 13 sacha
## General presentation
6 1 sacha
7 13 sacha
### Who we are
8
9 37 xavier
[Aquilenet](https://www.aquilenet.fr) is a non profit organization since 2010 and a "do it yourself ISP", member of a Federation of similar ISP in France called [FFDN](https://www.ffdn.org). We are netneutrality builders, helping for more freedom and building networks using and participating to Libre Software.
10
We provide xDSL, VPN and we hope soon Fiber access, and a lot of services to our members (mail, nextcloud, hosting, VPS...) and others [searx](https://searx.aquilenet.fr), [Etherpad](https://pad.aquilenet.fr), [Pastebin](https://pastebin.aquilenet.fr), [Peertube](https://tube.aquilenet.fr), ...
11 1 sacha
12 37 xavier
### The need: to create a Libre format hosting at our cool place, "la mezzanine".
13 1 sacha
14 37 xavier
To allow members of our non-profit organization to locate the hardware they want (like Nuc, Raspberry pi, towers, etc...) at our Libre hosting space called "la mezzanine" with a Public IPv6 and IPv4 from our [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)), we need to announce our IPs from another place than our data center.
15 12 sacha
16 37 xavier
### How we do it
17 13 sacha
18 37 xavier
We have to tunnel all of our Libre Hosting networks to the Internets, and vice versa. This IP range, assigned to our [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)), is routed from our point of [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol) announce in our data center, to the Libre Hosting.
19 12 sacha
20 37 xavier
We first tried OpenVPN but its userland application consumed too much resources for the required bandwidth (200Mbps). We use IPSec.
21 1 sacha
22 37 xavier
From our Libre Hosting we have an [OPNSense](https://opnsense.org/) Firewall and in our data center, two clustered [OpenBSD](https://www.openbsd.org/).
23
24 38 sacha
A diagram to illustrate this:
25 1 sacha
26
![](https://atelier.aquilenet.fr/attachments/download/550/Aquilenet-IPSec-Logical_Scheme.png)
27
28 27 openbeelab
Diagram explanation:
29 1 sacha
30
* (1) [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)) & [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol)
31
32 37 xavier
With our [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)) with can do BGP to announce our IP addresses in our data center.
33 12 sacha
34 37 xavier
* (2) Firewalls in the data center
35 1 sacha
36 37 xavier
The job is done with a cluster of firewalls using the ®magic power© of [OpenBSD](https://www.openbsd.org/) and [OpenBGPD](http://www.openbgpd.org/).
37 38 sacha
For firewall redundancy we use [CARP and pfsync](https://www.openbsd.org/faq/pf/carp.html).
38 12 sacha
39 1 sacha
* (3) La Mezzanine
40 12 sacha
41 37 xavier
La Mezzanine is our cool place where we meet up with other members and friends.
42
It has green power, thanks to the folks at a French electric utility cooperative company, [Enercoop](https://en.wikipedia.org/wiki/Enercoop), who supplies only renewable energy.
43
Here, the firewall is a dedicated hardware running [OPNSense](https://opnsense.org/), than can afford filters & IPSec on 500Mbps. We chose an appliance with [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) support, a Qotom-Q355G4 Fanless X86 with an Intel® Core™ i5-5250U processor, coming from Shenzen. It also deals with the PPPoE connection on a general public fiber provider (yes--we had to hack a little to remove this ISP blackbox).
44 1 sacha
45
* (4) Public IPv6 and v4 from the Mezzanine to Internet
46 12 sacha
47 37 xavier
This firewall is routing, through an IPSec tunnel (for each OpenBSD firewall), our public IP address range, dedicated to this place. These IP addresses allow for our friends to locate their servers / Arduino's & co right here, on the Internet within our network <3
48 12 sacha
49 13 sacha
---
50
51 14 sacha
## Technical information, let's rock !
52 12 sacha
53 37 xavier
We configure IPSec with **2** × Phase-2 tunnels: one for IPv4, another for IPv6:
54 35 sacha
55 37 xavier
* Phase-1: IKEv1 AES (192 bits) + SHA1 + DH Group 14 (2048 bits)
56
* Phase-2: ESP IPv4 tunnel AES (256 bits) SHA512 Group 18 (8192 bits)
57
* Phase-2: ESP IPv6 tunnel AES (256 bits) SHA512 Group 18 (8192 bits) 
58 35 sacha
59 12 sacha
### [OpenBSD](https://www.openbsd.org/) configuration
60
61 37 xavier
We define [OpenBSD](https://www.openbsd.org/)'s IPSec configuration file:
62 3 sacha
63 15 sacha
**/etc/ipsec.conf**
64 1 sacha
65
~~~
66 3 sacha
ike passive esp from any to $LIBRE_HOSTING_PUB_IP_V4/26 \
67
    peer $OPSense_Public_IP\
68 1 sacha
    main group modp2048\
69 17 sacha
    psk "My super secret shared key"
70 1 sacha
71
flow esp from any to $LIBRE_HOSTING_PUB_IP_V4/26 peer $OPSense_Public_IP
72 3 sacha
flow esp from any to $LIBRE_HOSTING_PUB_IP_V6/48 peer $OPSense_Public_IP
73 1 sacha
~~~
74
75 7 sacha
~~~
76 1 sacha
chmod 500 /etc/ipsec.conf
77 7 sacha
~~~
78 1 sacha
79 15 sacha
**Launch the tunnel:**
80 1 sacha
81 7 sacha
~~~
82
isakmpd -K  
83
ipsecctl -f /etc/ipsec.conf
84
~~~
85 1 sacha
86 9 sacha
To make it permanent add this to /etc/rc.conf.local:
87 4 sacha
88 1 sacha
~~~
89
isakmpd_flags="-K"
90
ipsec_rules=/etc/ipsec.conf
91
ipsec=YES
92
~~~
93
94 37 xavier
To verify the configuration:
95 1 sacha
96 7 sacha
~~~
97
ipsecctl -sa 
98 1 sacha
~~~
99 7 sacha
100 37 xavier
Enable some logging on Isakmpd:
101 1 sacha
102
~~~
103 5 sacha
sh -c "echo S \> /var/run/isakmpd.fifo"  
104 1 sacha
less /var/run/isakmpd.result
105
~~~
106 8 sacha
107 1 sacha
and:
108 8 sacha
109 1 sacha
~~~
110 8 sacha
isakmpd -d -DA=70 -K
111 1 sacha
~~~
112 8 sacha
113 37 xavier
Check IPSec flows:
114 8 sacha
115 1 sacha
~~~
116 8 sacha
ipsecctl -F
117
~~~
118 29 sacha
119 1 sacha
## [OPNSense](https://opnsense.org/) configuration
120 20 sacha
121 37 xavier
* **IPSec Phase-1**
122 34 sacha
123 19 sacha
![](https://atelier.aquilenet.fr/attachments/download/556/OPNSense-Tunnel_Settings_IPsec_VPN-Phase1.png)
124 20 sacha
125 37 xavier
* **IPSec Phase-2, IPv4**
126 18 sacha
127 19 sacha
![](https://atelier.aquilenet.fr/attachments/download/553/OPNSense-Tunnel_Settings_IPsec-Phase2-IPv4.png)
128 20 sacha
129 37 xavier
* **IPSec Phase-2, IPv6**
130 23 sacha
131 37 xavier
To get an IPv6 Phase-2 working with a IPv4 Phase-1 on Opnsense, better make sure you got [this patch](https://github.com/opnsense/core/commit/a79b20c12759007f2079ffd6e0cad26d04b00808)
132 21 sacha
![](https://atelier.aquilenet.fr/attachments/download/554/OPNSense-Tunnel_Settings_IPsec-Phase2-IPv6.png)
133
134
* **IPSec Advanced Settings**
135 30 sacha
136 37 xavier
**Important!** On the LAN side, exclude the traffic targeted at the routed network from the tunnel, so *all* of its hosts may keep talking and responding to each other (example: ARP/ICMP request to the gateway)! Otherwise that traffic will go die at the other end of the IPSec tunnel (and the whole Planet could be endangered, too. Who knows?).![](https://atelier.aquilenet.fr/attachments/download/555/OPNSense-Tunnel_Settings_IPsec-Advanced_Settings.png)
137 25 sacha
138 26 sacha
## Hosted server settings
139 25 sacha
140
IP: The public IP/Mask
141 37 xavier
Gateway: OPNSense IP
142 1 sacha
mtu 1378
143 26 sacha
144
---