Projet

Général

Profil

Librehosting » Historique » Version 35

sacha, 02/08/2018 10:47

1 27 openbeelab
# Configuration of a Libre Hosting aka Anouncing our public IPs to Internet from an indifferent place
2 1 sacha
3 13 sacha
technical resume: IPSec between OPNSense and OpenBSD to announce our ASN IP from an indifferent ISP place. 
4 1 sacha
5 13 sacha
## General presentation
6 1 sacha
7 13 sacha
### Who we are
8
9 32 sacha
[Aquilenet](https://www.aquilenet.fr) is a non profit organization since 2010 and a "do it yourself ISP", member of a Federation of similar ISP in France called [FFDN](https://www.ffdn.org). We are netneutrality builders, helping for more freedom and building networks using and participating Libre Software.
10 1 sacha
We provide xDSL, VPN and we hope soon Fiber accesses, and a lot of services for our members (mail, nextcloud, hosting, VPS...) and for others [searx](https://searx.aquilenet.fr), [Etherpad](https://pad.aquilenet.fr), [Pastebin](https://pastebin.aquilenet.fr), [Peertube](https://tube.aquilenet.fr), ...
11
12 13 sacha
### The need: create a Libre format hosting in our cool local called "la mezzanine".
13 1 sacha
14 12 sacha
To allow our the members of our non-profit organization to connect the hardware they want (like Nuc, Raspberry pi, tower, etc...) in our Libre hosting space called "la mezzanine" with a Public IPv6 and IPv4 and from our [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)), we need to announce our IPs from another place than our Datacenter.
15 1 sacha
16 13 sacha
### How we do
17 1 sacha
18 12 sacha
We have to tunnel all the network from the Libre Hosting to Internet and vice versa. We have tried first with OpenVPN but the userland application use to much resources for needed bandwidth (200Mbps).
19 28 sacha
This IP range will be routed by IPSec to our [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)) point of [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol) announce in our datacenter, then they will route them to the Libre Hosting.
20 1 sacha
21 29 sacha
From our Libre Hosting we have a [OPNSense](https://opnsense.org/) Firewall and in our Datacenter two clustered [OpenBSD](https://www.openbsd.org/).
22 1 sacha
23 27 openbeelab
A diagram to explain this:
24 1 sacha
25
![](https://atelier.aquilenet.fr/attachments/download/550/Aquilenet-IPSec-Logical_Scheme.png)
26
27 27 openbeelab
Diagram explanation:
28 1 sacha
29 28 sacha
* (1) [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)) & [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol)
30 1 sacha
31 28 sacha
With our [ASN](https://en.wikipedia.org/wiki/Autonomous_system_(Internet)) with can do BGP to announce our IP adresses in our Datacenter.
32 12 sacha
33
* (2) Firewalls in the Datacenter
34
35
The job is done with a cluster of firewalls using the magic power of [OpenBSD](https://www.openbsd.org/) and [OpenBGPD](http://www.openbgpd.org/).
36 27 openbeelab
For firewall redundancy we use [CARP and pfsync](https://www.openbsd.org/faq/pf/carp.html)
37 12 sacha
38
* (3) La Mezzanine
39
40
Is our cool place where we can meet our members and friends.
41 31 sacha
This place use green power with a supplier called [Enercoop](https://en.wikipedia.org/wiki/Enercoop), it's a French electric utility cooperative company, which only uses renewable energy.
42 33 sacha
Here, the firewall is a special hardware to afford filter & IPSec on 500Mbps [OPNSense](https://opnsense.org/), we choose an hardware with [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) and here we have a Qotom-Q355G4 Fanless X86Intel ® Core ™i5-5250Uprocessor coming from Shenzen. He is doing the PPPOE connection on a general public fiber provider (Yes we had to hack a little to remove this ISP blackbox).
43 1 sacha
44
* (4) Public IPv6 and v4 from the Mezzanine to Internet
45 12 sacha
46 27 openbeelab
This firewall is routing threw an IPSec tunnel (for each OpenBSD firewall) our public dedicated IP for this place. These IP allow our friend's to put there their servers / Arduino's & co to Internet with our network <3
47 12 sacha
48 13 sacha
---
49
50 14 sacha
## Technical information, let's rock !
51 12 sacha
52 35 sacha
We configure IPSec tunnels with 
53
54
* Phase 1: IKEv1 AES (192 bits) + SHA1 + DH Group 14 (2048 bits)
55
* Phase 2: ESP IPv4 tunnel ES (256 bits) SHA512 Group 18 (8192 bits) 
56
57 12 sacha
### [OpenBSD](https://www.openbsd.org/) configuration
58
59
We configure the [OpenBSD](https://www.openbsd.org/)'s IPSec configuration file for 2 Phase 2 tunnels one for IPv4 another on for IPv6
60 3 sacha
61 15 sacha
**/etc/ipsec.conf**
62 1 sacha
63
~~~
64 3 sacha
ike passive esp from any to $LIBRE_HOSTING_PUB_IP_V4/26 \
65
    peer $OPSense_Public_IP\
66 1 sacha
    main group modp2048\
67 17 sacha
    psk "My super secret shared key"
68 1 sacha
69 3 sacha
flow esp from any to $LIBRE_HOSTING_PUB_IP_V4/26 peer $OPSense_Public_IP
70
flow esp from any to $LIBRE_HOSTING_PUB_IP_V6/48 peer $OPSense_Public_IP
71 1 sacha
~~~
72
73 7 sacha
~~~
74
chmod 500 /etc/ipsec.conf
75
~~~
76 1 sacha
77 15 sacha
**Launch the tunnel:**
78 1 sacha
79 7 sacha
~~~
80
isakmpd -K  
81
ipsecctl -f /etc/ipsec.conf
82
~~~
83 1 sacha
84 9 sacha
To make it permanent add this to /etc/rc.conf.local:
85 4 sacha
86 3 sacha
~~~
87 1 sacha
isakmpd_flags="-K"
88
ipsec_rules=/etc/ipsec.conf
89
ipsec=YES
90
~~~
91
92 7 sacha
check the configuration:
93 1 sacha
94 7 sacha
~~~
95
ipsecctl -sa 
96
~~~
97
98 8 sacha
Enable some logs on Isakmpd:
99
100 7 sacha
~~~
101 5 sacha
sh -c "echo S \> /var/run/isakmpd.fifo"  
102 1 sacha
less /var/run/isakmpd.result
103
~~~
104
105 8 sacha
and:
106 1 sacha
107 8 sacha
~~~
108 1 sacha
isakmpd -d -DA=70 -K
109 8 sacha
~~~
110 1 sacha
111 8 sacha
Check IPSec Flows:
112 1 sacha
113 8 sacha
~~~
114 1 sacha
ipsecctl -F
115 8 sacha
~~~
116
117 29 sacha
## [OPNSense](https://opnsense.org/) configuration
118 1 sacha
119 20 sacha
* **IPSec Phase 1**
120 24 sacha
121 34 sacha
![](https://atelier.aquilenet.fr/attachments/download/556/OPNSense-Tunnel_Settings_IPsec_VPN-Phase1.png)
122 19 sacha
123 20 sacha
* **IPSec Phase 2 IPv4**
124 24 sacha
125 18 sacha
![](https://atelier.aquilenet.fr/attachments/download/553/OPNSense-Tunnel_Settings_IPsec-Phase2-IPv4.png)
126 19 sacha
127 20 sacha
* **IPSec Phase 2 IPv6**
128 24 sacha
129 23 sacha
To have an IPv6 phase 2 with a IPv4 phase on Opnsense ensure you got [this patch](https://github.com/opnsense/core/commit/a79b20c12759007f2079ffd6e0cad26d04b00808)
130 20 sacha
![](https://atelier.aquilenet.fr/attachments/download/554/OPNSense-Tunnel_Settings_IPsec-Phase2-IPv6.png)
131 21 sacha
132
* **IPSec Advanced Settings**
133
134 30 sacha
Important exclude the routed network to allow it to talk to itself (exemple ARP/ICMP request to the gateway) 
135 21 sacha
![](https://atelier.aquilenet.fr/attachments/download/555/OPNSense-Tunnel_Settings_IPsec-Advanced_Settings.png)
136 25 sacha
137 26 sacha
## Hosted server settings
138 25 sacha
139
IP: The public IP/Mask
140 29 sacha
Gateway: OPNSense's IP
141 1 sacha
mtu 1378
142 26 sacha
143
---