Selfhosting Reloaded

Recently I’ve been thinking a lot about hosting.
I’ve been running and managing hosted services for quite a while now,
ranging from simple stuff like Pleroma, to recent more complex additions like E-Mail and DNS.
One thing was however bugging me for quite a while, and I never had a really good solution for it.

* How do I get rid of “the cloud”? *

Yes, I know, the answer is “obvious”: “Just host at home”.
I’m however not one of those lucky people who can call a VDSL or DOCSIS connection their own.
Here in $RURAL_CITY, we’ve been surfing with ADSL2+ for the last 8 years, with no upgrade in sight.
This means about 14Mbit/s of downstream and 1Mbit/s of upstream on a good day.

“Just use LTE” you say?
Yes that could work, and I actually have a matching router and contract.
The connection is however, uhhh, let’s call it “under-average”.
When it works and we happen to be outside of the main usage window it’s neat,
but it frequently drops out and takes what feels like an eternity to reconnect.

With all of that in mind, hosting something from home seems like a lost cause.

Well fortunately I do have an idea how to fix this,
so join me on my journey of making rural internet suck less - one packet at a time.

The Master-Plan

A picture says more than a thousand words, so without further ado here’s our end-goal:

 +----------+                                        +-------------+
 |Homeserver|                          +-----------> |You are here!|
 +----------+                          |             +-------------+
    |                             +---------+
    |    +-----------+            |Cheap VPS|
    +--> |Edge Router|            +---------+
         +-----------+                 ^
          |         |                  |
          v         v             +--------+
     +----------+ +----------+    |Internet|
     |DSL Router| |LTE Router|    +--------+
     +----------+ +----------+         ^
          |            |               |
          v            v               |
       +-----+      +-----+            |
       |ISP A|      |ISP B|            |
       +-----+      +-----+            |
          |            |               |
          +------------+---------------+

Homeserver is a very standard PC, in fact it’s my legacy “gaming rig”.
Nothing special there really, except that it’s WAY more beefy than my previous Hetzner root.
You can use anything there though, even your old RaspberryPi which you bought
years ago for “getting into hardware”, only to let it collect dust ever since.
ahem, moving on.

The job of Edge Router is to figure out which ISP is less shitty at the moment,
and transparently direct our traffic through the best possible path.
You really don’t need a lot of power here. I managed to pull this off with my old and
insanely underpowered Netgear WNR22001 running OpenWRT2, MWAN33, and VLAN.
I can however wholeheartedly recommend taking a good look at Ubiquiti’s EdgeRouter X4 instead.
No matter what you choose, getting this running is comparably easy so I’ll only give you a quick
rundown of the rough steps instead of a full tutorial.

OpenWRT

If you choose OpenWRT you need MWAN3 and some VLAN magic.
Normal routers usually have one WAN port and a 4-port LAN switch.
We can however address those seperately on OpenWRT, so let’s do that!

Go into your VLAN configuration, and set it up like this:

| VLAN ID | CPU    | LAN1     | LAN2     | LAN3     | LAN4     |
| 1       | tagged | untagged | off      | off      | off      |
| 2       | tagged | off      | untagged | off      | off      |
| 3       | tagged | off      | off      | untagged | untagged |

Gz you just configured your router to have LAN1 and LAN2 act as new WAN ports.
LAN3 and LAN4 will act as a 2-port mini LAN switch for your home network.

Set up LAN1 and LAN2 as WAN interfaces with corresponding firewall zones.
Then connect them to the DSL and LTE routers.
The OpenWRT wiki5 has lots of docs on that topic, so I’ll leave it as an excercise to the reader.
After this works, install the MWAN3 package and configure it according to the wiki page6.

Ubiquiti

If you succumb to Ubiquiti, use their Load Balancing wizard.
Make sure to check Only if the other fails on the second port.
Click apply.
Yeah really that’s it.

Getting an IP

We’ve got our load-balanced connection down, neat!, but double NAT doesn’t make hosting exactly easy.
Not to mention the carrier-grade NAT my LTE provider performs with no way to break out.

Wouldn’t it be a nice if we had a way to have a publicly reachable IP that never changes?
And also have it directly mapped to our PC here at home?
That’s where the VPS comes in.

+----+    +--------+    +-----+    +----------------------+    +----------------------+    +--+
|User| -> |Internet| -> | VPS | -> |Current non-shitty ISP| -> |The Trinity of Routers| -> |PC|
+----+    +--------+    +-----+    +----------------------+    +----------------------+    +--+

Buy the smallest VM your provider of choice has to offer, and add a “floating IP” or “extra IP”.
On Hetzner this will set you back about $4 per month.
Then we simply remap $FLOATING_IP to the machine at home by NATing it through a WireGuard tunnel7.

Here’s what neeeds to be done:

#!/bin/bash

set -eux

mode="$1"
iface="$2"

act=""
if [[ "$mode" == "up" ]]; then
    act="-A"
elif [[ "$mode" == "down" ]]; then
    act="-D"
else
    exit 1
fi

# Allow forwarding traffic from/to the wireguard interface
iptables $act FORWARD -i "$iface" -j ACCEPT
iptables $act FORWARD -o "$iface" -j ACCEPT

# Forward all traffic arriving at the floating IP to the homeserver
iptables -t nat $act PREROUTING -d 5.6.7.8 -j DNAT --to-destination 10.10.32.2

# Act as if outgoing traffic of the homeserver came from the floating IP
iptables -t nat $act POSTROUTING -s 10.10.32.2 -j SNAT --to-source 5.6.7.8

End-Result

Well then, does it actually work?
You tell me.

The blog you’re reading right now - among other things - is hosted on the mentioned PC,
routed through this exact setup to a Hetzner VPS.
From every conceivable angle, it’s as if that PC was in a datacenter, instead of under my desk.

Now of course this setup doesn’t turn my upstream into 1Gbit/s optic fibre.
If the EdgeRouter selects the LTE path, I can get between 5 and 10 Mbit/s of upstream bandwidth.
This means I can’t realistically host my Git repos or Pleroma from here,
but for other stuff with modest upstream requirements it’s perfect, as private as humanly possible,
and (imo) pretty fun to build and maintain.

That’s all folks.


  1. https://openwrt.org/toh/netgear/wnr2200 ↩︎

  2. https://openwrt.org/ ↩︎

  3. https://openwrt.org/docs/guide-user/network/wan/multiwan/mwan3 ↩︎

  4. https://www.ui.com/edgemax/edgerouter-x/ ↩︎

  5. https://openwrt.org/docs/ ↩︎

  6. https://openwrt.org/docs/guide-user/network/wan/multiwan/mwan3 ↩︎

  7. NB: This would also work with any other VPN protocol but I really like WireGuard↩︎