So I've been banging my head on this issue for a while, hopefully someone can help me here.
My idea is to run a proxy (nginx, traefik, etc) on the public server, with authelia or some other kind of auth helper, and redirect to some services that can be run on the container network behind NAT (think like matrix.org, gitlab server, grafana)
Let's set the stage: there is a physical network, with a router and internet connection behind CG-NAT. The network router is configured with 5 different subnets:
192.168.101.1/24 - servers- ...
192.168.105.1/24 - containers
Each subnet has it's first IP 192.168.10x.1 as the gateway which is physically the router itself.
I thought of k3s, but to be honest it is quite a lot of complexity to just access some local services. Let's not even go to k8s. So then to simplify things I thought of solving the networking layer2 issue:
- tried tailscale, but that's not lvl2
- ngrok uses their website as a proxy, we don't want that
- zerotier bridges at lvl2, so it seems like a good solution!
So the public server has its installation of zerotier, and there is a machine in the office network destined to be a zerotier bridge, and bridge onto the 105 subnet. For this I created a ubuntu server with fixed IP 192.168.105.3, installed zerotier 1.8.4.
Now if I select any subnet that's not also a physical network, everything works: computers can ping each other on that virtual network and iperf3 gets to a reasonable 80mbits-ish in and out of the NAT.
But, I want to bridge to a real subnet, so I proceeded to configure the bridging. Initially I took inspiration from the guide available [here][2], and it doesn't work.
I then killed it completely, re-installed ubuntu server and followed a basic bridging guide like [this one][3]. And still it doesn't work.
In zerotier I selected:
subnet 192.168.105.x- split the dhcp range between zerotier and the dhcp
- the bridge machine has no IP assigned by zerotier, but fixed IP address 192.168.105.3
- bridging allowed, br0 iface configured and showing the right IP 192.168.105.3
I'm now convinced this is a routing and masking issue: what kind of routing and masking do I need to set on the bridge machine so that it can relay lvl2 packets to the physical network?
At the same time, I haven't found any mention of how to tell the physical network that a machine different from the gateway, should now be the endpoint of another route! It's like all tutorials related to bridging only solve the issue one way, but the other way is not even discussed?
Anything on the physical network trying to access the rest of the network space behind zerotier will still helplessly ask the gateway which won't have a clue, right?
Is there a better solution than zerotier to achieve my objective?
[1]:
[2]: https://zerotier.atlassian.net/wiki/spaces/SD/pages/193134593/Bridge+your+ZeroTier+and+local+network+with+a+RaspberryPi
[3]: bridging two network interfaces in ubuntu linux 12.10 AND being able to access it from that machine
