2 Public IPs - 2 Hosts - Same Unsafe Network

[enzopsc]

I've a big network with 3 public IPs.

They're all located in different places and with different providers.

I've set 1 lighthouse and 3 nodes from 3 different public IP.

Below I'm showing only 2 of them because I was still testing nebula.

Behind these I've a single big network on 192.168.0.0/16

I was supposing to use unsafe_routes:
192.168.0.0/16 on both but when I tried to configure lighthouse that way to test reachability of them it wasn't working.

When node 172.16.50.4 goes down, it wasn't able to reach machines through 172.16.50.2

  unsafe_routes:
    #PublicIPServer2
    - route: 192.168.0.0/24
      via: 172.16.50.4
      metric: 100
    #PublicIPServer1
    - route: 192.168.0.0/24
      via: 172.16.50.2
      metric: 99

Basically it should give me ability to reach my clients from any public IP I've to grant me 100% uptime from outside my private network.

How I should configure it, is it possible?

Thanks in advance.

[johnmaguire]

Hi @enzopsc - have you given this a read through? https://nebula.defined.net/docs/guides/unsafe_routes/

You'll need to make sure your routing nodes have the subnets defined in their certificate, IP forwarding must be enabled on the host, you need routing rules configured outside Nebula (e.g. using iptables), as well as creating Nebula firewall rules to allow this traffic.

[enzopsc]

Hi @johnmaguire
I guess that I did that already:
#Lighthouse

pki:
  ca: /usr/local/nebula/nebula/ca.crt
  cert: /usr/localt/nebula/lighthouse1.crt
  key: /usr/local/nebula/lighthouse1.key
lighthouse:
  am_lighthouse: true
  interval: 60
listen:
  host: 0.0.0.0
  port: 4242
punchy:
  punch: true
cipher: aes
relay:
  am_relay: true
  use_relays: true
tun:
  disabled: false
  dev: nebula1
  drop_local_broadcast: false
  drop_multicast: false
  tx_queue: 500
  mtu: 1300
  routes:
  unsafe_routes:
    - route: 192.168.0.0/16
      via: 172.16.50.3
      metric: 100  
    - route: 192.168.0.0/16
      via: 172.16.50.2
      metric: 99
logging:
  level: debug
  format: text
firewall:
  outbound_action: drop
  inbound_action: drop
  conntrack:
    tcp_timeout: 12m
    udp_timeout: 3m
    default_timeout: 10m
    max_connections: 100000
  outbound:
    - port: any
      proto: any
      host: any
  inbound:
    - port: any
      proto: any
      host: any
      

#server1

pki:
  ca: /usr/local/nebula/ca_nebula.crt
  cert: /usr/local/nebula/ufficio1.crt
  key: /usr/local/nebula/ufficio1.key
static_host_map:
  "172.16.50.1": ["dnsofmylighthouse:4242"]
lighthouse:
  am_lighthouse: false
  interval: 30
  hosts:
    - "172.16.50.1"
listen:
  host: 0.0.0.0
  port: 0
punchy:
  punch: true
cipher: aes
relay:
  relays:
    - 172.16.50.1
  am_relay: false
  use_relays: true
tun:
  disabled: false
  drop_local_broadcast: false
  drop_multicast: false
  tx_queue: 500
  mtu: 1300
  routes:
logging:
  level: debug
  format: text
firewall:
  outbound_action: drop
  inbound_action: drop
  conntrack:
    tcp_timeout: 12m
    udp_timeout: 3m
    default_timeout: 10m
  outbound:
    - port: any
      proto: any
      host: any
  inbound:
    - port: any
      proto: any
      host: any

#server2:

pki:
  ca: /usr/local/nebula/ca_nebula.crt
  cert: /usr/local/nebula/server2.crt
  key: /usr/local/nebula/server2.key
static_host_map:
  "172.16.50.1": ["dnsofmylighthouse:4242"]
lighthouse:
  am_lighthouse: false
  interval: 30
  hosts:
    - "172.16.50.1"
listen:
  host: 0.0.0.0
  port: 0
punchy:
  punch: true
cipher: aes
relay:
  relays:
    - 172.16.50.1
  am_relay: false
  use_relays: true
tun:
  disabled: false
  drop_local_broadcast: false
  drop_multicast: false
  tx_queue: 500
  mtu: 1300
  routes:
logging:
  level: debug
  format: text
firewall:
  outbound_action: drop
  inbound_action: drop
  conntrack:
    tcp_timeout: 12m
    udp_timeout: 3m
    default_timeout: 10m
  outbound:
    - port: any
      proto: any
      host: any
  inbound:
    - port: any
      proto: any
      host: any

I've already followed that guide https://nebula.defined.net/docs/guides/unsafe_routes/ for :

• Forwarding rules
• Certificates' subnets
• unsafe_routes declaration

and it works.

Server1 can reach Server2 and viceversa but if lighthouse tries to reach Server2 (172.16.50.3) and is not online it doesn't switch to Server1 (172.16.50.2).
Did I do something wrong?

[enzopsc]

I ended up doing a script that pings nebula hosts and switch configuration based on that.
Thanks for help.