MAC Interface source in failover mode - CARP unicast - flood trafic

[blacknote942]

Hi,
Not certain if it's the good place to ask, please could you have a look on that case and tell me.

OPNsense 24.7.11_2-amd64 - FAILOVER

Case CARP in multicast:
While no ISC DHCP is running or any other service that generate unicast trafic from the slave makes it fully silent.
So far, its mac-address disapears from the core switch.
In that case, any unicast packet to the SLAVE without arp request would be broadcasted to the entire network.
CARP VRRP packets are multicasted all over network to client or guest terminal. This is ugly and unsecure to me. I would prefer to use the unicast mode.

Case CARP in unicast:
On interfaces without any source service or regulary trafic from the salve is doing the same: the mac from the slave times out on switches and then, any unicast packets from CARP are broadcasted all over the networks... no luck.

So the CARP unicast mode would be great to circle/reduce the data path between the master and slave but, without dhcp-failover or any services providing data from the salve, the consequences are "worst".
I had to add static mac on the CORE SWITCH to respond but this is ugly too.

My idea would be to be abble to get a GARP option on interfaces. I've seen sthg like that on Stormshield for VIP.
Looked for arping but doesnt exist anymore as plugin?
A check box for GARP any 60sec per Interface would be optimal. Maybe an option on the CARP configuration too.

Thanks for your replies and any help - I'm certain about the consequences - got backtraces - i might be wrong on something anyway.
Feel free to let me know.

Then thank you for all works done.

Regards
BN.

[fichtner]

Hi,

Maybe you are looking for https://forum.opnsense.org/index.php?topic=31680.msg168662#msg168662

Cheers,
Franco

[fichtner]

Maybe that was the wrong hint, there is also https://reviews.freebsd.org/D7695 so net.link.ether.inet.garp_rexmit_count

Note these options are per system, not interface.

[blacknote942]

thank you both for help, knowledge and time.
Thank for moving the post.
I'm testing net.link.ether.inet.garp_rexmit_count. That should do the job.

[blacknote942]

I have configured it in tunables + reboot but nothing in tcpdump for about 2 hours..

I should have had a GARP about every 4 min.

double check - still nothing
sysctl -a | grep garp > confirmed well included.
no garp detected on interface.

[blacknote942]

So i decided to try to script a auto ping interface from slave to VIP- master. it works.
Aim is to be abble to use the UNICAST CARP mode without having to declare static mac on core switch - slave mac without trafic would be timeout and packets would be floodded all over interface and networks.

now, the core switch learns from 4 min ping the slave mac and i migrate again to the unicast carp mode.
now, i'm facing again the second issue i already noticed. If im wrong on sthg, please correct and suggest.

CARP UNICAST mode is active and so far, it uses the multicast mac to provioded the "gateway". The firewall, uses that multicast mac only to receive but uses its real mac to reply. The multicast mac is so far unknown on the network. devices lean it from arp but no trafic is provided by the firewall from the gateway address with that mac.

consequences: any trafic to the multicast GW mac is flooded all over the network.

bad luck or really bad configuration from mine.
I already post some on the opnsense forum and it doesnt look so far that it has been noticed so much. Consequences are somehow terrible.

got to roll back in multicast mode now - from me, unicast mode would be better to not flood devices with vrrp/carp packets to clients devices - impacts in unicast mode is terrible and this mode shoulnt be used.

in resume the situation:

in some cases, the slave doenst make any trafic so its mac timeout on the network - any direct packets without arp are flooded - visible with the unicast carp mode.

static mac on the core switch or script to ping from the slave resolves the situation

In unicast mode, the multicast gw isnt used as a source from the master so packets from devices to the GW are flooded all over the network.
See exemple below - then i received a trace few weeks ago from a terminal port on access switch and i was so surprised to see the traffic beeing flooded - at this moment, this was the unicast vrrp trafic from master to slave wich was flooded.

from master
22:37:28.964500 ARP, Request who-has 10.73.10.254 tell 10.73.10.111, length 46
22:37:28.964506 ARP, Reply 10.73.10.254 is-at 00:00:5e:00:01:0a (oui IANA), length 28

from core sw > no multicast mac from the vlan10 - i havent migrated the others.

SWC# show mac-address-table detail | i 00:00
00:00:5e:00:01:49 20 dynamic lag300 300 false false
00:00:5e:00:01:64 100 dynamic lag100 300 false false
00:00:5e:00:01:6e 110 dynamic lag100 300 false false
00:00:5e:00:01:77 119 dynamic lag100 300 false false
00:00:5e:00:01:7d 125 dynamic lag100 300 false false
00:00:5e:00:01:82 130 dynamic lag100 300 false false
00:00:5e:00:01:8c 140 dynamic lag100 300 false false
00:00:5e:00:01:ac 172 dynamic lag100 300 false false
00:00:5e:00:01:b4 180 dynamic lag100 300 false false
00:00:5e:00:01:be 190 dynamic lag100 300 false false
00:00:5e:00:01:c8 200 dynamic lag100 300 false false
00:00:5e:00:01:fa 250 dynamic lag100 300 false false

SWC# show mac-address-table detail | i f4:90 > each mac from firewall are well known since the cron ping is active
f4:90:ea:01:27:d2 10 dynamic lag100 300 false false
f4:90:ea:01:27:ca 10 dynamic lag200 300 false false

on the SLAVE: it received the packet wich is destined to 00:00:5e:00:01:0a

root@fw-slave:~ # tcpdump -i vlan0.10 ether host 00:00:5e:00:01:0a -XXX
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vlan0.10, link-type EN10MB (Ethernet), snapshot length 262144 bytes
22:52:37.395560 IP 10.73.10.200 > 10.73.10.254: ICMP echo request, id 36219, seq 1, length 108
0x0000: 0000 5e00 010a 4cd5 8750 3900 0800 4500 ..^...L..P9...E.

[blacknote942]

sharing the auto ping from BACKUP Firewall

Relativement à CARP UNICAST
A la non émission de packet du SLAVE si aucun service n'en founit (ie ISCDHCP)
Nécessite de faire un GARP ou autre trafic depuis le SLAVE pour mettre à jour ARP du SW et éviter la diffusion de packet UNICAST dans le réseau
Action est faite par défaut sur toutes les IF SLAVE en BACKUP - simplification et assurance de réaliser l'opération quelques soit le service et les interfaces anciennes ou nouvelles.

Une ACL permettant ICMP entre toutes les interfaces du FW vers la VIP est nécessaire.
En cas de nouvelle Interface, l'ALIAS et l'ACL (IF) doivent être maj.

ATTENTION: TYPE INTERFACE A DEFINIR sauf vtnet/vlan/igc

SCRIPT

cd "your path"
mkdir script-monit && cd script-monit
vi slave_monit_master_garp.sh

#!/bin/sh

envoie un ping vers VIP si CARP BACKUP

ref: garp slave en carp unicast

iftypes="vtnet igc vlan"

for iftype in $iftypes; do
netstat -rnW | grep -v default | grep $iftype | awk '{print $6}' > /tmp/fileif
while IFS= read -r line; do
ifconfig $line | grep BACKUP > /dev/null
if [ $? = 0 ]; then
ifconfig $line | grep inet | grep vhid | awk '{print $2}' > /tmp/fileifip
masterip=sed -n "1p" /tmp/fileifip
ping -c 1 -t 1 $masterip > /dev/null
fi
done < "/tmp/fileif"
done

exit

chmod 744 slave_monit_master_garp.sh

ACTION

cd /usr/local/opnsense/service/conf/actions.d

vi actions_slave_monit_master_garp.conf

root@fw-slave:/usr/local/opnsense/service/conf/actions.d # cat slave_monit_master_garp.conf

[check]
command:"your path"/script-monit/slave_monit_master_garp.sh
parameters:
type:script_output
message:slavepingmastervip
description: slave_monit_master_garp

service configd restart
#TEST:
configctl slave_monit_master_garp check

CRON JOB > */4

[blacknote942]

In fact, the final trick would be to force the MASTER to use the multicast mac adress while sending the unicast vrrp packets to the slave.
then, force the master to answer with that multicast GW mac when answering to clients. Not with its real mac.

MULTICAST VRRP
23:11:15.659892 IP 10.73.10.251 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 10, prio 10, authtype none, intvl 10s, length 36
0x0000: 0100 5e00 0012 0000 5e00 010a 0800 45e0 ..^.....^.....E.

UNICAST VRRP
23:11:53.890942 IP 10.73.10.251 > 10.73.10.252: VRRPv2, Advertisement, vrid 10, prio 10, authtype none, intvl 10s, length 36
0x0000: f490 ea01 27ca f490 ea01 27d2 0800 45e0 ....'.....'...E.

ANSWER FROM MASTER TO CLIENT with the real mac IF
22:24:11.323742 IP 10.73.20.254.domain > 10.73.20.249.41308: 39164 0/1/0 (127)
0x0000: 28de 65cd ef48 f490 ea01 27d2 0800 4500 (.e..H....'...E.