arp flood if forward enabled

[AlexHighTower]

Hello,

I'm using fastd v18 on ubuntu 20 and debian 10
have 5 nodes
tap interfaces
switch mode
two on nodes has "forward yes" setting, other - no
this setup works stable
as soon as I change at any of other three nodes forward from "no" to "yes" I get arp flood like this

15:03:28.800343 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.800432 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.801812 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.801893 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.801904 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.802356 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.802365 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.806504 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.837806 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.837913 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.837923 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.839092 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.842301 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.844137 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.844154 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.844230 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.846769 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.846779 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.850793 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.856999 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.879528 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.883158 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.883174 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.886000 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.887729 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28
15:03:28.887837 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.887847 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.889483 ARP, Request who-has 10.10.99.215 tell 10.10.99.10, length 28
15:03:28.889492 ARP, Reply 10.10.99.215 is-at d2:7f:ba:10:91:fd, length 28
15:03:28.889547 ARP, Request who-has 10.10.99.225 tell 10.10.99.10, length 28

is it bug or feature?

[neocturne]

forward yes can only be used in tree topologies - as soon as you build a loop of forwarding instances, packets will loop endlessly. Generally, the "forward" feature is mostly meant for star topologies as a simple replacement for OpenVPNs client-server configuration.

For anything more complex, disable forwarding in fastd, and instead put a bridge interface with STP on top the TAP interfaces of fastd running in multitap mode. Another option is to run a routing protocol like batman-adv on top of fastd (which may result in more efficient forwarding paths in meshed topologies).

I wonder if I should simply disallow connecting two fastd instances with forwarding enabled (which will prevent such misconfigurations, but also remove the possibility to use fastd in non-star tree configurations without additional setup...)

[AlexHighTower]

Looks like I completely misunderstood idea of "mesh" in this case )))
I have setup where each node has connection to each other
And I expected that each node will be able to access any other node via internal address.
But my tests showed me that it is not so if there is no direct connection between nodes by external ip and "forward no"
"start" topology has disadvantage - central server, if it is not available, nodes looses connections...
At the moment I have big amount of WG links between nodes, separate subnets for this connections and bgp over, it works quite well but complex in configuration and each node has its own subnet and internal ip from it, and I wanted to make it simple and ip from one subnet...

[neocturne]

Right, fastd does not have any builtin mesh capabilities - instead, it is often used as a transport layer under a mesh routing protocol. If your network is based on Linux, batman-adv is usually a good choice, as it doesn't require any configuration except for unique MAC addresses, and it can provide a redundant L2 overlay network over a bunch of non-forwarding fastd instances in TAP mode (no need for multitap in this case).

[AlexHighTower]

I've tested batman-adv on my setup
It looks like work, but now I see big amount of traffic between fastd nodes
Even tcpdump shows no traffic on bat0 interface at the same time it shows a lot "ethertype Unknown (0x4305)" on fastd tap interface