nftables set for ip6 in ip table

[derbasti381]

What happened?

I wanted to try crowdsec for my local firewall which is based on nftables. I installed crowdsec-firewall-bouncer-nftables 0.0.28 on Debian and adjusted the configuration to only create the ip sets.
When I restart the service, this is what happens:

# nft monitor
add set ip mangle crowdsec-blacklists { type ipv4_addr; flags timeout; } 
add set ip mangle crowdsec6-blacklists { type ipv6_addr; flags timeout; } 

What did you expect to happen?

I expected the rule to be inserted in ip6, because as it goes to ip, it can never even be used by any ip6 flow.

add set ip6 mangle crowdsec6-blacklists { type ipv6_addr; flags timeout; } would be the correct one. Even if i create the Set in ip6 before, it still get's added to ip only.

How can we reproduce it (as minimally and precisely as possible)?

nftables:
  ipv4:
    enabled: true
    set-only: true
    table: mangle
    chain: crowdsec
    priority: -10
  ipv6:
    enabled: true
    set-only: true
    table: mangle
    chain: crowdsec
    priority: -10

#!/usr/sbin/nft -f
flush ruleset
add table ip mangle;
add table ip6 mangle;
add chain ip mangle crowdsec;
add chain ip6 mangle crowdsec;

Apply nftables, restart firewall-bouncer

Anything else we need to know?

It would also be suitable to add both sets to inet. Like this both protocols could access the sets.

Crowdsec version

2023/10/18 02:07:44 version: v1.5.4-debian-pragmatic-amd64-e4dcdd25728b914823525f1efabf18d5c454902b 2023/10/18 02:07:44 Codename: alphaga 2023/10/18 02:07:44 BuildDate: 2023-09-20_12:17:53 2023/10/18 02:07:44 GoVersion: 1.20.5 2023/10/18 02:07:44 Platform: linux 2023/10/18 02:07:44 libre2: C++ 2023/10/18 02:07:44 Constraint_parser: >= 1.0, <= 2.0 2023/10/18 02:07:44 Constraint_scenario: >= 1.0, < 3.0 2023/10/18 02:07:44 Constraint_api: v1 2023/10/18 02:07:44 Constraint_acquis: >= 1.0, < 2.0

OS version

$ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" $ uname -a Linux opnutm 6.1.0-13-amd64 crowdsecurity/crowdsec#1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29) x86_64 GNU/Linux

Enabled collections and parsers

$ cscli hub list -o raw
# paste output here

Acquisition config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* # paste output here

On Windows:

C:> Get-Content C:\ProgramData\CrowdSec\config\acquis.yaml

paste output here

Config show

$ cscli config show
# paste output here

Prometheus metrics

$ cscli metrics
# paste output here

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

[github-actions]

@derbasti381: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[LaurenceJJones]

Moving over to firewall repository as issue is not from CrowdSec Security Engine

[LaurenceJJones]

Debugging

cs-firewall-bouncer/pkg/nftables/nftables_context.go

Lines 128 to 164 in af6e7e2

	func (c *nftContext) initSetOnly() error {
	var err error
	// Use existing nftables configuration
	log.Debugf("nftables: ip%s set-only", c.version)
	c.table, err = c.lookupTable()
	if err != nil {
	return err
	}
	set, err := c.conn.GetSetByName(c.table, c.blacklists)
	if err != nil {
	log.Debugf("nftables: could not find ip%s blacklist '%s' in table '%s': creating...", c.version, c.blacklists, c.tableName)
	set = &nftables.Set{
	Name: c.blacklists,
	Table: c.table,
	KeyType: c.typeIPAddr,
	KeyByteOrder: binaryutil.BigEndian,
	HasTimeout: true,
	}
	if err := c.conn.AddSet(set, []nftables.SetElement{}); err != nil {
	return err
	}
	if err := c.conn.Flush(); err != nil {
	return err
	}
	}
	c.set = set
	log.Debugf("nftables: ip%s set '%s' configured", c.version, c.blacklists)
	return nil
	}

Issue happens in setonly function, we only interact with the conn (ipv4) chain rather than conn6 (ipv6)

Need to debug further as, technically we shouldn't even bother adding addr type v6 to ip chain as it has no value

edit edit: will spin up a vm to test shortly

[derbasti381]

Hello guys,

it's been a while since I posted this bug, but no activity yet. It doesn't seem too complicated to me. Is someone willing to check this?

Thanks a lot
Bastian

[LaurenceJJones]

Hello guys,

it's been a while since I posted this bug, but no activity yet. It doesn't seem too complicated to me. Is someone willing to check this?

Thanks a lot Bastian

Hey 👋🏻

I checked and only get access to these IP types based on nftables lib we use from Google

https://github.com/google/nftables/blob/ef45dd3322d6742ea14356317afa1c19b4e0a505/set.go#L80-L81

Same for family types

https://github.com/google/nftables/blob/ef45dd3322d6742ea14356317afa1c19b4e0a505/table.go#L36-L37

and this is what we set the v6 conn too

cs-firewall-bouncer/pkg/nftables/nftables_context.go

Lines 93 to 94 in 06416b4

	tableFamily: nftables.TableFamilyIPv6,
	typeIPAddr: nftables.TypeIP6Addr,

So either it not exposed via the lib or missing 🤷🏻

[derbasti381]

Thanks for having another look.
In metrics.go there is a function ipFamily which returns exactly what is needed, without the tableFamily, but with the string. The question is, why is it not used in the sets mechanism.

Am patiently waiting for someone digging. ;-)

[blind-oracle]

Same thing here.

# nft list table ip filter
table ip filter {
	set crowdsec {
		type ipv4_addr
		flags timeout
	}

	set crowdsec6 {
		type ipv6_addr
		flags timeout
	}

Probably gonna have to research myself...

[blind-oracle]

It seems that using ListTablesOfFamily is needed here, otherwise it gives all tables and the first that matches the name ("filter") is returned.

https://pkg.go.dev/github.com/google/nftables?utm_source=godoc#Conn.ListTablesOfFamily

here's a oneliner fix PR #354

Test in our environment - works as intended now.