Linux Conntrack Offload: High-Scale eBPF State Tracking for VPN Gateways

Operating high-capacity VPN gateways that aggregate tens of thousands of concurrent client tunnels presents acute kernel bottlenecks under heavy traffic loads. While modern symmetric multi-processing (SMP) servers feature 64 or 128 hardware threads, default Linux networking pipelines frequently experience severe packet drop rates long before hardware link capacities saturate. In high-density environments, the primary culprit is often not cryptographic cipher processing or memory bus limits, but the Linux Netfilter connection tracking subsystem (nf_conntrack).

Designed originally for stateful home and enterprise firewalls, nf_conntrack maintains global state for every bidirectional IP flow crossing the operating system. In high-scale VPN concentrators, where millions of ephemeral UDP and TCP streams churn through encrypted tunnels every minute, this architectural design introduces severe performance degradation, global spinlock contention, and unrecoverable packet drop conditions.

The Conntrack Bottleneck: Spinlocks and Table Saturation

Whenever an IP datagram enters the Linux network stack, Netfilter computes a hash of the 5-tuple (source IP, destination IP, source port, destination port, and L4 protocol) to locate or instantiate a struct nf_conn object inside a global hash table:

[Default Ingress Datapath with Conntrack]
Incoming Packet ──> PRE_ROUTING Hook ──> nf_conntrack_in()
                                              │
    Acquire Bucket Spinlock (nf_conntrack_locks) ◄── [Lock Contention Across 64+ Cores]
                                              │
                   Lookup / Allocate struct nf_conn (Slab Cache: kmalloc-512)
                                              │
              Release Spinlock ──> Forwarding / Route Lookup ──> Egress

Under enterprise VPN workloads, this pipeline encounters three distinct points of failure:

1. Table Saturation and Silent Packet Drops: The global state table size is bounded by net.netfilter.nf_conntrack_max. When concurrent client connections exceed this ceiling, the kernel issues the notorious error nf_conntrack: table full, dropping packet. Incoming SYN packets and WireGuard handshake initiations are silently discarded, breaking active user tunnels.
2. Atomic Spinlock Contention Across CPU Cores: The hash table is protected by a partitioned array of spinlocks (nf_conntrack_locks). As multi-queue network interface cards (NICs) distribute packets across dozens of CPU cores via Receive Side Scaling (RSS), disparate worker threads repeatedly attempt to acquire the same hash bucket lock. This creates catastrophic cache line bouncing across CPU sockets, driving CPU softirq utilization to 100% while actual throughput collapses.
3. Memory Bloat and Slab Fragmentation: Each tracked connection consumes approximately 320 to 384 bytes of unswappable kernel slab memory (kmalloc-512). Tracking 3 million active connections requires over 1.2 GB of raw RAM, imposing substantial pressure on kernel page tables and TLB caches.

Eliminating Conntrack Overhead: Declarative NOTRACK Rules

The most immediate method to protect VPN gateways from connection tracking exhaustion is instructing Netfilter to skip state tracking entirely for tunnel infrastructure ports.

By attaching rules to the raw table—which evaluates packets prior to connection tracking initiation at the PREROUTING and OUTPUT hooks—operators eliminate state allocations for tunnel encapsulation:

# nftables configuration to omit connection tracking for WireGuard traffic
nft add table inet raw
nft add chain inet raw prerouting { type filter hook prerouting priority -300 \; }
nft add chain inet raw output { type filter hook output priority -300 \; }

# Mark VPN UDP listener port 51820 with NOTRACK
nft add rule inet raw prerouting udp dport 51820 notrack
nft add rule inet raw output udp sport 51820 notrack

In traditional iptables syntax, the equivalent configuration applies to the raw table:

iptables -t raw -A PREROUTING -p udp --dport 51820 -j NOTRACK
iptables -t raw -A OUTPUT -p udp --sport 51820 -j NOTRACK

Applying NOTRACK reduces per-packet CPU cycle consumption by up to 28% on the outer UDP transport layer. However, simply disabling conntrack disables traditional stateful firewalling (ct state established,related), requiring network architects to implement stateless packet validation or program custom fastpaths.

Lockless In-Kernel State Tracking with eBPF

To retain intelligent session tracking, traffic policing, and symmetric routing without the scalability penalties of nf_conntrack, modern infrastructure implements state tables directly in eBPF using BPF_MAP_TYPE_LRU_HASH or Per-CPU hash maps.

Unlike Netfilter's global bucket locks, an eBPF Least Recently Used (LRU) hash table manages memory reclamation automatically. When the map approaches capacity, older idle entries are pruned locklessly without dropping valid incoming packets.

Below is an in-kernel eBPF traffic classifier attached to the Traffic Control (clsact) hook that inspects outer tunnel datagrams and records flow metrics:

#include <linux/bpf.h>
#include <linux/pkt_cls.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/udp.h>
#include <bpf/bpf_helpers.h>

struct flow_key {
    __u32 src_ip;
    __u32 dst_ip;
    __u16 src_port;
    __u16 dst_port;
    __u8  protocol;
};

struct flow_metrics {
    __u64 packets;
    __u64 bytes;
    __u64 last_seen;
};

/* Lockless LRU map for high-capacity state tracking */
struct {
    __uint(type, BPF_MAP_TYPE_LRU_HASH);
    __uint(max_entries, 2097152); /* 2 million flows */
    __type(key, struct flow_key);
    __type(value, struct flow_metrics);
} vpn_flow_table SEC(".maps");

SEC("tc")
int track_vpn_flows(struct __sk_buff *skb)
{
    void *data = (void *)(long)skb->data;
    void *data_end = (void *)(long)skb->data_end;

    /* Boundary check: Ethernet header */
    if (data + sizeof(struct ethhdr) > data_end)
        return TC_ACT_OK;

    struct ethhdr *eth = data;
    if (eth->h_proto != __constant_htons(ETH_P_IP))
        return TC_ACT_OK;

    struct iphdr *iph = data + sizeof(struct ethhdr);
    if ((void *)(iph + 1) > data_end || iph->protocol != IPPROTO_UDP)
        return TC_ACT_OK;

    struct udphdr *udph = (void *)iph + (iph->ihl * 4);
    if ((void *)(udph + 1) > data_end)
        return TC_ACT_OK;

    /* Filter for VPN traffic */
    if (udph->dest == __constant_htons(51820) || udph->source == __constant_htons(51820)) {
        struct flow_key key = {
            .src_ip = iph->saddr,
            .dst_ip = iph->daddr,
            .src_port = udph->source,
            .dst_port = udph->dest,
            .protocol = iph->protocol
        };

        struct flow_metrics *val = bpf_map_lookup_elem(&vpn_flow_table, &key);
        if (val) {
            __sync_fetch_and_add(&val->packets, 1);
            __sync_fetch_and_add(&val->bytes, skb->len);
            val->last_seen = bpf_ktime_get_ns();
        } else {
            struct flow_metrics init = {
                .packets = 1,
                .bytes = skb->len,
                .last_seen = bpf_ktime_get_ns()
            };
            bpf_map_update_elem(&vpn_flow_table, &key, &init, BPF_NOEXIST);
        }
    }

    return TC_ACT_OK;
}

char _license[] SEC("license") = "GPL";

Because bpf_map_lookup_elem on LRU maps operates with lockless RCU semantics, multi-core worker threads update counters simultaneously without serializing across CPU sockets.

Comparative Performance Metrics

In high-concurrency synthetic testing across dual AMD EPYC 9654 servers (192 physical cores) saturated with 5 million concurrent UDP sessions, the difference between default Netfilter connection tracking and eBPF lockless flow tracking is evident:

Production Engineering in RiceGuard Gateways

RiceGuard implements full connection tracking offloading across its global Anycast edge clusters:

• Immunity to State Exhaustion Attacks: Distributed denial-of-service (DDoS) campaigns frequently flood random source UDP ports to inflate connection tracking tables. RiceGuard's ingress pipeline processes datagrams statelessly through eBPF before validating cryptographic Noise identities in memory, rendering state exhaustion attacks ineffective.
• Microsecond Packet Forwarding: By omitting heavy Netfilter hooks and spinlock contention, transit processing time on multi-gigabit client tunnels remains consistently below 1.5 microseconds.
• Architectural Coexistence: Core Linux IP routing and BGP Anycast daemons continue functioning uninhibited, maintaining full operational visibility through standard eBPF telemetry without sacrificing dataplane throughput.

By replacing legacy connection tracking with declarative NOTRACK rules and lockless eBPF flow tables, high-scale VPN architectures deliver linear multi-core scaling and wire-speed packet processing under demanding multi-tenant workloads.