The Classification Chokepoint: Why Legacy Filters Stumble at 100G

Carrier-grade VPN concentrators and high-density termination hubs process staggering volumes of network traffic. When aggregation links reach 25 Gbps, 40 Gbps, or 100 Gbps, incoming packet rates frequently exceed tens of millions of packets per second (Mpps). At this scale, packet classification—the process of parsing header fields to determine routing, firewall filtering, rate limiting, and tunnel assignment—becomes the primary system bottleneck.

Historically, Linux administrators configured packet inspection using iptables or early nftables tables. While functional on low-bandwidth routers, these software-defined filtering frameworks suffer from fundamental structural limitations:

  1. Linear Rule Traversal in Software Interrupt Context: Software firewalls evaluate filter lists sequentially. When packets enter the networking stack via Linux softirq routines (ksoftirqd), evaluating hundreds of linear rules consumes significant CPU clock cycles. If an attacker unleashes an unauthenticated UDP packet flood against a VPN port, host cores saturate simply traversing filtering chains before discarding malicious payloads.
  2. Brittle Bit-Offset Arithmetic in Legacy TC Classifiers: The legacy Linux Traffic Control classifier u32 required manual bit-shift math at raw packet offsets. Variable-length IPv4 header options, IPv6 extension headers, and nested tunnel encapsulations often cause u32 offset calculations to misalign or fail outright.
  3. Severe Memory Bus Congestion and Cache Thrashing: In pure software processing, every unsolicited packet must traverse the host PCIe bus and allocate an intermediate socket buffer (sk_buff) in main system memory. Under massive ingress volume, the CPU L2/L3 cache is repeatedly evicted, starving the ChaCha20-Poly1305 and AES-GCM cryptographic pipelines of execution resources.

To maintain wire-speed transit and eliminate softirq saturation, high-throughput VPN gateways need a modern, structured classifier capable of shifting filter logic directly into network adapter silicon: Linux TC Flower (cls_flower).

Architecture of Linux TC Flower (cls_flower)

Integrated into modern Linux kernels, the Flower classifier (cls_flower) transforms the Linux Traffic Control (tc) subsystem into a high-performance flow classification engine. Unlike the fragile offset calculations of u32, Flower uses the kernel's native flow dissector (net/core/flow_dissector.c) to parse structured protocol fields across Layer 2, Layer 3, and Layer 4 headers.

+-------------------------------------------------------------------------+
|                        Ingress Ethernet Frame                           |
+-------------------------------------------------------------------------+
                                    |
                                    v
+-------------------------------------------------------------------------+
|                  Linux Flow Dissector Engine (Kernel)                   |
|     - Extracts L2 MAC, VLAN ID, EtherType                               |
|     - Extracts L3 IPv4 / IPv6 Source & Destination Addresses, DSCP      |
|     - Extracts L4 UDP / TCP Ports, TCP Flags, ICMP Types                |
|     - Extracts Encapsulation Keys (VNI, Tunnel ID)                      |
+-------------------------------------------------------------------------+
                                    |
                                    v
+-------------------------------------------------------------------------+
|                  TC Flower Mask-and-Match Flow Table                    |
|   - Constant-time O(1) hash lookup across configured key tuples         |
+-------------------------------------------------------------------------+
         |                                  |
   Match Found                         No Match Found
         |                                  |
         v                                  v
+-----------------------+          +-----------------------+
|  Configured TC Action |          | Fallback / Default Q  |
|  - drop               |          | Normal IP Stack       |
|  - mirred redirect    |          +-----------------------+
|  - police / rate limit|
|  - skbedit priority   |
+-----------------------+

Key Architectural Advantages of TC Flower

  • Mask-and-Match Hash Tables: Flower groups filtering rules by their bitmask patterns. Instead of scanning rules one by one, incoming packets generate a single hash across relevant header fields, achieving near constant-time O(1) classification regardless of rule count.
  • Deep Protocol Awareness: Flower natively matches on complex network attributes without script calculation, including VLAN tags (vlan_id, vlan_prio), DSCP values (ip_tos), TCP connection flags (tcp_flags syn), and UDP payload signatures.
  • Composable Action Chains: Once a flow matches, Flower attaches modular traffic control actions, such as immediate discarding (action drop), hardware packet redirection (action mirred egress redirect), rate shaping (action police), or socket buffer metadata modification (action skbedit).

Attaching a Flower filter to a network interface's ingress queue requires attaching the ingress qdisc first:

# Attach the ingress queuing discipline to physical device enp1s0f0
tc qdisc add dev enp1s0f0 ingress

# Filter and drop non-VPN UDP traffic targeting internal service ports
tc filter add dev enp1s0f0 ingress protocol ip prio 10 flower \
    ip_proto udp \
    dst_port 53 \
    action drop

Achieving Line Rate with TC Flower ASIC Hardware Offload

While Flower provides fast software classification in kernel memory, its most potent capability lies in its native support for ASIC Hardware Offload via the switchdev driver architecture.

Modern enterprise network interface cards (such as Intel Columbiaville E810 and Mellanox ConnectX-5/6/7) embed programmable packet-processing engines with Ternary Content Addressable Memory (TCAM). When configured with hardware offload flags, the Linux kernel translates TC Flower rules into microcode and writes them directly into the NIC's internal classification tables.

+-------------------------------------------------------------------+
|               Physical Fiber Optical Interface                    |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
|               NIC Transceiver & MAC / PHY Layer                   |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
|            On-Chip Hardware Parser & TCAM Flow Tables             |
|                  (TC Flower Offloaded Silicon)                    |
|                                                                   |
|   Rule: Drop UDP Floods (skip_sw)    --> [HARDWARE DISCARD]       |
|   Rule: Direct RiceGuard Port 51820  --> [DMA RING TO CORE 4-7]   |
+-------------------------------------------------------------------+
            | (Only valid encrypted VPN packets proceed)
            v
+-------------------------------------------------------------------+
|               Host PCIe Bus & System RAM (DMA)                    |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
|             RiceGuard Multi-Threaded Decryption Engine            |
+-------------------------------------------------------------------+

The Three Offload Execution Modes

When deploying TC Flower rules, network architects specify where execution takes place:

  1. Automatic Offload (Default): If the hardware driver supports the rule, the kernel programs the NIC ASIC while maintaining an identical shadow rule in software memory. If hardware resources become exhausted, fallback to software execution occurs automatically.
  2. skip_hw: Restricts the filter strictly to kernel memory space, ensuring no hardware ASIC tables are consumed. Useful for complex rules containing unsupported proprietary action modules.
  3. skip_sw (Pure Hardware Execution): Forces the rule to install exclusively inside the physical NIC silicon. If the card cannot offload the exact match pattern or action, the tc command immediately aborts with an error.

To illustrate pure hardware mitigation on a 100 GbE interface:

# Drop spoofed IP subnet traffic directly in hardware silicon
tc filter add dev enp1s0f0 ingress protocol ip prio 1 flower \
    src_ip 198.51.100.0/24 \
    skip_sw \
    action drop

# Steer verified VPN UDP flows directly into high-priority queue
tc filter add dev enp1s0f0 ingress protocol ip prio 2 flower \
    ip_proto udp \
    dst_port 51820 \
    skip_sw \
    action skbedit priority 1

With skip_sw active, malicious or unauthorized packets are discarded at the physical Ethernet PHY/MAC layer inside the card. Zero bytes cross the PCIe bus, zero socket buffers are allocated in host RAM, and zero CPU cycles are spent in ksoftirqd.

Deploying Wire-Speed Security Policies in RiceGuard Gateways

In high-capacity distributed VPN architectures such as VPNRice, the RiceGuard engine combines modern cryptographic primitives with silicon-level packet steering to ensure sustained throughput and strict isolation.

+-------------------------------------------------------------------+
|                    VPNRice Edge Gateway Node                      |
|                                                                   |
|   +-----------------------------------------------------------+   |
|   |         Physical NIC (100 GbE Intel / Mellanox)           |   |
|   |         Hardware TCAM: TC Flower Rules (skip_sw)          |   |
|   +-----------------------------------------------------------+   |
|         |                                           |             |
|   [Port 51820 Valid VPN]                  [Unauthenticated / Flood|
|         |                                           |             |
|         v                                           v             |
|   Dedicated RX Ring DMA                      Dropped in Silicon   |
|         |                                    (0% Host CPU Load)   |
|         v                                                         |
|  +-------------------------------------------------------------+  |
|  |             RiceGuard Kernel Cryptographic Core             |  |
|  |             ChaCha20-Poly1305 Line-Rate Decryption          |  |
|  +-------------------------------------------------------------+  |
|         |                                                         |
|         v                                                         |
|  +-------------------------------------------------------------+  |
|  | Isolated Network Namespace & Encrypted DNS Resolvers        |  |
|  | Absolute Protection Against Outbound Plaintext Leaks       |  |
|  +-------------------------------------------------------------+  |
+-------------------------------------------------------------------+

1. Ingress Flow Demultiplexing

Rather than processing all incoming packets through a single kernel software queue, RiceGuard installs TC Flower filters that identify encrypted WireGuard/RiceGuard handshake packets and established UDP sessions. The NIC ASIC steers these packets directly into dedicated receive rings aligned with specific CPU cores via Receive Side Scaling (RSS). This layout prevents lock contention and guarantees consistent sub-millisecond packet latency.

2. Pre-Authentication Flood Protection

Unauthenticated packet floods can degrade VPN performance by exhausting server cryptographic handshakes. Using TC Flower hardware policing (action police rate 100mbit burst 10k drop skip_sw), gateway nodes enforce strict ingress bandwidth envelopes on handshake endpoints inside the physical NIC silicon. Volumetric attacks are mitigated upstream of the CPU, keeping decryption worker threads responsive.

3. Namespace Isolation and Zero-Leak Safeguards

Each client termination zone within the RiceGuard mesh is partitioned into isolated Linux network namespaces. TC Flower egress redirect rules ensure that unencrypted inner transit packets cannot leak onto external physical interfaces. In parallel, local loopback DNS proxies inside each namespace validate domain queries under strict encryption, ensuring that subscriber traffic remains private, protected, and fully isolated under heavy multi-gigabit workloads.