6. Hardware Offload

SPP provides hardware offload functions.

Note

We tested following use cases at Connect-X 5 by Mellanox only. Even if you cannot use these use cases on different NIC, we don’t support.

6.1. Hardware Classification

Some hardware provides packet classification function based on L2 mac address. This use case shows you how to use L2 classification.

../_images/spp_hardware_offload_classify.svg

6.1.1. Setup

Before using hardware packet classification, you must setup number of queues in hardware.

In bin/config.sh.

PRI_PORT_QUEUE=(
 "0 rxq 10 txq 10"
 "1 rxq 16 txq 16"
)

Above example includes the line 0 rxq 10 txq 10. 0 of this line specifies physical port number, rxq 10 is for 10 rx-queues, txq 10 is for 10 tx-queues.

You should uncomment the following block in bin/config.sh to indicate hardware white list. The option dv_flow_en=1 is for MLX5 poll mode driver.

PRI_WHITE_LIST=(
 "0000:04:00.0,dv_flow_en=1"
 "0000:05:00.0"
)

After editing bin/config.sh, you can launch SPP as following.

$ bin/start.sh
Start spp-ctl
Start spp_primary
Waiting for spp_primary is ready .................... OK! (8.5[sec])
Welcome to the SPP CLI. Type `help` or `?` to list commands.
spp >

Then, you can launch spp_vf like this.

spp > pri; launch vf 1 -l 2,3,4,5 -m 512 --file-prefix spp \
-- --client-id 1 -s 127.0.0.1:6666
...

6.1.2. Configuration

Before configure the flow of classifying packets, you can validate such rules by using flow validate command.

spp > pri; flow validate phy:0 ingress pattern eth dst is \
10:22:33:44:55:66 / end actions queue index 1 / end
spp > pri; flow validate phy:0 ingress pattern eth dst is \
10:22:33:44:55:67 / end actions queue index 2 / end

Then, you can configure flow using flow create command like this.

spp > pri; flow create phy:0 ingress pattern eth dst is \
10:22:33:44:55:66 / end actions queue index 1 / end
spp > pri; flow create phy:0 ingress pattern eth dst is \
10:22:33:44:55:67 / end actions queue index 2 / end

You can confirm created flows by using flow list or flow status commands. flow list command provides the flow information of specified physical port.

spp > pri; flow list phy:0
ID      Group   Prio    Attr    Rule
0       0       0       i--     ETH => QUEUE
1       0       0       i--     ETH => QUEUE

To get detailed information for specific rule. The following example shows the case where showing detailed information for rule ID 0 of phy:0.

spp > pri; flow status phy:0 0
Attribute:
  Group   Priority Ingress Egress Transfer
  0       0        true    false  false
Patterns:
  - eth:
    - spec:
      - dst: 10:22:33:44:55:66
      - src: 00:00:00:00:00:00
      - type: 0x0000
    - last:
    - mask:
      - dst: FF:FF:FF:FF:FF:FF
      - src: 00:00:00:00:00:00
      - type: 0x0000
Actions:
    - queue:
      - index: 1
spp >

In this use case, two components fwd1 and fwd2 simply forward the packet to multi-tx queues. You can start these components like this.

spp > vf 1; component start fwd1 2 forward
spp > vf 1; component start fwd2 3 forward

For each fwd1 and fwd2, configure the rx port like this.

spp > vf 1; port add phy:0 nq 1 rx fwd1
spp > vf 1; port add phy:0 nq 2 rx fwd2

Then, you can configure tx ports like this.

spp > vf 1; port add phy:1 nq 1 tx fwd1
spp > vf 1; port add phy:1 nq 2 tx fwd2

For confirming above configuration, you can use ping and tcpdump as described in Classify ICMP Packets.

Also, when you destroy the flow created above, commands will be like the following.

spp > pri; flow destroy phy:0 0
spp > pri; flow destroy phy:0 1

Or you can destroy all rules on specific hardware by using flow destroy command with ALL parameter.

spp > pri; flow destroy phy:0 ALL

6.2. Manipulate VLAN tag

Some hardware provides VLAN tag manipulation function. This use case shows you the case where incoming VLAN tagged packet detagged and non-tagged packet tagged when outgoing using hardware offload function.

../_images/spp_hardware_offload_vlan.svg

After having done above use case, you can continue to following. In this use case, we are assuming incoming packets which includes vid=100 to phy:0, these vid will be removed(detagged) and transferred to fwd1. Tx packets from fwd1 are sent to queue#0 on phy:1 with tagged by vid=100. Packets which includes vid=200 to phy:0 are to be sent to fwd2 with removing the vid, Tx packets from fwd2 are sent to queue#1 on phy:1 with tagged by vid=200.

For detagging flow creation.

spp > pri; flow create phy:0 ingress group 1 pattern eth dst is \
10:22:33:44:55:66 / vlan vid is 100 / end actions queue index 1 \
/ of_pop_vlan / end
spp > pri; flow create phy:0 ingress group 1 pattern eth dst is \
10:22:33:44:55:67 / vlan vid is 200 / end actions queue index 2 \
/ of_pop_vlan / end
spp > pri; flow create phy:0 ingress group 0 pattern eth / end \
actions jump group 1 / end

For tagging flow creation.

spp > pri; flow create phy:1 egress group 1 pattern eth dst is \
10:22:33:44:55:66 / end actions of_push_vlan ethertype 0x8100 \
/ of_set_vlan_vid vlan_vid 100 / of_set_vlan_pcp vlan_pcp 3 / end
spp > pri; flow create phy:1 egress group 1 pattern eth dst is \
10:22:33:44:55:67 / end actions of_push_vlan ethertype 0x8100 \
/ of_set_vlan_vid vlan_vid 200 / of_set_vlan_pcp vlan_pcp 3 / end
spp > pri; flow create phy:1 egress group 0 pattern eth / end \
actions jump group 1 / end

If you want to send vlan-tagged packets, the NIC connected to phy:0 will be configured by following.

$ sudo ip l add link ens0 name ens0.100 type vlan id 100
$ sudo ip l add link ens0 name ens0.200 type vlan id 200
$ sudo ip a add 192.168.140.1/24 dev ens0.100
$ sudo ip a add 192.168.150.1/24 dev ens0.100
$ sudo ip l set ens0.100 up
$ sudo ip l set ens0.200 up

6.3. Connecting with VMs

This use case shows you how to configure hardware offload and VMs.

../_images/spp_hardware_offload_vm.svg

First, we should clean up flows and delete ports.

spp > vf 1; port del phy:0 nq 0 rx fwd1
spp > vf 1; port del phy:0 nq 1 rx fwd2
spp > vf 1; port del phy:1 nq 0 tx fwd1
spp > vf 1; port del phy:1 nq 1 tx fwd2
spp > pri; flow destroy phy:0 ALL
spp > pri; flow destroy phy:1 ALL

Configure flows.

spp > pri; flow create phy:0 ingress group 1 pattern eth dst is \
10:22:33:44:55:66 / vlan vid is 100 / end actions queue index 1 \
/ of_pop_vlan / end
spp > pri; flow create phy:0 ingress group 1 pattern eth dst is \
10:22:33:44:55:67 / vlan vid is 200 / end actions queue index 2 \
/ of_pop_vlan / end
spp > pri; flow create phy:0 ingress group 0 pattern eth / end \
actions jump group 1 / end
spp > pri; flow create phy:0 egress group 1 pattern eth src is \
10:22:33:44:55:66 / end actions of_push_vlan ethertype 0x8100 \
/ of_set_vlan_vid vlan_vid 100 / of_set_vlan_pcp vlan_pcp 3 / end
spp > pri; flow create phy:0 egress group 1 pattern eth src is \
10:22:33:44:55:67 / end actions of_push_vlan ethertype 0x8100 \
/ of_set_vlan_vid vlan_vid 200 / of_set_vlan_pcp vlan_pcp 3 / end
spp > pri; flow create phy:0 egress group 0 pattern eth / end \
actions jump group 1 / end

Start components.

spp > vf 1; component start fwd3 4 forward
spp > vf 1; component start fwd4 5 forward

Start and setup two VMs as described in SSH Login to VMs. Add ports to forwarders.

spp > vf 1; port add phy:0 nq 1 rx fwd1
spp > vf 1; port add vhost:0 tx fwd1
spp > vf 1; port add phy:0 nq 2 rx fwd2
spp > vf 1; port add vhost:1 tx fwd2
spp > vf 1; port add vhost:0 rx fwd3
spp > vf 1; port add phy:0 nq 3 tx fwd3
spp > vf 1; port add vhost:1 rx fwd4
spp > vf 1; port add phy:0 nq 4 tx fwd4

Then you can login to each VMs.

Note that you must add arp entries of MAC addresses statically to be resolved.

# terminal 1 on remote host
# set MAC address
$ sudo arp -i ens0 -s 192.168.140.31 10:22:33:44:55:66
$ sudo arp -i ens0 -s 192.168.150.32 10:22:33:44:55:67

6.4. Reference

The following features are tested.

MT27710 Family [ConnectX-4 Lx] 1015 - dstMAC - dstMAC(range)

MT27800 Family [ConnectX-5] 1017 - dstMAC - dstMAC(range) - vlan vid - vlan vid+dstMAC - tagging+detagging

Ethernet Controller XXV710 for 25GbE SFP28 158b - dstMAC