Skip to content

[STACKED on #1442] virtio-net: enable IPv6 checksum and TSO offload#1444

Draft
gburd wants to merge 39 commits into
cloudius-systems:masterfrom
gburd:pr/virtio-net-ipv6-offload
Draft

[STACKED on #1442] virtio-net: enable IPv6 checksum and TSO offload#1444
gburd wants to merge 39 commits into
cloudius-systems:masterfrom
gburd:pr/virtio-net-ipv6-offload

Conversation

@gburd

@gburd gburd commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Note

Stacked on #1442 (the IPv6 forward-port). This branch is based on
pr/ipv6-forward-port, so the diff below is cumulative and re-includes the
IPv6 stack. Once #1442 merges I will rebase onto master and this reduces to
the two-file drivers/virtio-net.{cc,hh} change. Please review #1442 first.

Performance follow-up to #1442. IPv6 already inherits OSv's high-value fast
paths (net channels, LRO, RCU connection classifier, AF-agnostic TCP header
prediction), but the virtio-net hardware offload paths were hardcoded to
ETHERTYPE_IP:

  • TX built only VIRTIO_NET_HDR_GSO_TCPV4;
  • RX rejected everything but IPv4 (with a literal // How come - no support for IPv6?! comment);
  • if_hwassist/if_capabilities advertised IFCAP_TSO4 but not IFCAP_TSO6
    or the CSUM_*_IPV6 bits.

So IPv6 computed checksums in software and never used TSO, burning more CPU than
IPv4 on high-throughput sends/receives even though tcp_output already emits
CSUM_TCP_IPV6.

This change:

  • negotiates VIRTIO_NET_F_HOST_TSO6 / VIRTIO_NET_F_GUEST_TSO6;
  • advertises IFCAP_TSO6 and CSUM_TCP_IPV6 | CSUM_UDP_IPV6;
  • TX: adds an ETHERTYPE_IPV6 case (csum_start past the 40-byte base header,
    gso_type = VIRTIO_NET_HDR_GSO_TCPV6); packets with extension headers have
    ip6_nxt != TCP/UDP and fall back to software checksums;
  • RX: accepts ETHERTYPE_IPV6 in the checksum-offload path (the TCP/UDP
    checksum-offset logic is identical for v4 and v6);
  • uses large receive buffers when GUEST_TSO6 is negotiated.

Boot-tested with IPv6 SLAAC over virtio-net (loopback + concurrent dual-stack
TCP pass, global address configured through the modified TX/RX paths). A
throughput A/B (v6 with vs without offload) is the natural benchmark once the
base merges.

cmyers-spirent and others added 30 commits December 11, 2019 12:58
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-2-git-send-email-charles.myers@spirent.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-3-git-send-email-charles.myers@spirent.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-4-git-send-email-charles.myers@spirent.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-5-git-send-email-charles.myers@spirent.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-6-git-send-email-charles.myers@spirent.com>
classifier changed to object pointer instead of embedded in struct ifnet
to reduce compile dependencies.
There was also issue with including ip6.h in this file, but that could probably
be fixed with struct ip6_hdr forward delcaration and including in6.h instead.

Signed-off-by: Charles Myers <charles.myers@spirent.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Message-Id: <1533610192-4019-7-git-send-email-charles.myers@spirent.com>
lltable_foreach(), lltable_foreach_lle() added to support NETLINK sockets

Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-8-git-send-email-charles.myers@spirent.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-9-git-send-email-charles.myers@spirent.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-10-git-send-email-charles.myers@spirent.com>
Use AF_INET instead of AF_UNIX

Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-11-git-send-email-charles.myers@spirent.com>
…socket

libc code is from musl library with minor change to prevent partial reads if faster than sender.

Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-12-git-send-email-charles.myers@spirent.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-13-git-send-email-charles.myers@spirent.com>
…ESTAMP

l_size_t in linux.h was not correct size for 64-bit long

Signed-off-by: Charles Myers <charles.myers@spirent.com>
Message-Id: <1533610192-4019-14-git-send-email-charles.myers@spirent.com>
…ESTAMP

Signed-off-by: Charles Myers <charles.myers@spirent.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Message-Id: <1533610192-4019-16-git-send-email-charles.myers@spirent.com>
Signed-off-by: Charles Myers <charles.myers@spirent.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Message-Id: <1533610192-4019-17-git-send-email-charles.myers@spirent.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
# Conflicts:
#	modules/tests/Makefile
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
# Conflicts:
#	modules/tests/Makefile
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
# Conflicts:
#	Makefile
#	modules/tests/Makefile
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
wkozaczuk and others added 5 commits October 16, 2022 23:59
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Merge the upstream ipv6 branch (FreeBSD 9 netinet6 import + OSv glue) and add
boot-time IPv6 autoconfiguration:

- netinet6 stack (ip6, icmp6, nd6, in6_*, udp6, raw_ip6, mld6) behind conf-INET6
- OSv net-channel fast path, LRO, RCU classifier extended for IPv6/TCP
- Linux socket-option compat for 43 IPV6_* options; getifaddrs/netlink v6
- SLAAC: kernel Router Solicitation (nd6_rs_output) + accept-RA per-interface
  flag fix so a unikernel (no rtsold) autoconfigures from Router Advertisements
- DHCPv6 client (RFC 8415 stateful: SOLICIT/ADVERTISE/REQUEST/REPLY, DUID-LLT,
  IA_NA) triggered when the RA sets the Managed (M) flag, e.g. AWS VPC
- loader picks SLAAC vs DHCPv6 from RA flags; --ipv6 for IPv6-only boot
- all IPv6 code compile-gated (conf-INET6, conf_networking_dhcp6)
- tests: tst-tcp-v6-smoke, tst-ipv6-internet, tst-ifaddr-dump
@gburd
gburd force-pushed the pr/virtio-net-ipv6-offload branch from b01c483 to 604b865 Compare July 14, 2026 11:02
…r and DHCPv6 client

Security hardening for the two network-reachable memory-safety issues in the
new IPv6/DHCPv6 code (found in audit):

1. classify_ipv6_tcp() OOB read (core/net_channel.cc). ip6_lasthdr_nofrag only
   guarantees the base IPv6 header + extension headers fit; it did NOT
   guarantee a full TCP header is present and contiguous, yet the code then
   dereferenced tcp_hdr (th_flags at +13, ports at +0..3) on the lock-free RX
   fast path. A crafted IPv6/TCP frame (short, or with extension headers
   pushing the L4 offset toward the end, or split across a mergeable-buffer
   mbuf chain via GRO/jumbo) could read out of bounds in the driver softirq.
   Require nxt_off + sizeof(tcphdr) <= mh_len before touching the TCP header;
   otherwise fall through to the slow path (tcp_input), which pulls up.

2. DHCPv6 parse() OOB read across an mbuf chain (bsd/porting/dhcp6.cc). The
   option walk bounded off against pkthdr.len (whole chain) while reading via
   mtod (first mbuf only); udp6_input makes only the UDP header contiguous. A
   malicious on-link DHCPv6 server (DHCPv6 is unauthenticated; the client is
   auto-started on a Managed-flag RA) sending a >4 KiB reply on a jumbo/GRO
   link becomes an mbuf chain and the walk reads past the first mbuf. Clamp
   blen to the contiguous first-mbuf length (a well-formed reply is far smaller
   than one RX buffer).

3. DHCPv6 unbounded state growth (DoS). process_packet() parsed every packet
   even when BOUND, and parse() appended to _dns/_server_duid, so an on-link
   attacker who observed our SOLICIT xid could spray REPLYs and grow those
   without bound. Ignore packets outside SOLICITING/REQUESTING, and reset the
   per-reply learned state only after the xid matches (so a spoofed packet
   cannot wipe good state and repeated replies do not accumulate).

Verified: IPv6 loopback + dual-stack TCP smoke still passes.
@gburd
gburd force-pushed the pr/virtio-net-ipv6-offload branch from 604b865 to 2b961f1 Compare July 14, 2026 12:49
This branch predated the merges of the tgkill/rt_sigtimedwait (osv_sigtimedwait)
and pthread_mutex_timedlock work; its diff vs master had begun to revert them
(and delete tst-signal-fills.cc).  Restore libc/signal.cc, libc/pthread.cc,
linux.cc and tst-signal-fills.cc to master and re-add tst-signal-fills.so to the
test list, so this branch touches only the IPv6 forward-port.
@gburd
gburd force-pushed the pr/virtio-net-ipv6-offload branch from 2b961f1 to 28960e5 Compare July 15, 2026 13:53
gburd added 2 commits July 15, 2026 11:00
# Conflicts:
#	linux.cc
#	modules/tests/Makefile
The virtio-net offload paths were hardcoded to ETHERTYPE_IP: the TX path only
built VIRTIO_NET_HDR_GSO_TCPV4 and the RX checksum path rejected everything but
IPv4 (with a literal "How come - no support for IPv6?!" comment). As a result
IPv6 traffic computed checksums in software and never used TCP segmentation
offload, so high-throughput IPv6 sends/receives burned more CPU than IPv4 even
though the stack (tcp_output) already emits CSUM_TCP_IPV6.

This teaches the driver about IPv6:
- negotiate VIRTIO_NET_F_HOST_TSO6 / VIRTIO_NET_F_GUEST_TSO6
- advertise IFCAP_TSO6 and CSUM_TCP_IPV6 | CSUM_UDP_IPV6 in if_hwassist/caps
- TX: add an ETHERTYPE_IPV6 case that sets csum_start past the 40-byte base
  header and gso_type = VIRTIO_NET_HDR_GSO_TCPV6 (packets with extension
  headers have ip6_nxt != TCP/UDP and fall back to software checksums)
- RX: accept ETHERTYPE_IPV6 in the checksum-offload path (the TCP/UDP checksum
  offset logic is identical for v4 and v6)
- use large receive buffers when GUEST_TSO6 is negotiated too

Boot-tested with IPv6 SLAAC over virtio-net: loopback + concurrent dual-stack
TCP pass and a global address is configured through the modified TX/RX paths.
@gburd
gburd force-pushed the pr/virtio-net-ipv6-offload branch from 28960e5 to defa463 Compare July 15, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants