--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: linux-image-4.14.0-0.bpo.3-amd64: NAT66 (IPv6 SNAT masquerade) does not rewrite source address on kernel 4.14
- From: Rob Andrews <rob@aphlor.org>
- Date: Thu, 29 Mar 2018 21:40:38 +0100
- Message-id: <20180329204035.p7xiolzbyfr7v7bj@sulphur.aphlor.org>
Package: src:linux
Version: 4.14.13-1~bpo9+1
Severity: normal
Dear Maintainer,
I use Docker for application segregation on a remote server. The host has a single v4 and a single v6 IP address allocated. On the host system, I have NAT66 (SNAT, masquerading) setup with the following rules in order to provide segregated IPv6 connectivity to Docker containers:
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER all any any anywhere anywhere ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER all any any anywhere anywhere ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all any docker0 anywhere anywhere ADDRTYPE match src-type LOCAL
0 0 MASQUERADE all any !docker0 fd00:bee:cafe::/64 anywhere
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
On the standard stretch 4.9.x kernel, NAT66 works just fine as witnessed by this tcpdump of a ping (note that the 2001:: address is that of the AAAA record for debian.org):
% sudo tcpdump -i ens3 -n 'not tcp and not udp'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes
19:56:24.980979 IP6 2a03:my:machines:v6:addr > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 0, length 64
19:56:24.988597 IP6 2001:41c8:1000:21::21:4 > 2a03:my:machines:v6:addr: ICMP6, echo reply, seq 0, length 64
19:56:25.981716 IP6 2a03:my:machines:v6:addr > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 1, length 64
19:56:25.989010 IP6 2001:41c8:1000:21::21:4 > 2a03:my:machines:v6:addr: ICMP6, echo reply, seq 1, length 64
19:56:26.982894 IP6 2a03:my:machines:v6:addr > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 2, length 64
1 9:56:26.990022 IP6 2001:41c8:1000:21::21:4 > 2a03:my:machines:v6:addr: ICMP6, echo reply, seq 2, length 64
Whilst on the backports kernel 4.14.x, NAT66 fails to rewrite the source address, and all IPv6 traffic (not just ICMP) fails as a result:
% sudo tcpdump -i ens3 -n 'not tcp and not udp'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes
20:00:39.711554 IP6 fd00:bee:cafe::242:ac11:2 > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 0,length 64
20:00:40.712591 IP6 fd00:bee:cafe::242:ac11:2 > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 1,length 64
20:00:41.713768 IP6 fd00:bee:cafe::242:ac11:2 > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 2,length 64
20:00:42.714934 IP6 fd00:bee:cafe::242:ac11:2 > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 3,length 64
20:00:43.716088 IP6 fd00:bee:cafe::242:ac11:2 > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 4,length 64
20:00:44.717264 IP6 fd00:bee:cafe::242:ac11:2 > 2001:41c8:1000:21::21:4: ICMP6, echo request, seq 5,length 64
The expected behaviour is what is witnessed with kernel 4.9.x - the source address is rewritten with the outgoing interface address.
Ignoring the Docker aspect of this, the witnessed behaviour can be reproduced without having to use a container: use 'ping6 -I <my-ula-address> debian.org'. You'll need to setup some NAT66 rules and create a virtual interface with a ULA address to simulate the behaviour - this stanza in an interfaces file will suffice (make sure you have bridge-utils installed):
iface br-nat-virt inet6 static
bridge_ports none
address fd00:bee:f00d:cafe::1/64
And the following NAT66 rule:
ip6tables -A POSTROUTING -s fd00:bee:f00d:cafe::/64 ! -o br-nat-virt -j MASQUERADE
Then reproduce the behaviour using 'ping6 -I fd00:bee:f00d:cafe::1 debian.org'.
I hope this makes sense. It's not a major biggy, I'm just falling back to using the non-backports kernel in stretch in the meantime.
Kind regards,
rob.
-- Package-specific info:
** Version:
Linux version 4.14.0-0.bpo.3-amd64 (debian-kernel@lists.debian.org) (gcc version 6.3.0 20170516 (Debian 6.3.0-18)) #1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14)
** Command line:
BOOT_IMAGE=/boot/vmlinuz-4.14.0-0.bpo.3-amd64 root=UUID=be3dc831-e7fa-449f-94e9-d302faa31fdb ro rootflags=subvol=ROOT/default
** Not tainted
** Kernel log:
Unable to read kernel log; any relevant messages should be attached
** Model information
sys_vendor: DigitalOcean
product_name: Droplet
product_version: 20171212
chassis_vendor: Bochs
chassis_version:
bios_vendor: DigitalOcean
bios_version: 20171212
** Loaded modules:
xt_nat
xt_tcpudp
veth
ipt_MASQUERADE
nf_nat_masquerade_ipv4
nf_conntrack_netlink
nfnetlink
xfrm_user
xfrm_algo
iptable_nat
nf_conntrack_ipv4
nf_defrag_ipv4
nf_nat_ipv4
iptable_filter
xt_conntrack
br_netfilter
bridge
stp
llc
ip6t_MASQUERADE
nf_nat_masquerade_ipv6
xt_addrtype
ip6table_nat
nf_conntrack_ipv6
nf_defrag_ipv6
nf_nat_ipv6
nf_nat
nf_conntrack
libcrc32c
ip6table_filter
ip6_tables
sb_edac
kvm_intel
kvm
irqbypass
crct10dif_pclmul
crc32_pclmul
qxl
ghash_clmulni_intel
ppdev
ttm
joydev
evdev
button
drm_kms_helper
serio_raw
pcspkr
drm
virtio_balloon
parport_pc
pvpanic
parport
ip_tables
x_tables
autofs4
btrfs
crc32c_generic
xor
zstd_decompress
zstd_compress
xxhash
raid6_pq
ata_generic
crc32c_intel
virtio_blk
virtio_scsi
virtio_net
aesni_intel
aes_x86_64
crypto_simd
cryptd
glue_helper
psmouse
floppy
ata_piix
virtio_pci
virtio_ring
virtio
uhci_hcd
ehci_hcd
i2c_piix4
usbcore
usb_common
libata
scsi_mod
** PCI devices:
not available
** USB devices:
not available
-- System Information:
Debian Release: 9.4
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 4.14.0-0.bpo.3-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages linux-image-4.14.0-0.bpo.3-amd64 depends on:
ii initramfs-tools [linux-initramfs-tool] 0.130
ii kmod 23-2
ii linux-base 4.5
Versions of packages linux-image-4.14.0-0.bpo.3-amd64 recommends:
pn apparmor <none>
pn firmware-linux-free <none>
pn irqbalance <none>
Versions of packages linux-image-4.14.0-0.bpo.3-amd64 suggests:
pn debian-kernel-handbook <none>
ii grub-pc 2.02~beta3-5
pn linux-doc-4.14 <none>
Versions of packages linux-image-4.14.0-0.bpo.3-amd64 is related to:
pn firmware-amd-graphics <none>
pn firmware-atheros <none>
pn firmware-bnx2 <none>
pn firmware-bnx2x <none>
pn firmware-brcm80211 <none>
pn firmware-cavium <none>
pn firmware-intel-sound <none>
pn firmware-intelwimax <none>
pn firmware-ipw2x00 <none>
pn firmware-ivtv <none>
pn firmware-iwlwifi <none>
pn firmware-libertas <none>
pn firmware-linux-nonfree <none>
pn firmware-misc-nonfree <none>
pn firmware-myricom <none>
pn firmware-netxen <none>
pn firmware-qlogic <none>
pn firmware-realtek <none>
pn firmware-samsung <none>
pn firmware-siano <none>
pn firmware-ti-connectivity <none>
pn xen-hypervisor <none>
-- no debconf information
--- End Message ---