Instructions for Ipv6 routed NIC without NAT

LXD Support
1

Intro
Hello team, thanks for developing LXD/LXC!
It seems to be a really powerful tool and i have spent a week or so tinkering with it.
Usually i don’t write forum posts, but this time i really need some help.

There is a lot of networking complications, NAT, proxy and forwarding shenaningans in the docs, but i think what i want to achieve must be simpler than that.

Goal
I’d like to assign and route the traffic, targeting one of my available ipv6 adresses, directly into a container (and back).

Environment
My hoster provides me with a /64 block of Ipv6 adresses.
The default gateway is fe80::1

My main host interface is called eth0, and i have set an ipv4 adress, aswell as the ipv6 adress that i want to route in to the container, statically to this interface.

I’m using custom nftables and i’ve incoorperated the rules that LXC introduces. (Please also advise, if i even need them, since i’m not using lxdbr0 or its internal dhcp capabilities)

I am using the alpine/edge image for the container.

I have statically configured the Ipv6 in the container.

Steps taken
So far, as advised in this similar forum post, i have deleted the lxdbr0 interface and created a new container with the nic device type “routed” with its parent beeing my host eth0 interface.

And i assigned the ipv6 to the config aswell as statically in the container OS.

Problems
Ping from inside doesn’t work
Connection from outside doesn’t work

I wanted to post more details regarding my setup, but only have little time after work and it’s been two days of drafting this post, maybe you can already see something wrong.

If not i will post more details and outputs of the usual commands.

2

Please can you show lxc config show <instance> --expanded and ip -6 a and ip -6 r from both host and container.

3

Hi @colorspace, here are the reference docs for routed nic devices: https://documentation.ubuntu.com/lxd/en/latest/reference/devices_nic/#nictype-routed

It is important that the sysctl rules are applied.

Here is a simple example of a routed nic in a container (here using a network managed by LXD, you should be able to skip that and use your host eth0 interface):

lxc network create n1 ipv4.address=192.0.2.1/24 ipv6.address=2001:db8::1/64
sudo sysctl net.ipv4.conf.n1.forwarding=1
sudo sysctl net.ipv6.conf.all.forwarding=1
sudo sysctl net.ipv6.conf.n1.forwarding=1
sudo sysctl net.ipv6.conf.all.proxy_ndp=1
sudo sysctl net.ipv6.conf.n1.proxy_ndp=1
lxc init ubuntu:jammy c1
lxc config device add c1 eth0 nic name=eth0 nictype=routed parent=n1 ipv4.address=192.0.2.254 ipv6.address=2001:db8::ffff
lxc start c1
ping -c2 192.0.2.254
ping6 -c2 2001:db8::ffff

Hope this helps! If not please do post the output that @tomp has asked for, thanks.

4

Thanks for your replies @tomp, @markylaing!

I’ve finally found the motivation to try this again, here are the exact steps, i’ve taken:
(some ip’s were anonymized, but the structure of the ip is the same)

As far as i’ve understood
My hoster allows me all addresses inside 2001:2000:20:20a::/64
And from it, i picked the target ipv6 address 2001:2000:20:20a:c::c1

Host

sudo sysctl net.ipv6.conf.all.forwarding=1
sudo sysctl net.ipv6.conf.eth0.forwarding=1
sudo sysctl net.ipv6.conf.all.proxy_ndp=1
sudo sysctl net.ipv6.conf.eth0.proxy_ndp=1

sudo lxc init ubuntu:jammy c1

sudo lxc config device add c1 eth0 nic name=eth0 nictype=routed parent=eth0 ipv6.address=2001:2000:20:20a:c::c1

ip -6 r

 2001:2000:20:20a::/64 dev eth0 proto kernel metric 256 pref medium
 fe80::/64 dev eth0 proto kernel metric 256 pref medium
 default via fe80::1 dev eth0 proto static metric 1024 pref medium

ip -6 a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2001:2000:20:20a:c::c1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::d550:550:550:550/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever

sudo lxc start c1

Inside C1

vim /etc/netplan/01-netcfg.yaml

    network:
      version: 2
      renderer: networkd
      ethernets:
        eth0:
          dhcp4: true
          dhcp6: false
          addresses:
            - 2001:2000:20:20a:c::c1/64
          nameservers:
            addresses:
              - 2001:4860:4860::8888
              - 2001:4860:4860::8844
          routes:
              - to: "::/0"
                via: "fe80::1"
                on-link: true

netplan apply

Host

sudo lxc restart c1

sudo lxc config show c1 --expanded

architecture: x86_64
config:
  image.architecture: amd64
  image.description: ubuntu 22.04 LTS amd64 (release) (20230815)
  image.label: release
  image.os: ubuntu
  image.release: jammy
  image.serial: "20230815"
  image.type: squashfs
  image.version: "22.04"
  volatile.base_image: ebed156c664de5e9f86c1a2eee7b61031af072af35e90048d2944b3e485f8303
  volatile.cloud-init.instance-id: ef81daf9-78be-479e-98e8-f96e0751b003
  volatile.eth0.host_name: veth8a6d6890
  volatile.eth0.hwaddr: 00:16:3e:17:c9:a1
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.last_state.ready: "false"
  volatile.uuid: 45b624ed-7bfb-4b40-bbaa-2c5d94060242
  volatile.uuid.generation: 45b624ed-7bfb-4b40-bbaa-2c5d94060242
devices:
  eth0:
    ipv6.address: 2001:2000:20:20a:c::c1
    name: eth0
    nictype: routed
    parent: eth0
    type: nic
  root:
    path: /
    pool: default
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

Inside C1

ip -6 a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
7: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2001:2000:20:20a:c::c1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe17:c9a1/64 scope link
       valid_lft forever preferred_lft forever

ip -6 r

2001:2000:20:20a::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::1 dev eth0 proto static metric 1024 onlink pref medium

Host

sudo nft flush ruleset

deactivate firewall completely

Inside C1

ping -6 google.com
^C

timeout

ping 2001:4860:4860::8888

PING 2001:4860:4860::8888(2001:4860:4860::8888) 56 data bytes
From 2001:2000:20:20a:c::c1 icmp_seq=1 Destination unreachable: Address unreachable
From 2001:2000:20:20a:c::c1 icmp_seq=2 Destination unreachable: Address unreachable
From 2001:2000:20:20a:c::c1 icmp_seq=3 Destination unreachable: Address unreachable
^C
--- 2001:4860:4860::8888 ping statistics ---
6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 5065ms

I’m pertty sure something is wrong with this, just not sure what, can you help me out?

5

@tomp I can’t see anything wrong with this configuration. I wonder if you have any insights?

@colorspace just to cover all bases, I assume the DNS servers you have specified in the container are reachable from the host?

6

Yes, the DNS servers specified are googles IPV6 DNS adresses and i can ping those from the Host, but not from the Guest

7

One issue that I can see is that the address inside the container should be configured as /128 subnet rather than /64 otherwise the container’s OS won’t consider the routing table when trying to reach an address in the /64.

8

Also why do you have the container’s IPv6 address 2001:2000:20:20a:c::c1 configured on the LXD host?

9

Hello,
I am sorry for posting here but I cant figure out how can I create new topic. I am login with Ubuntu one and I can’t find any button for create topic on lxd page. Sorry for posting non theme here question and thank you for help.

10

I’ve increased your trust level.

11

Thank you very much.

12

Finally, thanks @tomp!

I’ve corrected the subnet to 128 on the static ip in the container and removed the static configuration on the host.

There was no reason for this, i just incorrectly assumed that this is how its supposed to be set up.

Also i’ve added a DNS server for both ipv6 and ipv4 on the container and tested the connection with iperf3 and everything works wonderfully.

2 Likes
13

Glad to hear you got it solved.

We are also working on a guide for this with cloud-init here:

https://documentation.ubuntu.com/lxd/en/latest/howto/instances_routed_nic_vm/

Although we have some tweaks coming in this PR:

https://github.com/canonical/lxd/pull/12207