How to manage Multipass on macOS

Multipass Documentation
,
1

Set the driver to VirtualBox

If you want to (or have to) use Multipass with VirtualBox as the hypervisor, you need to tell Multipass so:

$ multipass set local.driver=virtualbox

From then on, all instances started with multipass launch will use VirtualBox behind the scenes.

Contents:

Finding Multipass instances in VirtualBox

Multipass runs as the root user, so to see the instances in VirtualBox, or through the VBoxManage command, you have to run those as root, too:

$ sudo VirtualBox

Multipass instances in VirtualBox

To list the instances on the command line:

$ sudo VBoxManage list vms
"primary" {395d5300-557d-4640-a43a-48100b10e098}

NOTE: You can still use the multipass client and the system menu icon, and any changes you make to the configuration of the instances in VirtualBox will be persistent. They may not be represented in Multipass commands such as multipass info , though.

Port forwarding

To expose a service running inside the instance on your host, you can use VirtualBox’s port forwarding feature, for example:

$ sudo VBoxManage controlvm "primary" natpf1 "myservice,tcp,,8080,,8081"

You can then open, say, http://localhost:8081/, and the service running inside the instance on port 8080 will be exposed.

Bridging

An often requested Multipass feature is network bridging. You can add a second network interface to the instance and expose it on your physical network. You will need to stop the instance:

$ multipass stop primary

Find the network interface you want to bridge with (you want the identifier before the second colon):

$ VBoxManage list bridgedifs | grep ^Name:
Name:            en0: Ethernet
Name:            en1: Wi-Fi (AirPort)
Name:            en2: Thunderbolt 1
Name:            en3: Thunderbolt 2
...

And tell VirtualBox to use it as the “parent” for the second interface (see more info on bridging in VirtualBox documentation on the topic):

# Do not touch --nic1 as that's in use by Multipass
$ sudo VBoxManage modifyvm primary --nic2 bridged --bridgeadapter2 en0

You can then start the instance again and find the name of the new interface:

$ multipass start primary
$ multipass exec primary ip link | grep DOWN
3: enp0s8:  mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

And configure that new interface — Ubuntu uses netplan for that:

$ multipass exec -- primary sudo bash -c "cat > /etc/netplan/60-bridge.yaml" <<EOF
network:
  ethernets:
    enp0s8:                  # this is the interface name from above
      dhcp4: true
      dhcp4-overrides:       # this is needed so the default gateway
        route-metric: 200    # remains with the first interface
  version: 2
EOF
$ multipass exec primary sudo netplan apply

Finally, find the IP of the instance given by your router:

$ multipass exec primary ip address show dev enp0s8 up       
3: enp0s8:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:2a:5f:55 brd ff:ff:ff:ff:ff:ff
    inet 10.2.0.39/24 brd 10.2.0.255 scope global dynamic enp0s8
       valid_lft 86119sec preferred_lft 86119sec
    inet6 fe80::a00:27ff:fe2a:5f55/64 scope link 
       valid_lft forever preferred_lft forever

All the services running inside the instance should now be available on your physical network under http://<the ip>/.

If you want to switch back to the default driver:

$ sudo multipass set local.driver=hyperkit

Instances created with VirtualBox don’t get transferred, but you can always come back to them.

How to use a different terminal on macOS

See also: Install on macOS, shell.

If you want a different terminal application to be used by the Multipass system menu icon, you need to tell macOS to use that terminal for the .command file type. This document presents two ways of achieving this.

Contents:

Using duti

duti is a small helper application that can modify the default application preferences. It’s also available from brew.
Find out your preferred terminal’s bundle identifier using mdls:

$ mdls /Applications/iTerm.app/ | grep BundleIdentifier
kMDItemCFBundleIdentifier              = "com.googlecode.iterm2"

And make it the default for script files using duti:

$ duti -s com.googlecode.iTerm2 com.apple.terminal.shell-script shell

Using Finder

Create an empty file with a .command extension and find it in Finder. Select the file and press ⌘I. You should be presented with an information pane like this:

obraz

Expand the “Open with:” section, select your preferred terminal application and click on “Change All…”.

Change the daemon settings

On macOS you’ll need to add it to the service definition (again, replace <hostname>) and reload it:

$ sudo /usr/libexec/PlistBuddy \
  -c "Add :ProgramArguments: string --address" \
  -c "Add :ProgramArguments: string <hostname>:51001" \
  /Library/LaunchDaemons/com.canonical.multipassd.plist
$ sudo launchctl unload /Library/LaunchDaemons/com.canonical.multipassd.plist
$ sudo launchctl load /Library/LaunchDaemons/com.canonical.multipassd.plist

NB: reinstallation / upgrade will overwrite those changes

Improve mounts performance

Share a folder with SMB in MacOS

This can be entirely done using the GUI, but some steps can easily be done from the command-line. We’ll show the latter when possible.

To enable sharing system wide, we need these commands:

sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.smb.server.plist EnabledServices -array disk

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.smbd.plist

Then, for sharing a specific folder:

sudo sharing -a /my_path/

Finally, we need to allow the user to access the shared folders via SMB. For this, we need to navigate the System Preferences, and find there the Sharing icon. In the Sharing menu, there is a button labeled Options. This button shows us a window, on which we can choose which users will be able to mount. There, we choose and enter the passwords of the users we want to authorize.

All these steps will create a mount point named //hostname/my_path/ in the host.

Share a folder with NFS in MacOS

The NFS server included in MacOS is controlled with the nfsd command and the file /etc/exports. We need to add the folder we want to share to this file, with a line similar to

/my_path -mapall=host_user -network 192.168.0.0 -mask=255.255.0.0

where -network and -mask control the network from which the share can be accessed. Then, we start the server with the command

sudo nfsd start

(or restart if the server is already running).

Get stand-alone windows on X11

The procedure for MacOS should be almost the same as for Linux, but I don’t have a Mac at hand for testing it. Contributions are welcome :slight_smile:

Access logs

The log files on macOS are stored in /Library/Logs/Multipass, where multipassd.log has the daemon messages. You can also see instance logs here - the filename pattern is <instance>-hyperkit.log. You will need sudo to access any of those files by default.

Troubleshoot networking

See also: Install on macOS.

Contents:

Architecture

Multipass uses “hyperkit” to run instances, which utilises MacOS’ Hypervisor.framework. This framework manages the networking stack for the instances.

On creation of an instance, Hypervisor.framework on the host uses MacOS’ “Internet Sharing” mechanism to:

  1. create a virtual switch and connects each instance to it (subnet 192.168.64.*)
  2. provide DHCP and DNS resolution on this switch at 192.168.64.1 (via bootpd & mDNSResponder services running on the host); this is configured by an auto-generated file /etc/bootpd.plist - but editing this is pointless as MacOS re-generates it as it desires.

Note that according to “System Preferences” → “Sharing”, the "Internet Sharing"service can appear disabled - this is ok. In the background, it will still be enabled to support instances.

Tools known to interfere with Multipass

  • VPN software can be aggressive at managing routes, and may route 192.168.64 subnet through the VPN interface, instead of keeping it locally available.
    • Possible culprits: OpenVPN, F5, Dell SonicWall, Cisco AnyConnect, Citrix/Netscaler Gateway, Jupiter Junos Pulse / Pulse Secure
    • Tunnelblick doesn’t cause problems
  • Cisco Umbrella Roaming Client binds to localhost:53 which clashes with Internet Sharing, breaking instance’s DNS (Umbrella Roaming Client OS X and Internet Sharing)
  • dnscrypt-proxy/dnscrypt-wrapper/cloudflared-proxy
    Default configuration binds to localhost:53, clashing with Internet Sharing.
  • another dnsmasq process bound to localhost:53
  • custom DHCP server bound to port 67? (sudo lsof -iUDP:67 -n -P should show launchd and bootpd only)

Problem class

Generic networking problems

Unable to determine IP address usually implies some networking configuration is incompatible, or there is interference from a Firewall or VPN.

Troubleshooting (section to be expanded)

  1. Firewall
    1. Is Firewall enabled?
    2. If so, it must not “Block all incoming connections”
      • Blocking all incoming connections prevents a DHCP server from running locally, to give an IP to the instance.
      • It’s ok to block incoming connections to “multipassd” however.
  2. VPN
  3. Little Snitch - defaults are good, it should permit mDNSResponder and bootpd access to BPF
    If you’re having trouble downloading images and/or see Unknown errors when trying to multipass launch -vvv, Little Snitch may be interfering with multipassd’s network access (ref. #1169)
  4. Internet Sharing - doesn’t usually clash
  5. Is the bootpd DHCP server alive? (sudo lsof -iUDP:67 -n -P should mention bootpd)
    • start it by running sudo launchctl load -w /System/Library/LaunchDaemons/bootps.plist

Network routing problems

Could try

$ sudo route -nv add -net 192.168.64.0/24 -interface bridge100

If you get a “File exists” error, maybe delete and retry?

$ sudo route -nv delete -net 192.168.64.0/24
$ sudo route -nv add -net 192.168.64.0/24 -interface bridge100

Maybe -static route helps?

If using “Cisco AnyConnect” - try using “OpenConnect” (brew install openconnect) instead as it messes with routes less (but your company sysadmin/policy may not permit/authorize this).

Does your VPN software provide a “Split connection” option - where VPN sysadmin can designate a range of IP addresses to not be routed through the VPN.

  • Cisco does
  • Pulse Secure / Jupiter Junos Pulse do

Potential workaround for VPN conflicts (ref: #495)

After the nat … line (if there is one, otherwise at the end) in /etc/pf.conf, add this line:

$ nat on utun1 from bridge100:network to any -> (utun1)

and reload PF with $ sudo pfctl -f /etc/pf.conf.

Possible other option - configure Multipass to use a different subnet?

Edit /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist to change the Shared_Net_Address value to something other than 192.168.64.1 -. This works if you edit the .plist file and stay inside 192.168 range, as Multipass is hardcoded for this.

Note on this:
If you change the subnet and launch an instance, it will get an IP from that new subnet. But if you try changing it back, the change is reverted on next instance start. It appears that the DHCP server reads the last IP in /var/db/dhcpd_leases, decides the subnet from that, and updates Shared_Net_Address to match. So only way to really revert this change is edit/delete /var/db/dhcpd_leases.

DNS problems

Check if you can ping an IP address:

$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
^C
--- 1.1.1.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2030ms

Note that macOS’s firewall can block the ICMP packets that ping uses, which will interfere with this test. Make sure you disable “Stealth Mode” in “System Preferences”->“Security & Privacy” → “Firewall” just for this test.
Security & Privacy
Now try again:

$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=53 time=7.02 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=53 time=5.91 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=53 time=5.12 ms
^C
--- 1.1.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2143ms
rtt min/avg/max/mdev = 5.124/6.020/7.022/0.781 ms

This means the instance can indeed connect to the internet, but DNS resolution is broken. Testing DNS resolution using the dig tool may show it is broken:

$ dig google.ie
; <<>> DiG 9.10.3-P4-Ubuntu <<>> google.ie
;; global options: +cmd
;; connection timed out; no servers could be reached

If it’s working, it should show something like this:

$ dig google.ie
; <<>> DiG 9.10.3-P4-Ubuntu <<>> google.ie
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48163
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.ie.   		 IN    A

;; ANSWER SECTION:
google.ie.   	 15    IN    A    74.125.193.94

;; Query time: 0 msec
;; SERVER: 192.168.64.1#53(192.168.64.1)
;; WHEN: Thu Aug 01 15:17:04 IST 2019
;; MSG SIZE  rcvd: 54

To test further, try supplying an explicit DNS server:

$ dig @1.1.1.1 google.ie
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @1.1.1.1 google.ie
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11472
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1452
;; QUESTION SECTION:
;google.ie.   		 IN    A

;; ANSWER SECTION:
google.ie.   	 39    IN    A    74.125.193.94

;; Query time: 6 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Thu Aug 01 15:16:27 IST 2019
;; MSG SIZE  rcvd: 54

This implies the problem is with macOS’s “Internet Sharing” feature - for some reason its built-in DNS server is broken.

The built-in DNS server should be mDNSResponder which binds to localhost:53.

If you’re using Little Snitch or another per-process firewall, ensure mDNSResponder can establish outgoing connections. MacOS’ built-in firewall should not interfere with it.

Check what is bound to that port on the host:

$ sudo lsof -iTCP:53 -iUDP:53 -n -P
COMMAND   PID       	USER   FD   TYPE         	DEVICE SIZE/OFF NODE NAME
mDNSRespo 191 _mdnsresponder   17u  IPv4 0xa89d451b9ea11d87  	0t0  UDP *:53
mDNSRespo 191 _mdnsresponder   25u  IPv6 0xa89d451b9ea1203f  	0t0  UDP *:53
mDNSRespo 191 _mdnsresponder   50u  IPv4 0xa89d451b9ea8b8cf  	0t0  TCP *:53 (LISTEN)
mDNSRespo 191 _mdnsresponder   55u  IPv6 0xa89d451b9e2e200f  	0t0  TCP *:53 (LISTEN)

The above output shows the correct state while a instance is running. If no instance is running (and Internet Sharing disabled in System Preferences), the command should return nothing.

Any other command appearing in that output means a process is conflicting with Internet Sharing, and thus will break the DNS in the instance.

Possible workarounds

  • Configure DNS inside the instance to use an external working DNS server. Can do so by appending this line to /etc/resolv.conf manually:

    # 1.1.1.1 is a free DNS service provided by CloudFlare, but you can use your own.
$ nameserver 1.1.1.1

  • Use a custom cloud-init to set /etc/resolv.conf for you on first boot.

ARP problems

The macOS bridge used for hyperkit filters packets so that only the IP address originally assigned to the VM is allowed through. If you add an additional address (e.g. IP alias) to the VM, the ARP broadcast will get through but the ARP response will be filtered out.

This means that applications which rely on additional IP addresses, such as metallb under microk8s, will not work.

2 Likes