SSH from container to Host OS via hostname/domain instead connects to container's localhost

LXD Support
1

Just ran into an issue SSHing from a container to my Host OS using the Host OS hostname/domain name.

The hostname points to the public IP of the server / host OS in public DNS.

However, when connecting via ssh -v I see that ssh (in the container) is resolving that hostname to 127.0.1.1

In the /etc/hosts file of my Host OS, the hostname does point to 127.0.1.1. Specifically:

127.0.0.1 localhost
127.0.1.1 subdomain.example.com subdomain

But that config is not in the /etc/hosts of the container.

It appears as though the LXC container is picking up the Host OS hostname/domain IP as being 127.0.1.1 instead of the server public IP which is actually set in public DNS.

So when I “ssh user@subdomain.example.com” from within the container, I believe the container is simply trying to SSH to itself, due to picking up 127.0.1.1, instead of SSHing to the Host OS, and I receive a denied connection due to the publickey setup on the container.

Using the Host OS public IP or even the lxd private IP 10.0.0.1 enables me to SSH from the container to the Host OS correctly with no problem. But the hostname doesn’t.

Anyhow, maybe this is by design but wanted to point this out in case it’s not supposed to work this way.

2

I did some quick tests on my end and this inheritance of /etc/hosts configuration looks like intended behavior. Maybe we could consider making an exception for localhost IPs though (although I am not sure this is possible). Thanks for the warning :slight_smile:

3

That is AFAIK (as Pedro said) intentional as dnsmasq is configured to consult /etc/hosts for IP ↔ names associations.

If your host has a static IP, you could edit its /etc/hosts file to replace 127.0.1.1 by the static IP that is in the public DNS zone.

AFAIK, the 127.0.1.1 IP + FQDN + short name line is used for situations where a static IP address isn’t known but reliance on the DNS would be unacceptable. sudo trying to resolved the local FQDN being one such use case.

4

It is by design indeed to serve entries from the host’s /etc/hosts file.

There is a --no-hosts option for dnsmasq which is described as:

Don’t read the hostnames in /etc/hosts.

https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html

So this should mean that doing:

lxc network set lxdbr0 raw.dnsmaq="no-hosts"

Should enable that option.

However, despite it being written into /var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.raw it was not taking effect.

I also created a custom LXD binary that started dnsmasq with the command line flag --no-hosts and it still resolved entries in /etc/hosts.

So this looks like a bug in dnsmasq, or that option is not supposed to work the way I think it does :slight_smile:

5

So these settings, from the web hosting company, by default are something like:

127.0.0.1 localhost
127.0.1.1 server#####.webhostingdomain.tld server#####

And when I customized my hostname for the server, as part of that I updated the above to be:

127.0.0.1 localhost
127.0.1.1 subdomain.example.com subdomain

The hosting company’s default hostname does not have a public IP in DNS, so as you mentioned that’s probably why they have it set to 127.0.1.1 by default.

So with my customized/updated hostname, which is in the public DNS zone, the proper setting should rather be the public IP for the hostname instead of 127.0.1.1?

6

That is correct, that way the instance that will inherit that hostame definition through dnsmasq will use the public IP for the host when resolving the subdomain.example.com hostname.

Removing subdomain.example.com from /etc/hosts entirely should make the LXD instances rely on DNS to resolve this hostname, but I assume you would rather leave it defined on /etc/hosts so the approach you outlined should be more appropriate.

7

Found reference to this issue in the Debian docs at https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_hostname_resolution

For a system with a permanent IP address, that permanent IP address should be used here instead of 127.0.1.1 .

8

One more idea @john2k , you could overwrite the /etc/hosts configuration for some hostnames by editing a custom hosts file and using it to overwrite the /etc/hosts with lxc network set lxdbr0 raw.dnsmasq="addn-hosts=/path/to/custom/hosts".
Keep in mind this just overwrites the configuration for hostnames specified in both /etc/hosts and your custom file, anything else defined in /etc/hosts will work normally.
Hope this helps.

1 Like