Python code on geoip.ubuntu.com crashes on accented data

Ubuntu Support Template

Ubuntu Version:
In theory … all

Desktop Environment (if applicable):
n/a

Problem Description:
The installation process tries to guess the user’s timezone (etc) via a lookup on geoip.ubuntu.com
The code behind that service has problems with geoip data that has accented characters - for example in the timezone field.

Example:
For example - if someone in Costa Rica with an IP address of 102.38.232.1 (picked IP address from h t t p s://lite.ip2location.com/costa-rica-ip-address-ranges were to try it then it generates the error below
I suspect this is because the data that the Python2.7 (!) code is parsing has “San José” in it rather than “San Jose” (capital of Costa Rica).

Maybe the originator of the data has a fault - and they should restrict the characters to regular ASCII.
Perhaps the data ingester should normalise to use fallback characters.
If the data is valid then perhaps the Python code should be changed to normalise with fallback or output as UTF8 … but can the remote consumers of that data handle it?

Relevant System Information:
n/a

Screenshots or Error Messages:

MOD_PYTHON ERROR

ProcessId:      18413
Interpreter:    'geoip.ubuntu.com'

ServerName:     'geoip.ubuntu.com'
DocumentRoot:   '/srv/geoip.ubuntu.com/www'

URI:            '/lookup'
Location:       '/lookup'
Directory:      None
Filename:       '/srv/geoip.ubuntu.com/cgi-bin/lookup.py'
PathInfo:       ''

Phase:          'PythonHandler'
Handler:        'lookup'

Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1537, in HandlerDispatch
    default=default_handler, arg=req, silent=hlist.silent)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1229, in _process_target
    result = _execute_target(config, req, object, arg)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1128, in _execute_target
    result = object(arg)

  File "/srv/geoip.ubuntu.com/cgi-bin/lookup.py", line 61, in handler
    req.write(ans)

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 249: ordinal not in range(128)

What I’ve Tried:
Moving to Florida

No, the installation process does not try to guess the user location as some of the first options a user selects on install are the timezone and language. The link you posted is dead and a ping to the location results in a 100% packet loss which may be temporary? The purpose of the geoip software is described and explained at a number of sites including the one below.

https://www.ipv4.global/shorts/geo-ip/

Your output shows the specific files and the specific lines in each file where the problem lies. It also shows python2.7 which has not been supported for 5 years. That old version of Python should still work but there are no security updates or bug fixes. Which Ubuntu are you using?

This looks like a classic Python 2 Unicode issue.
The GeoIP data is valid — names like “San José” are perfectly correct Unicode.
The crash happens because the Python 2.7 code tries to write Unicode using an ASCII codec.

A minimal fix would be to explicitly encode the output as UTF-8 before req.write().
Long term, this service really should be migrated away from Python 2 entirely.

The full Ubuntu URL is
http://geoip.ubuntu.com/lookup
(there is an optional parameter that I have not included)

This is hosted by Ubuntu (or at least on their domain) so I assume it should really be fixed.

It appears that it used to be used by Ubiquity (old? Ubuntu Live CD installer) … and continues to be used by people outside of the Ubuntu world (which is how I came across it).

I see it is used at this line of code
https://git.launchpad.net/ubuntu/+source/ubuntu-geoip/tree/src/ubuntu-geoip-provider.c#n724
but I don’t know if that module still makes it way into current distributions.
Is it still in geoclue module ?
https://launchpad.net/ubuntu/+source/geoclue-2.0

I have no personal need for it but assumed that since it is hosted by unbuntu.com then someone in Ubuntu might like to know of the issue and resolve it.

If Cloudflare were to try it from their 1.1.1.1 address then they could generate a different error - presumably because there is no geo location set up.

MOD_PYTHON ERROR

ProcessId:      9532
Interpreter:    'geoip.ubuntu.com'

ServerName:     'geoip.ubuntu.com'
DocumentRoot:   '/srv/geoip.ubuntu.com/www'

URI:            '/lookup'
Location:       '/lookup'
Directory:      None
Filename:       '/srv/geoip.ubuntu.com/cgi-bin/lookup.py'
PathInfo:       ''

Phase:          'PythonHandler'
Handler:        'lookup'

Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1537, in HandlerDispatch
    default=default_handler, arg=req, silent=hlist.silent)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1229, in _process_target
    result = _execute_target(config, req, object, arg)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1128, in _execute_target
    result = object(arg)

  File "/srv/geoip.ubuntu.com/cgi-bin/lookup.py", line 52, in handler
    ans += "<Latitude>%.4f</Latitude>" % (gir.location.latitude)

TypeError: float argument required, not NoneType

I’m not sure what the intention of this thread is. Geoclue is Linux software and is not specific to Ubuntu. If there is a problem and you want to report a bug, this is not the place for it as these forums have no control over that. Reporting a bug on Ubuntu is explained at the link below.

https://help.ubuntu.com/stable/ubuntu-help/report-ubuntu-bug.html.en

The output in your posts show the use of python2.7 which hasn’t been supported for years so I doubt you would get a response with a bug report. This software is still included with the current LTS of Ubuntu.

The Python 2.x code is running behind the web server at geoip . ubuntu . com to provide a service.

My intention was to highlight a problem with the service that it provides - I presume it is provided by Ubuntu / Canonical.
If it is actively being used by the current version of geoclue (or anything else) then anyone running it from an IP address where the nearest city has an accent in its name then runs the risk of it generating this error rather than returning the data that they were expecting.

@paulwebster Welcome to Ubuntu Discourse :slight_smile:

As mentioned previously, you need to file a bug report.

This is not the right place to do so but you would be doing the community an important service by reporting your findings.

https://launchpad.net/ubuntu/+source/geoclue

Thanks

The fault is not in geoclue - it is in a back end hosted service provided, I think, by Ubuntu / Canonical that is (or was) used by geoclue and perhaps auto-installer and perhaps Ubiquity.
Is there a source repository or bug-filing location for that service?

I don’t want to raise a bug report in code that is obsolete (geoclue was replaced by geoclue2 I think) but I am not a Ubuntu user nor do I use that back-end service.

Is Ubiquity still in use?
Is the auto-installer documentation still representative of how things work?
This bit in particular …
https://canonical-subiquity.readthedocs-hosted.com/en/latest/reference/autoinstall-reference.html#geoip