I’ve been running the lighttpd server on ubuntu 23/Gnome for development work for over a year. Everything is fine. I upgrade to Ubuntu 24, and suddenly I cannot get lighttpd to report anything but UTC. My local time is fine. My RTC is the same as UTC. NTP is working. (RTP in local TZ is “off”, but it doesn’t change when it’s on.) No matter what I do, any application run in lighttpd is stuck in UTC time. Can anyone figure out what is wrong? Here is my timedatectl output:
Local time: Wed 2025-03-12 21:27:23 CDT
Universal time: Thu 2025-03-13 02:27:23 UTC
RTC time: Thu 2025-03-13 02:27:22
Time zone: America/Chicago (CDT, -0500)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
And, yes, I’ve been through setting the time in bios, setting it via timedatectl, and loading the special setenv module in lighttpd, and defining the TZ explicitly.
Seems like it’s getting the time from the RTC; have you tried:
hwclock --systohc --localtime
There’s a pretty intense man page on it… Traditionally this was an issue between Windows and linux as they both had different ways of setting the RTC which would munge the system clock. I don’t know about these days. I don’t bother much with Windows.
hwclock didn’t work. It’s either a lighttpd problem or an ubuntu 24 problem, I can’t make sense of it. It doesn’t seem to matter what is done to the RTC or HWC. This hasn’t been a problem the last few months, but I’m working on a project now that requires me to sync times and data across servers, and it’s making it almost impossible
A year or so back I had a very similar issue, So I’ll offer this attempt to help you.
If Lighttpd is stuck displaying UTC time despite setting the timezone, it might be due to issues with the NTP synchronization or the RTC (Real-Time Clock) settings.
Disabling NTP synchronization and then reactivating the systemd-timesyncd service can help resolve the issue.
Just additional info:
Additionally, setting the local RTC to use UTC can be done using the command VBoxManage setextradata “your-VM-name” “VBoxInternal/Devices/VMMDev/0/Config/UseUTC” 1 on a VirtualBox guest.
To ensure Lighttpd functions correctly with the timezone settings, it is recommended to check if the systemd-timedated service is running properly. If the service fails to start due to the ProtectHome setting being enabled, you can disable it and restart the service as follows:
This should correct the time display and retain the timezone settings in the system.
NOTE: I was not using the snap version of Lighttpd.
timedatectl
Local time: Fri 2025-03-14 12:12:41 MDT
Universal time: Fri 2025-03-14 18:12:41 UTC
RTC time: Fri 2025-03-14 18:12:41
Time zone: America/Denver (MDT, -0600)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
Test as an unprivileged user: printf 'server.document-root = "/tmp"\nserver.port=8080\n' | lighttpd -D -f -
Then Ctrl-C. Is the time printed in UTC or your local timezone? The trace should be in the local time zone.
If you are seeing only UTC in the accesslog.filename, then look into how lighttpd was built. If the build configuration fails to detect tm_gmtoff in struct tm, then mod_accesslog prints the accesslog using gmtime() instead of localtime().
Did Ubuntu change how it builds lighttpd? lighttpd can be built using autoconf, meson, CMake, or SCons, and maybe one of those build systems is not detecting tm_gmtoff on Ubuntu?
This is a good point @gstrauss brings to light (No Pun intended)
Build Systems for Lighttpd
Autoconf: This is a traditional build system that has been widely used for building Lighttpd. It is known for its reliability and compatibility with various systems.
Meson: Meson is a modern build system that has gained popularity for its simplicity and speed. However, it has been noted that Meson does not install man pages by default, which is a known issue.
CMake: CMake is another modern build system that is widely used. It has been reported that CMake has issues with detecting tm_gmtoff and other configurations. Specifically, there have been problems with the detection of MySQL headers and libraries, which are required for certain modules
SCons: SCons is a Python-based build system that is also supported for building Lighttpd.
I recall reading about Issues with CMake.
One of the issues with CMake is that it does not correctly handle the detection of tm_gmtoff. This can lead to build failures or incorrect behavior. Additionally, CMake has been reported to have issues with the detection of MySQL headers and libraries, which can affect the build process for modules that depend on MySQL
While Lighttpd supports multiple build systems, there are known issues with Meson and CMake that may affect the build process. If you encounter issues with tm_gmtoff or other configurations, you might want to consider using autoconf or SCons, which have fewer reported issues in this regard.
@1fallen thank you for providing your detailed time-related troubleshooting steps in your post further above.
FYI: Regarding the lighttpd build options, I am a lighttpd developer and do not know of any specific issues with Meson or CMake. Personally, I prefer Meson.
I want to ask @geekgirl where they are seeing the UTC, e.g. only in the accesslog or elsewhere, too?
If there are bugs in the lighttpd build scripts for those other build options, I’d like to fix them. autoconf/autotools is the historical default and gets the most use. Meson is used to build lighttpd in numerous distros, including OpenWRT.
I just tested lighttpd builds with autoconf, meson, cmake, and scons and tm_gmtoff is properly detected by the build on my Fedora Linux system. While this is worth verifying in the Ubuntu build, there does not appear to be a bug in lighttpd build scripts.
FTR, if I fixed something over two years ago and $distro is negligent in backporting, then that is the reason that I generally tell people there are better distros than Ubuntu (based on Debian). Ubuntu and Debian releases are not “maintained”, they are “neglected”. neglected != maintained. (Volunteers or not, neglected is neglected and users tripping over bugs fixed over two years ago upstream, but not in (recent) $distro release, constitute neglect by $distro to maintain the release.) This is not a dig at any specific person, but is a logical conclusion based on repeated and voluminous observations of Ubuntu and Debian releases often being far out-of-date from software which is maintained upstream. Other $distros generally do a better job at using up-to-date upstream releases. (To be fair, Ubuntu and Debian releases do generally pick up security fixes if there is a CVE, but again, Ubuntu and Debian are often slow doing so, and if something is not a CVE, Ubuntu and Debian generally are slow or do not ever get around to backporting fixes to Ubuntu and Debian releases.)
ok… wow. I was away and busy for a day, and the thread suddenly explodes with feedback and ideas! Now give me a moment to wake up, read all this and report back lol You guys are da best
ok… finally getting back into things. (My project manager just demanded an entire website layout be converted from a horizontal to a vertical orientation - not a responsive design thing, just a general layout thing. Ugh.)
The server.document yields this:
2025-03-19 16:57:52: (server.c.1939) server started (lighttpd/1.4.76)
And timedatectl yields this:
Local time: Wed 2025-03-19 16:58:36 CDT
Universal time: Wed 2025-03-19 21:58:36 UTC
RTC time: Wed 2025-03-19 16:58:35
Time zone: America/Chicago (CDT, -0500)
System clock synchronized: yes
NTP service: active
RTC in local TZ: yes
This appears correct(!)… but I am still forcing RTC into local. And, despite this, lighttpd is still stuck in UTC
The idea of a build bug is intriguing… I forget how it was installed. Probably synaptic, but it definitely was not snap. Could there have been an issue if lighttpd was installed before the system had rebooted with other newly-installed packages?
The line you posted from your lighttpd error log with “server started” is in localtime.
Where is lighttpd still stuck in UTC? The accesslog?
As I described above the accesslog uses localtime_r() if at build time of the lighttpd package, the lighttpd build detected tm_gmtoff, otherwise the accesslog uses gmtime_r() and date output is in UTC.
Before starting scripts, lighttpd clears the environment and then, for CGI, adds the CGI HTTP environment for the request, which means that shells and scripts use whatever defaults they have when environment is empty, e.g. PATH is empty, TMPDIR is empty, TZ is empty, etc.
If you want to set entries in the environment, see https://wiki.lighttpd.net/mod_fastcgi"bin-environment" and "bin-copy-environment", or for CGI, https://wiki.lighttpd.net/mod_setenvsetenv.set-environment, and you can set PATH, TMPDIR, TZ, etc. [Edit: for starting FastCGI, SCGI, other backend servers from lighttpd, the environment is inherited from lighttpd unless "bin-copy-environment" is set.]
This is historical behavior in lighttpd; it is not new behavior in lighttpd, so I do not know what changed in Ubuntu to change the behavior of your scripts.
Interesting… I will check this later today. I am not using fastCGI, just the regular 10-CGI.conf. My previous 3 installations of lighttpd did this as well, without any issues. In fact, when I installed ubuntu24, I literally used the same configs, to ensure consistency. I’ve been using lighttpd for local dev work for almost 6yrs now, generally like it, and this is my first problem with it. I’ve installed lighttpd on several dozen systems by now - desktops and especially raspbianOS - and never run into this problem before. In fact, I have 3 raspberryPis running raspbianOS and a beelink running ubuntu23 in my house right now doing various things; none of them exhibit this problem. (Which is why I focused on ubuntu24 as the source)
