Service - Migrating from OpenLDAP 2.4.x to 2.5.x

sergiodj · August 19, 2021, 7:53pm

Changes to OpenLDAP 2.5.x

It took more than 10 years for OpenLDAP 2.5.x to be released, and as
such it is important to understand the changes that can impact
production environments and how to deal with them.

The most important change in this release is the removal of the
following backends:

BDB
HDB
Shell

Installations using these backends will need to migrate to a supported
backend instead, like LMDB (preferred).

The nssov overlay has also been dropped from the package.

There are also a few additions:

The new slapd-asyncmeta(5) backend.
New core overlays: slapd-homedir(5), slapd-otp(5) and
slapd-remoteauth(5).

At the time of this writing, you can find the upstream announcement
containing the full list of changes in the 2.5.x series at
https://www.openldap.org/software/release/announce.html.

Migrating your configuration to OpenLDAP 2.5.x

The package upgrade process (using maintainer scripts) exports your
databases to LDIF format using slapcat(8), then updates the slapd
package, and finally imports the LDIF files using slapadd(8). If
the slapadd process fails, it must be completed manually after
resolving whatever issues caused by the failure.

Generally, these issues will likely be related to the use of old
backends or overlays that have been removed in the 2.5.x series. For
example, you can encounter problems if you are still deploying
OpenLDAP using a BDB or an HDB backend. In this case, the upgrade
process will not be able to automatically migrate the existing backend
to LMDB, and you will have to manually intervene.

In this document we present the most common failure scenario and how
to fix the problem. Bear in mind that there might be more complex
cases which are not covered here and that will likely require more
actions to be resolved.

For testing purposes, we are using an LXD container configured with
OpenLDAP 2.4.x and deployed with a BDB backend. This container has
been upgraded all the way up to Ubuntu Impish, which ships with
OpenLDAP 2.5.x, but the OpenLDAP packages were held back. Let’s now
upgrade them:

# apt-get update
...
# apt-get upgrade
...
Setting up slapd (2.5.6+dfsg-1~exp1ubuntu1) ...
Installing new version of config file /etc/ldap/schema/core.ldif ...
Installing new version of config file /etc/ldap/schema/core.schema ...
Installing new version of config file /etc/ldap/schema/cosine.ldif ...
Installing new version of config file /etc/ldap/schema/cosine.schema ...
Installing new version of config file /etc/ldap/schema/dyngroup.ldif ...
Installing new version of config file /etc/ldap/schema/dyngroup.schema ...
Installing new version of config file /etc/ldap/schema/pmi.ldif ...
Installing new version of config file /etc/ldap/schema/pmi.schema ...
  Backing up /etc/ldap/slapd.d in /var/backups/slapd-2.4.57+dfsg-2ubuntu1... done.
  Moving old database directories to /var/backups:
  - directory dc=example,dc=com... done.
  Loading from /var/backups/slapd-2.4.57+dfsg-2ubuntu1: 
  - directory dc=example,dc=com... failed.

Loading the database from the LDIF dump failed with the following
error while running slapadd:
    lt_dlopenext failed: (back_bdb) file not found
    config error processing cn=module{0},cn=config: <olcModuleLoad> handler exited with 1
    slapadd: bad configuration directory!
Removing obsolete conffile /etc/ldap/schema/ppolicy.schema ...
Removing obsolete conffile /etc/ldap/schema/ppolicy.ldif ...
Job for slapd.service failed because the control process exited with error code.
See "systemctl status slapd.service" and "journalctl -xeu slapd.service" for details.
invoke-rc.d: initscript slapd, action "start" failed.
× slapd.service - LSB: OpenLDAP standalone server (Lightweight Directory Access Protocol)
     Loaded: loaded (/etc/init.d/slapd; generated)
    Drop-In: /usr/lib/systemd/system/slapd.service.d
             └─slapd-remain-after-exit.conf
             /run/systemd/system/service.d
             └─zzz-lxc-service.conf
     Active: failed (Result: exit-code) since Wed 2021-08-18 17:42:39 UTC; 6ms ago
       Docs: man:systemd-sysv-generator(8)
    Process: 1713 ExecStart=/etc/init.d/slapd start (code=exited, status=1/FAILURE)

Aug 18 17:42:39 openldap-test-db-migration slapd[1718]: lt_dlopenext failed: (back_bdb) file not found
Aug 18 17:42:39 openldap-test-db-migration slapd[1718]: config error processing cn=module{0},cn=config: <olcModuleLoad> handler exited with 1
Aug 18 17:42:39 openldap-test-db-migration slapd[1718]: DIGEST-MD5 common mech free
Aug 18 17:42:39 openldap-test-db-migration slapd[1718]: DIGEST-MD5 common mech free
Aug 18 17:42:39 openldap-test-db-migration slapd[1718]: slapd stopped.
Aug 18 17:42:39 openldap-test-db-migration slapd[1718]: connections_destroy: nothing to destroy.
Aug 18 17:42:39 openldap-test-db-migration slapd[1713]:    ...fail!
Aug 18 17:42:39 openldap-test-db-migration systemd[1]: slapd.service: Control process exited, code=exited, status=1/FAILURE
Aug 18 17:42:39 openldap-test-db-migration systemd[1]: slapd.service: Failed with result 'exit-code'.
Aug 18 17:42:39 openldap-test-db-migration systemd[1]: Failed to start LSB: OpenLDAP standalone server (Lightweight Directory Access Protocol).
...

As you can see, the upgrade process has failed. You should also have
seen a debconf warning saying:

Error while performing post-installation tasks

There has been one or more errors while performing some
post-installation tasks. This probably means that the slapd package
could not automatically migrate one or more LDAP databases, or that a
backend being used by the current OpenLDAP installation is not
supported anymore.

The maintainer script responsible for executing the post-installation
tasks has exited, but the slapd service has NOT been (re)started. You
will need to manually fix the problem and then start the service.

For more information on possible problematic scenarios and how to
address them, please take a look at the README.Debian file (under
/usr/share/doc/slapd/).

This means that the slapd service has not been properly restarted
and will require manual intervention (as explained above), but in
order to prevent upgrade breakages the apt-get upgrade command has
finished with a successful code anyway.

We now need to proceed with the manual steps required to convert our
OpenLDAP installation from BDB to the LMDB backend. Let’s assume that
your configuration database is stored in the default location
(/etc/ldap/slapd.d).

You need to locate the backup LDIF file exported during the upgrade
process. This is the configuration file used by your
installation. It should be located at:

/var/backups/slapd-<OLDVERSION>/cn=config.ldif

<OLDVERSION> should most likely be the latest 2.4.x version you
had installed before attempting to upgrade to 2.5.x. In our
example, our file is named
/var/backups/slapd-2.4.57+dfsg-2ubuntu1/cn=config.ldif.

Let’s make a copy of the file for working on.
```
# mkdir /tmp/openldap-migration
# cp /var/backups/slapd-2.4.57+dfsg-2ubuntu1/cn=config.ldif /tmp/openldap-migration
```

Edit your copy of the cn=config.ldif file to fix the issues noted
by slapdadd, such as removed or renamed modules or backends.
In our case, here is what we should do:

Change olcModuleLoad: back_bdb or back_hdb to back_mdb.
If you have an olcBackend: bdb or hdb entry, change it to mdb, or
delete it if you don’t have to override any global LMDB settings.
For each configured BDB or HDB database:
Change objectClass: olcBdbConfig or olcHdbConfig to
olcMdbConfig. Also update structuralObjectClass.
Change olcDatabase: bdb or hdb to mdb. Also update the
attribute in the DN, for example:
olcDatabase={1}mdb,cn=config.
Delete any olcDbConfig attributes.
Add the olcDbMaxSize attribute to set the maximum size of the
database, in bytes. If not configured, the default is 10 MiB.

To illustrate the changes that are necessary here, here is a diff
from the original vs. the modified files in the example scenario
we’re considering in this guide:

--- /var/backups/slapd-2.4.57+dfsg-2ubuntu1/cn=config.ldif	2021-08-18 17:42:11.350169777 +0000
+++ /tmp/openldap-migration/cn=config.ldif	2021-08-18 18:18:20.724858076 +0000
@@ -17,7 +17,7 @@
 objectClass: olcModuleList
 cn: module{0}
 olcModulePath: /usr/lib/ldap
-olcModuleLoad: {0}back_bdb
+olcModuleLoad: {0}back_mdb
 structuralObjectClass: olcModuleList
 entryUUID: 216f3358-632c-103b-8f5c-159102fdad19
 creatorsName: cn=config
@@ -617,9 +617,9 @@
 modifiersName: cn=config
 modifyTimestamp: 20210616202109Z
 
-dn: olcBackend={0}bdb,cn=config
+dn: olcBackend={0}mdb,cn=config
 objectClass: olcBackendConfig
-olcBackend: {0}bdb
+olcBackend: {0}mdb
 structuralObjectClass: olcBackendConfig
 entryUUID: 216fb710-632c-103b-8f5d-159102fdad19
 creatorsName: cn=config
@@ -658,10 +658,10 @@
 modifiersName: cn=config
 modifyTimestamp: 20210616202109Z
 
-dn: olcDatabase={1}bdb,cn=config
+dn: olcDatabase={1}mdb,cn=config
 objectClass: olcDatabaseConfig
-objectClass: olcBdbConfig
-olcDatabase: {1}bdb
+objectClass: olcMdbConfig
+olcDatabase: {1}mdb
 olcDbDirectory: /var/lib/ldap
 olcSuffix: dc=example,dc=com
 olcAccess: {0}to attrs=userPassword by self write by anonymous auth by * non
@@ -672,15 +672,11 @@
 olcRootDN: cn=admin,dc=example,dc=com
 olcRootPW:: e1NTSEF9UDFDMlA1ZDhxbzZuTWR0c0p3NU50c3BBLzJBSm54T0s=
 olcDbCheckpoint: 512 30
-olcDbConfig: {0}set_cachesize 0 2097152 0
-olcDbConfig: {1}set_lk_max_objects 1500
-olcDbConfig: {2}set_lk_max_locks 1500
-olcDbConfig: {3}set_lk_max_lockers 1500
 olcDbIndex: objectClass eq
 olcDbIndex: cn,uid eq
 olcDbIndex: uidNumber,gidNumber eq
 olcDbIndex: member,memberUid eq
-structuralObjectClass: olcBdbConfig
+structuralObjectClass: olcMdbConfig
 entryUUID: 216fc25a-632c-103b-8f5e-159102fdad19
 creatorsName: cn=config
 createTimestamp: 20210616202109Z

Move away or delete the contents of /etc/ldap/slapd.d/ so that it
is an empty directory.

# mkdir /tmp/openldap-migration/old-etc-slapd
# mv /etc/ldap/slapd.d/* /tmp/openldap-migration/old-etc-slapd

Load your edited cn=config.ldif file into the cn=config
database:
```
# slapadd -F /etc/ldap/slapd.d -n0 -l /tmp/openldap-migration/cn=config.ldif
```
If the command above has failed, go back to step 2.
After the slapadd command succeeds, change the permissions on the
/etc/ldap/slapd.d/ directory to be owned by the openldap user:
```
# chown -R openldap:openldap /etc/ldap/slapd.d/
```
The file permissions should be correct, but you may want to double
check that regular files have 0600 (-rw-------) permissions,
and directories have 0750 (drwxr-x---) permissions.

Now you can proceed with reloading the remaining databases.

Migrating your databases to OpenLDAP 2.5.x

For each configured database, you should:

Locate the backup LDIF file exported by the upgrade process. This
file will probably be named like:

/var/backups/slapd-<OLDVERSION>/<SUFFIX>.ldif

where <SUFFIX> is the database suffix, such as
dc=example,dc=com, and <OLDVERSION> is the same version as the
one you used to adjust the configuration file (step 1 of the
previous section). In our case, the file is named:

/var/backups/slapd-2.4.57+dfsg-2ubuntu1/dc=example,dc=com.ldif
Ensure the directory where the database is stored (for example,
/var/lib/ldap/) is empty. By default, the upgrade process moves
away the database files to a directory named
/var/backups/<SUFFIX>-<OLDVERSION>.ldapdb, so you should not see
any files under /var/lib/ldap/. If you do, just move them away.

Reload the data using slapadd:

# slapadd -l /var/backups/slapd-2.4.57+dfsg-2ubuntu1/dc=example,dc=com.ldif

Make sure the slapadd command succeeded, and then change the
permissions on the data directory:
```
# chown -R openldap:openldap /var/lib/ldap
```

After all of your databases have been reloaded successfully, you
should be able to start the slapd service again:

# service slapd start
# service slapd status
● slapd.service - LSB: OpenLDAP standalone server (Lightweight Directory Access Protocol)
     Loaded: loaded (/etc/init.d/slapd; generated)
    Drop-In: /usr/lib/systemd/system/slapd.service.d
             └─slapd-remain-after-exit.conf
             /run/systemd/system/service.d
             └─zzz-lxc-service.conf
     Active: active (running) since Wed 2021-08-18 18:19:57 UTC; 2s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 1960 ExecStart=/etc/init.d/slapd start (code=exited, status=0/SUCCESS)
      Tasks: 3 (limit: 18735)
     Memory: 4.3M
     CGroup: /system.slice/slapd.service
             └─1966 /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -F /etc/ldap/slapd.d

Aug 18 18:19:57 openldap-test-db-migration systemd[1]: Starting LSB: OpenLDAP standalone server (Lightweight Directory Access Protocol)...
Aug 18 18:19:57 openldap-test-db-migration slapd[1960]:  * Starting OpenLDAP slapd
Aug 18 18:19:57 openldap-test-db-migration slapd[1965]: @(#) $OpenLDAP: slapd 2.5.6+dfsg-1~exp1ubuntu1 (Aug 17 2021 18:06:00) $
                                                                Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Aug 18 18:19:57 openldap-test-db-migration slapd[1966]: slapd starting
Aug 18 18:19:57 openldap-test-db-migration slapd[1960]:    ...done.
Aug 18 18:19:57 openldap-test-db-migration systemd[1]: Started LSB: OpenLDAP standalone server (Lightweight Directory Access Protocol).

More information

The example above covers a simple scenario that may not reflect what
you have installed in production environments. If you face problems
that are not covered in this guide, take a look at the slapd
package’s README.Debian file, which contains a more extensive guide for troubleshooting some
problems.

weebl · April 5, 2023, 5:02pm

You’re the best. The instructions worked flawlessly on an old install originating from Ubuntu 12.04 or earlier!

jamanning · January 10, 2024, 1:48pm

When I try to reload the data using the above command, I’m getting the following error:

Available database(s) do not allow slappadd.

I’ve been reviewing this with no success so far.